nltk.translate.lepor module

LEPOR score implementation.

nltk.translate.lepor.alignment(ref_tokens: List[str], hyp_tokens: List[str])[source]

This function computes the context-dependent n-gram word alignment tasks that takes into account the surrounding context (neighbouring words) of the potential word to select a better matching pairs between the output and the reference.

This alignment task is used to compute the ngram positional difference penalty component of the LEPOR score. Generally, the function finds the matching tokens between the reference and hypothesis, then find the indices of longest matching n-grams by checking the left and right unigram window of the matching tokens.

Parameters:

  • ref_tokens (List[str]) – A list of tokens in reference sentence.

  • hyp_tokens (List[str]) – A list of tokens in hypothesis sentence.

nltk.translate.lepor.corpus_lepor(references: List[List[str]], hypothesis: List[str], alpha: float = 1.0, beta: float = 1.0, tokenizer: Callable[[str], List[str]] = None) → List[List[float]][source]

Calculate LEPOR score for list of sentences from Han, A. L.-F. (2017). LEPOR: An Augmented Machine Translation Evaluation Metric. https://arxiv.org/abs/1703.08748v2

>>> hypothesis = ['a bird is on a stone.', 'scary crow was not bad.']

>>> references = [['a bird behind the stone.', 'a bird is on the rock'],
...              ['scary cow was good.', 'scary crow was elegant.']]

>>> corpus_lepor(references, hypothesis)
[[0.7824248013113159, 0.7931427828105261], [0.5639427891892225, 0.7860963170056643]]
Parameters:

  • references (list(list(str))) – Reference sentences

  • hypothesis (list(str)) – Hypothesis sentences

  • alpha (float) – A parameter to set weight fot recall.

  • beta (float) – A parameter to set weight fot precision.

  • tokenizer (Callable[[str], List[str]]) – A callable tokenizer that will accept a string and returns a list of tokens.

Returns:

The Lepor score. Returns a list for all sentences

Return type:

list(list(float))

nltk.translate.lepor.harmonic(match_count: int, reference_length: int, hypothesis_length: int, alpha: float, beta: float) → float[source]

Function will calculate the precision and recall of matched words and calculate a final score on wighting using alpha and beta parameters.

Parameters:

  • match_count (int) – Number of words in hypothesis aligned with reference.

  • reference_length (int) – Length of the reference sentence

  • hypothesis_length (int) – Length of the hypothesis sentence

  • alpha (float) – A parameter to set weight fot recall.

  • beta (float) – A parameter to set weight fot precision.

Returns:

Harmonic mean.

Return type:

float

nltk.translate.lepor.length_penalty(reference: List[str], hypothesis: List[str]) → float[source]

This function calculates the length penalty(LP) for the LEPOR metric, which is defined to embrace the penaltyvfor both longer and shorter hypothesis compared with the reference translations. Refer from Eq (2) on https://aclanthology.org/C12-2044

Parameters:

  • reference (str) – Reference sentence

  • hypothesis (str) – Hypothesis sentence

Returns:

Penalty of difference in length in reference and hypothesis sentence.

Return type:

float

nltk.translate.lepor.ngram_positional_penalty(ref_tokens: ~typing.List[str], hyp_tokens: ~typing.List[str]) -> (<class 'float'>, <class 'float'>)[source]

This function calculates the n-gram position difference penalty (NPosPenal) described in the LEPOR paper. The NPosPenal is an exponential of the length normalized n-gram matches between the reference and the hypothesis.

Parameters:

  • ref_tokens (List[str]) – A list of words in reference sentence.

  • hyp_tokens (List[str]) – A list of words in hypothesis sentence.

Returns:

A tuple containing two elements: - NPosPenal: N-gram positional penalty. - match_count: Count of matched n-grams.

Return type:

tuple

nltk.translate.lepor.sentence_lepor(references: List[str], hypothesis: str, alpha: float = 1.0, beta: float = 1.0, tokenizer: Callable[[str], List[str]] = None) → List[float][source]

Calculate LEPOR score a sentence from Han, A. L.-F. (2017). LEPOR: An Augmented Machine Translation Evaluation Metric. https://arxiv.org/abs/1703.08748v2

>>> hypothesis = 'a bird is on a stone.'

>>> reference1 = 'a bird behind the stone.'
>>> reference2 = 'a bird is on the rock.'

>>> sentence_lepor([reference1, reference2], hypothesis)
[0.7824248013113159, 0.7739937377760259]
Parameters:

  • references (list(str)) – Reference sentences

  • hypothesis (str) – Hypothesis sentence

  • alpha (float) – A parameter to set weight fot recall.

  • beta (float) – A parameter to set weight fot precision.

  • tokenizer (Callable[[str], List[str]]) – A callable tokenizer that will accept a string and returns a list of tokens.

Returns:

The list of Lepor scores for a hypothesis with all references.

Return type:

list(float)