LevenshteinRatio

class opik.evaluation.metrics.LevenshteinRatio(case_sensitive: bool = False, name: str = 'levenshtein_ratio_metric', track: bool = True)

Bases: BaseMetric

A metric that calculates the Levenshtein ratio between two strings.

This metric returns a score between 0.0 and 1.0, representing the similarity between the output and reference strings. A score of 1.0 indicates identical strings, while 0.0 indicates completely different strings. The comparison can be made case-sensitive or case-insensitive.

For more information on Levenshtein distance, see: https://en.wikipedia.org/wiki/Levenshtein_distance

Parameters:

  • case_sensitive – Whether the comparison should be case-sensitive. Defaults to False.

  • name – The name of the metric. Defaults to “levenshtein_ratio_metric”.

  • track – Whether to track the metric. Defaults to True.

Example

>>> from opik.evaluation.metrics import LevenshteinRatio
>>> levenshtein_metric = LevenshteinRatio(case_sensitive=True)
>>> result = levenshtein_metric.score("Hello, World!", "Hello, World")
>>> print(result.value)
0.96
score(output: str, reference: str, **ignored_kwargs: Any) → ScoreResult

Calculate the Levenshtein ratio between the output and reference strings.

Parameters:

  • output – The output string to compare.

  • reference – The reference string to compare against.

  • **ignored_kwargs – Additional keyword arguments that are ignored.

Returns:

A ScoreResult object with a value between 0.0 and 1.0,

representing the Levenshtein ratio between the output and reference strings.

Return type:

score_result.ScoreResult

async ascore(*args: Any, **kwargs: Any) → ScoreResult | List[ScoreResult]

Async public method that can be called independently.