Contains

class opik.evaluation.metrics.Contains(case_sensitive: bool = False, name: str = 'contains_metric')

Bases: BaseMetric

A metric that checks if a reference string is contained within an output string.

This metric returns a score of 1.0 if the reference string is found within the output string, and 0.0 otherwise. The comparison can be made case-sensitive or case-insensitive.

Parameters:

• case_sensitive – Whether the comparison should be case-sensitive. Defaults to False.

• name – The name of the metric. Defaults to “contains_metric”.

Example

>>> from comet_llm_eval.evaluation.metrics import Contains
>>> contains_metric = Contains(case_sensitive=True)
>>> result = contains_metric.score("Hello, World!", "World")
>>> print(result.value)
1.0
>>> result = contains_metric.score("Hello, World!", "world")
>>> print(result.value)
0.0

score(output: str, reference: str, **ignored_kwargs: Any) → ScoreResult

Calculate the score based on whether the reference string is contained in the output string.

Parameters:

• output – The output string to check.

• reference – The reference string to look for in the output.

• **ignored_kwargs – Additional keyword arguments that are ignored.

Returns:

A ScoreResult object with a value of 1.0 if the reference

is found in the output, 0.0 otherwise.

Return type:

score_result.ScoreResult

async ascore(*args: Any, **kwargs: Any) → ScoreResult | List[ScoreResult]

Async public method that can be called independently.