Contains

class opik.evaluation.metrics.Contains(case_sensitive: bool = False, reference: str | None = None, name: str = 'contains_metric', track: bool = True, project_name: str | None = None)

Bases: BaseMetric

A metric that checks if a reference string is contained within an output string.

This metric returns a score of 1.0 if the reference string is found within the output string, and 0.0 otherwise. The comparison can be made case-sensitive or case-insensitive.

Parameters:

• case_sensitive – Whether the comparison should be case-sensitive. Defaults to False.

• reference – Optional default reference string. If provided, it will be used unless a reference is explicitly passed to score().

• name – The name of the metric. Defaults to “contains_metric”.

• track – Whether to track the metric. Defaults to True.

• project_name – Optional project name to track the metric in for the cases when there are no parent span/trace to inherit project name from.

Examples

>>> # Using a default reference at initialization
>>> contains_metric = Contains(reference="world")
>>> result = contains_metric.score("Hello, World!")
>>> print(result.value)
1.0

>>> # Overriding the default reference at score time
>>> result = contains_metric.score("Hello, World!", reference="there")
>>> print(result.value)
0.0

>>> # If no reference is set at all, score() raises an error
>>> contains_metric = Contains()
>>> contains_metric.score("Hello")
Traceback (most recent call last):
    ...
ValueError: No reference string provided. Either pass `reference` to `score()` or set a default reference when creating the metric.

>>> # Empty reference string is invalid
>>> contains_metric = Contains(reference="")
>>> contains_metric.score("Hello")
Traceback (most recent call last):
    ...
ValueError: Invalid reference string provided. Reference must be a non-empty string.

score(output: str, reference: str | None = None, **ignored_kwargs: Any) → ScoreResult

Calculate the score based on whether the reference string is contained in the output string.

Parameters:

• output – The output string to check.

• reference – The reference string to look for in the output. If None, falls back to the default reference provided at initialization.

• **ignored_kwargs – Additional keyword arguments that are ignored.

Returns:

A ScoreResult object with a value of 1.0 if the reference

is found in the output, 0.0 otherwise.

Return type:

score_result.ScoreResult

async ascore(*args: Any, **kwargs: Any) → ScoreResult | List[ScoreResult]

Async public method that can be called independently.