Manage datasets

Datasets can be used to track test cases you would like to evaluate your LLM on. Each dataset is made up of dictionary with any key value pairs. When getting started, we recommend having an input and optional expected_output fields for example. These datasets can be created from:

• Python SDK: You can use the Python SDK to create an dataset and add items to it.
• Traces table: You can add existing logged traces (from a production application for example) to a dataset.
• The Opik UI: You can manually create a dataset and add items to it.

Once a dataset has been created, you can run Experiments on it. Each Experiment will evaluate an LLM application based on the test cases in the dataset using an evaluation metric and report the results back to the dataset.

Creating a dataset using the SDK

You can create a dataset and log items to it using the get_or_create_dataset method:

1
from opik import Opik
2
3
# Create a dataset
4
client = Opik()
5
dataset = client.get_or_create_dataset(name="My dataset")

If a dataset with the given name already exists, the existing dataset will be returned.

Insert items

Inserting dictionary items

You can insert items to a dataset using the insert method:

1
from opik import Opik
2
3
# Get or create a dataset
4
client = Opik()
5
dataset = client.get_or_create_dataset(name="My dataset")
6
7
# Add dataset items to it
8
dataset.insert([
9
    {"user_question": "Hello, world!", "expected_output": {"assistant_answer": "Hello, world!"}},
10
    {"user_question": "What is the capital of France?", "expected_output": {"assistant_answer": "Paris"}},
11
])

Once the items have been inserted, you can view them them in the Opik UI:

Inserting items from a JSONL file

You can also insert items from a JSONL file:

1
dataset.read_jsonl_from_file("path/to/file.jsonl")

The format of the JSONL file should be a JSON object per line. For example:

{"user_question": "Hello, world!"}
{"user_question": "What is the capital of France?", "expected_output": {"assistant_answer": "Paris"}}

Inserting items from a Pandas DataFrame

You can also insert items from a Pandas DataFrame:

1
dataset.insert_from_pandas(dataframe=df)

The keys_mapping parameter maps the column names in the DataFrame to the keys in the dataset items, this can be useful if you want to rename columns before inserting them into the dataset:

1
dataset.insert_from_pandas(dataframe=df, keys_mapping={"Expected output": "expected_output"})

Deleting items

You can delete items in a dataset by using the delete method:

1
from opik import Opik
2
3
# Get or create a dataset
4
client = Opik()
5
dataset = client.get_dataset(name="My dataset")
6
7
dataset.delete(items_ids=["123", "456"])

1
from opik import Opik
2
3
# Get or create a dataset
4
client = Opik()
5
dataset = client.get_dataset(name="My dataset")
6
7
dataset.clear()

Downloading a dataset from Opik

You can download a dataset from Opik using the get_dataset method:

1
from opik import Opik
2
3
client = Opik()
4
dataset = client.get_dataset(name="My dataset")

Once the dataset has been retrieved, you can access it’s items using the to_pandas() or to_json methods:

1
from opik import Opik
2
3
client = Opik()
4
dataset = client.get_dataset(name="My dataset")
5
6
# Convert to a Pandas DataFrame
7
dataset.to_pandas()
8
9
# Convert to a JSON array
10
dataset.to_json()