Prompt management

Opik provides a prompt library that you can use to manage your prompts. Storing prompts in a library allows you to version them, reuse them across projects, and manage them in a central location.

Using a prompt library does not mean you can’t store your prompt in code, we have designed the prompt library to be work seamlessly with your existing prompt files while providing the benefits of a central prompt library.

Managing prompts stored in code

The recommend way to create and manage prompts is using The Prompt object. This will allow you to continue versioning your prompts in code while also getting the benefit of having prompt versions managed in the Opik platform so you can more easily keep track of your progress.

1
import opik
2
3
# Prompt text stored in a variable
4
PROMPT_TEXT = "Write a summary of the following text: {{text}}"
5
6
# Create a prompt
7
prompt = opik.Prompt(
8
    name="prompt-summary",
9
    prompt=PROMPT_TEXT,
10
)
11
12
# Print the prompt text
13
print(prompt.prompt)
14
15
# Build the prompt
16
print(prompt.format(text="Hello, world!"))

The prompt will now be stored in the library and versioned:

Using the low level SDK

If you would rather keep prompts in the Opik platform and manually update / download them, you can use the low-level Python SDK to manage you prompts.

Creating prompts

You can create a new prompt in the library using both the SDK and the UI:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a new prompt
7
prompt = client.create_prompt(name="prompt-summary", prompt="Write a summary of the following text: {{text}}")

Downloading your prompts

Once a prompt is created in the library, you can download it in code using the Opik.get_prompt method:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Get the prompt
7
prompt = client.get_prompt(name="prompt-summary")
8
9
# Create the prompt message
10
prompt.format(text="Hello, world!")

If you are not using the SDK, you can download a prompt by using the REST API.

Linking prompts to Experiments

Experiments allow you to evaluate the performance of your LLM application on a set of examples. When evaluating different prompts, it can be useful to link the evaluation to a specific prompt version. This can be achieved by passing the prompt parameter when creating an Experiment:

1
import opik
2
3
opik.configure()
4
client = opik.Opik()
5
6
# Create a prompt
7
prompt = opik.Prompt(name="My prompt", prompt="...")
8
9
# Run the evaluation
10
evaluation = evaluate(
11
    experiment_name="My experiment",
12
    dataset=dataset,
13
    task=evaluation_task,
14
    scoring_metrics=[hallucination_metric],
15
    prompt=prompt,
16
)

The experiment will now be linked to the prompt allowing you to view all experiments that use a specific prompt: