Log multimodal traces

Opik supports multimodal traces allowing you to track not just the text input and output of your LLM, but also images.

Log a trace with an image using OpenAI SDK

Images logged to a trace in both base64 encoded images and as URLs are displayed in the trace sidebar.

We recommend that you use the track_openai wrapper to ensure the OpenAI API call is traced correctly:

1from opik.integrations.openai import track_openai
2from openai import OpenAI
3
4# Create the OpenAI client and enable Opik tracing
5client = track_openai(OpenAI())
6
7response = client.chat.completions.create(
8  model="gpt-4o-mini",
9  messages=[
10    {
11      "role": "user",
12      "content": [
13        {"type": "text", "text": "What’s in this image?"},
14        {
15          "type": "image_url",
16          "image_url": {
17            "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
18          },
19        },
20      ],
21    }
22  ],
23  max_tokens=300,
24)
25
26print(response.choices[0])

Manually logging images

If you are not using the OpenAI SDK, you can still log images to the platform. The UI will automatically detect images based on regex rules as long as the images are logged as base64 encoded images or urls ending with .png, .jpg, .jpeg, .gif, .bmp, .webp:

1{
2  "image": "<url or base64 encoded image>"
3}

Let’s us know on Github if you would like to us to support additional image formats or models.

Built with