Using Opik with Instructor

Instructor is a Python library for working with structured outputs for LLMs built on top of Pydantic. It provides a simple way to manage schema validations, retries and streaming responses.

Creating an account on Comet.com

Comet provides a hosted version of the Opik platform, simply create an account and grab you API Key.

You can also run the Opik platform locally, see the installation guide for more information.

1 %pip install --upgrade --quiet opik instructor

1 import opik
2 
3 opik.configure(use_local=False)

For this demo we are going to use an OpenAI so we will need to configure our OpenAI API key:

1 import os
2 import getpass
3 
4 if "OPENAI_API_KEY" not in os.environ:
5     os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")

Using Opik with Instructor library

In order to log traces from Instructor into Opik, we are going to patch the instructor library. This will log each LLM call to the Opik platform.

For all the integrations, we will first add tracking to the LLM client and then pass it to the Instructor library:

1 from opik.integrations.openai import track_openai
2 import instructor
3 from pydantic import BaseModel
4 from openai import OpenAI
5 
6 
7 # We will first create the OpenAI client and add the `track_openai`
8 # method to log data to Opik
9 openai_client = track_openai(OpenAI())
10 
11 # Patch the OpenAI client for Instructor
12 client = instructor.from_openai(openai_client)
13 
14 
15 # Define your desired output structure
16 class UserInfo(BaseModel):
17     name: str
18     age: int
19 
20 
21 user_info = client.chat.completions.create(
22     model="gpt-4o-mini",
23     response_model=UserInfo,
24     messages=[{"role": "user", "content": "John Doe is 30 years old."}],
25 )
26 
27 print(user_info)

Thanks to the track_openai method, all the calls made to OpenAI will be logged to the Opik platform. This approach also works well if you are also using the opik.track decorator as it will automatically log the LLM call made with Instructor to the relevant trace.

Trace screenshot

Integrating with other LLM providers

The instructor library supports many LLM providers including OpenAI, Bedrock, Gemini, etc. Opik supports the majority of these.

Here are some code snippets needed for the integration.

OpenAI

1 from opik.integrations.openai import track_openai
2 import instructor
3 from openai import OpenAI
4 
5 # Add Opik tracking
6 openai_client = track_openai(OpenAI())
7 
8 # Patch the OpenAI client for Instructor
9 client = instructor.from_openai(openai_client)

Anthropic

1 from opik.integrations.anthropic import track_anthropic
2 import instructor
3 from anthropic import Anthropic
4 
5 # Add Opik tracking
6 anthropic_client = track_anthropic(Anthropic())
7 
8 # Patch the Anthropic client for Instructor
9 client = instructor.from_openai(anthropic_client)

Gemini

1 from opik.integrations.genai import track_genai
2 import instructor
3 import google.generativeai as genai
4 
5 # Add Opik tracking
6 genai_client = track_genai(
7     genai.GenerativeModel(
8         model_name="models/gemini-1.5-flash-latest",
9     )
10 )
11 
12 # Patch the GenAI client for Instructor
13 client = instructor.from_openai(genai_client, mode=instructor.Mode.GEMINI_JSON)

1	import os
2	import getpass
3
4	if "OPENAI_API_KEY" not in os.environ:
5	os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")

1	from opik.integrations.openai import track_openai
2	import instructor
3	from pydantic import BaseModel
4	from openai import OpenAI
5
6
7	# We will first create the OpenAI client and add the `track_openai`
8	# method to log data to Opik
9	openai_client = track_openai(OpenAI())
10
11	# Patch the OpenAI client for Instructor
12	client = instructor.from_openai(openai_client)
13
14
15	# Define your desired output structure
16	class UserInfo(BaseModel):
17	name: str
18	age: int
19
20
21	user_info = client.chat.completions.create(
22	model="gpt-4o-mini",
23	response_model=UserInfo,
24	messages=[{"role": "user", "content": "John Doe is 30 years old."}],
25	)
26
27	print(user_info)

1	from opik.integrations.anthropic import track_anthropic
2	import instructor
3	from anthropic import Anthropic
4
5	# Add Opik tracking
6	anthropic_client = track_anthropic(Anthropic())
7
8	# Patch the Anthropic client for Instructor
9	client = instructor.from_openai(anthropic_client)

1	from opik.integrations.genai import track_genai
2	import instructor
3	import google.generativeai as genai
4
5	# Add Opik tracking
6	genai_client = track_genai(
7	genai.GenerativeModel(
8	model_name="models/gemini-1.5-flash-latest",
9	)
10	)
11
12	# Patch the GenAI client for Instructor
13	client = instructor.from_openai(genai_client, mode=instructor.Mode.GEMINI_JSON)