Using Opik with watsonx

Opik integrates with watsonx to provide a simple way to log traces for all watsonx LLM calls. This works for all watsonx models.

Creating an account on Comet.com

Comet provides a hosted version of the Opik platform, simply create an account and grab you API Key.

You can also run the Opik platform locally, see the installation guide for more information.

1 %pip install --upgrade opik litellm

1 import opik
2 
3 opik.configure(use_local=False)

Preparing our environment

First, we will set up our watsonx API keys. You can learn more about how to find these in the Opik watsonx integration guide.

1 import os
2 
3 os.environ["WATSONX_URL"] = ""  # (required) Base URL of your WatsonX instance
4 # (required) either one of the following:
5 os.environ["WATSONX_API_KEY"] = ""  # IBM cloud API key
6 os.environ["WATSONX_TOKEN"] = ""  # IAM auth token
7 # optional - can also be passed as params to completion() or embedding()
8 # os.environ["WATSONX_PROJECT_ID"] = "" # Project ID of your WatsonX instance
9 # os.environ["WATSONX_DEPLOYMENT_SPACE_ID"] = "" # ID of your deployment space to use deployed models

Configure LiteLLM

Add the LiteLLM OpikTracker to log traces and steps to Opik:

1 import litellm
2 import os
3 from litellm.integrations.opik.opik import OpikLogger
4 from opik import track
5 from opik.opik_context import get_current_span_data
6 
7 os.environ["OPIK_PROJECT_NAME"] = "watsonx-integration-demo"
8 opik_logger = OpikLogger()
9 litellm.callbacks = [opik_logger]

Logging traces

Now each completion will logs a separate trace to LiteLLM:

1 # litellm.set_verbose=True
2 prompt = """
3 Write a short two sentence story about Opik.
4 """
5 
6 response = litellm.completion(
7     model="watsonx/ibm/granite-13b-chat-v2",
8     messages=[{"role": "user", "content": prompt}],
9 )
10 
11 print(response.choices[0].message.content)

The prompt and response messages are automatically logged to Opik and can be viewed in the UI.

watsonx Cookbook

Using it with the `track` decorator

If you have multiple steps in your LLM pipeline, you can use the track decorator to log the traces for each step. If watsonx is called within one of these steps, the LLM call with be associated with that corresponding step:

1 @track
2 def generate_story(prompt):
3     response = litellm.completion(
4         model="watsonx/ibm/granite-13b-chat-v2",
5         messages=[{"role": "user", "content": prompt}],
6         metadata={
7             "opik": {
8                 "current_span_data": get_current_span_data(),
9             },
10         },
11     )
12     return response.choices[0].message.content
13 
14 
15 @track
16 def generate_topic():
17     prompt = "Generate a topic for a story about Opik."
18     response = litellm.completion(
19         model="watsonx/ibm/granite-13b-chat-v2",
20         messages=[{"role": "user", "content": prompt}],
21         metadata={
22             "opik": {
23                 "current_span_data": get_current_span_data(),
24             },
25         },
26     )
27     return response.choices[0].message.content
28 
29 
30 @track
31 def generate_opik_story():
32     topic = generate_topic()
33     story = generate_story(topic)
34     return story
35 
36 
37 generate_opik_story()

The trace can now be viewed in the UI: