Log traces

LLM applications are complex systems that do more than just call an LLM API, they will often involve retrieval, pre-processing and post-processing steps. Tracing is a tool that helps you understand the flow of your application and identify specific points in your application that may be causing issues.

Opik’s tracing functionality allows you to track not just all the LLM calls made by your application but also any of the other steps involved.

Opik provides different ways to log your LLM calls and traces to the platform:

1. Using one of our integrations: This is the easiest way to get started.
2. Using the @track decorator: This allows you to track not just LLM calls but any function call in your application, it is often used in conjunction with the integrations.
3. Using the Python SDK: This allows for the most flexibility and customizability and is recommended if you want to have full control over the logging process.
4. Using the Opik REST API: If you are not using Python, you can use the REST API to log traces to the platform. The REST API is currently in beta and subject to change.

Logging with the Python SDK

In order to use the Opik Python SDK, you will need to install it and configure it:

$
# Install the SDK
>
pip install opik
>
>
# Configure the SDK
>
opik configure

Using an integration

When using one of Opik’s integration you will simply need to add a couple of lines of code to your existing application to track your LLM calls and traces. There are integrations available for many of the most popular LLM frameworks and libraries.

Here is a short overview of our most popular integrations:

$
pip install opik openai

1
from opik.integrations.openai import track_openai
2
from openai import OpenAI
3
4
client = OpenAI()
5
client = track_openai(client)
6
7
# Every call to the OpenAI API will be logged to the platform
8
response = client.chat.completions.create(
9
    model="gpt-3.5-turbo",
10
    messages=[
11
      {"role":"user", "content": "Hello, world!"}
12
    ]
13
)

If you are using a framework that Opik does not integrate with, we recommed you use the opik.track function decorator.

Using function decorators

Using the opik.track decorator is a great way to add Opik logging to your existing LLM application. We recommend using this method in conjunction with one of our integrations for the most seamless experience.

When you add the @track decorator to a function, Opik will create a span for that function call and log the input parameters and function output for that function. If we detect that a decorated function is being called within another decorated function, we will create a nested span for the inner function.

Decorating your code

You can add the @track decorator to any function in your application and track not just LLM calls but also any other steps in your application:

1
import opik
2
import openai
3
4
client = openai.OpenAI()
5
6
@opik.track
7
def retrieve_context(input_text):
8
    # Your retrieval logic here, here we are just returning a hardcoded list of strings
9
    context =[
10
        "What specific information are you looking for?",
11
        "How can I assist you with your interests today?",
12
        "Are there any topics you'd like to explore or learn more about?",
13
    ]
14
    return context
15
16
@opik.track
17
def generate_response(input_text, context):
18
    full_prompt = (
19
        f" If the user asks a question that is not specific, use the context to provide a relevant response.\n"
20
        f"Context: {', '.join(context)}\n"
21
        f"User: {input_text}\n"
22
        f"AI:"
23
    )
24
25
    response = client.chat.completions.create(
26
        model="gpt-3.5-turbo",
27
        messages=[{"role": "user", "content": full_prompt}]
28
    )
29
    return response.choices[0].message.content
30
31
@opik.track(name="my_llm_application")
32
def llm_chain(input_text):
33
    context = retrieve_context(input_text)
34
    response = generate_response(input_text, context)
35
36
    return response
37
38
# Use the LLM chain
39
result = llm_chain("Hello, how are you?")
40
print(result)

1
from opik.integrations.openai import track_openai
2
from openai import OpenAI
3
4
client = OpenAI()
5
client = track_openai(client)

Scoring traces

You can log feedback scores for traces using the opik_context.update_current_trace function. This can be useful if there are some metrics that are already reported as part of your chain or agent:

1
from opik import track, opik_context
2
3
@track
4
def llm_chain(input_text):
5
    # LLM chain code
6
    # ...
7
8
    # Update the trace
9
    opik_context.update_current_trace(
10
        feedback_scores=[
11
            {"name": "user_feedback", "value": 1.0, "reason": "The response was helpful and accurate."}
12
        ]
13
    )

Logging additional data

As mentioned above, the @track decorator only logs the input and output of the decorated function. If you want to log additional data, you can use the update_current_span function and update_current_trace function to manually update the span and trace:

1
from opik import track, opik_context
2
3
@track
4
def llm_chain(input_text):
5
    # LLM chain code
6
    # ...
7
8
    # Update the trace
9
    opik_context.update_current_trace(
10
        tags=["llm_chatbot"],
11
    )
12
13
    # Update the span
14
    opik_context.update_current_span(
15
        name="llm_chain"
16
    )

You can learn more about the opik_context module in the opik_context reference docs.

Configuring the project name

You can configure the project you want the trace to be logged to using the project_name parameter of the @track decorator:

1
import opik
2
3
@opik.track(project_name="my_project")
4
def my_function(input):
5
    # Function code
6
    return input

If you want to configure this globally for all traces, you can also use the environment variable:

1
import os
2
3
os.environ["OPIK_PROJECT_NAME"] = "my_project"

This will block the processing until the data is finished being logged.

Flushing the trace

You can ensure all data is logged by setting the flush parameter of the @track decorator to True:

1
import opik
2
3
@opik.track(flush=True)
4
def my_function(input):
5
    # Function code
6
    return input

Disabling automatic logging of function input and output

You can use the capture_input and capture_output parameters of the @track decorator to disable the automatic logging of the function input and output:

1
import opik
2
3
@opik.track(capture_input=False, capture_output=False)
4
def llm_chain(input_text):
5
    # LLM chain code
6
    return input_text

You can then use the opik_context module to manually log the trace and span attributes.

Disable all tracing

You can disable the logging of traces and spans using the enviornment variable OPIK_TRACK_DISABLE, this will turn off the logging for all function decorators:

1
import os
2
3
os.environ["OPIK_TRACK_DISABLE"] = "true"

Using the low-level Opik client

If you want full control over the data logged to Opik, you can use the Opik client to log traces, spans, feedback scores and more.

Logging traces and spans

Logging traces and spans can be achieved by first creating a trace using Opik.trace and then adding spans to the trace using the Trace.span method:

1
from opik import Opik
2
3
client = Opik(project_name="Opik client demo")
4
5
# Create a trace
6
trace = client.trace(
7
    name="my_trace",
8
    input={"user_question": "Hello, how are you?"},
9
    output={"response": "Comment ça va?"}
10
)
11
12
# Add a span
13
trace.span(
14
    name="Add prompt template",
15
    input={"text": "Hello, how are you?", "prompt_template": "Translate the following text to French: {text}"},
16
    output={"text": "Translate the following text to French: hello, how are you?"}
17
)
18
19
# Add an LLM call
20
trace.span(
21
    name="llm_call",
22
    type="llm",
23
    input={"prompt": "Translate the following text to French: hello, how are you?"},
24
    output={"response": "Comment ça va?"}
25
)
26
27
# End the trace
28
trace.end()

Logging feedback scores

You can log scores to traces and spans using the log_traces_feedback_scores and log_spans_feedback_scores methods:

1
from opik import Opik
2
3
client = Opik()
4
5
trace = client.trace(name="my_trace")
6
7
client.log_traces_feedback_scores(
8
    scores=[
9
        {"id": trace.id, "name": "overall_quality", "value": 0.85, "reason": "The response was helpful and accurate."},
10
        {"id": trace.id, "name": "coherence", "value": 0.75}
11
    ]
12
)
13
14
span = trace.span(name="my_span")
15
client.log_spans_feedback_scores(
16
    scores=[
17
        {"id": span.id, "name": "overall_quality", "value": 0.85, "reason": "The response was helpful and accurate."},
18
        {"id": span.id, "name": "coherence", "value": 0.75}
19
    ]
20
)

Ensuring all traces are logged

Opik’s logging functionality is designed with production environments in mind. To optimize performance, all logging operations are executed in a background thread.

If you want to ensure all traces are logged to Opik before exiting your program, you can use the opik.Opik.flush method:

1
from opik import Opik
2
3
client = Opik()
4
5
# Log some traces
6
client.flush()

Copying traces to a new project

You can copy traces between projects using the copy_traces method. This method allows you to move traces from one project to another without having to re-log them.

1
from opik import Opik
2
3
client = Opik()
4
5
client.copy_traces(
6
    project_name="<name of the project where the traces were created>",
7
    destination_project_name="<name of the new project>"
8
)

Logging with the JS / TS SDK

You can log your LLM calls using the Opik typescript SDK opik. We are actively adding functionality to the TypeScript SDK, if you have any suggestions on how we can improve it feel free to open an issue on GitHub.

You can find the reference documentation for the opik typescript SDK here.

Using the low-level Opik client

The easiest way to log your LLM calls is using the low-level Opik client. We do have support for decorators but this is currently considered experimental.

Setting up the Opik client

The first step is to install the Opik library:

$
npm install opik

Once the library is installed, you can initialize the Opik client with explicit configuration:

1
import { Opik } from "opik";
2
3
// Create a new Opik client with your configuration
4
const client = new Opik({
5
  apiKey: "<your-api-key>",
6
  apiUrl: "https://www.comet.com/opik/api",
7
  projectName: "<your-project-name>",
8
  workspaceName: "<your-workspace-name>",
9
});

Or using environment variables instead:

$
OPIK_API_KEY="<your-api-key>"
>
OPIK_URL_OVERRIDE=https://www.comet.com/opik/api # in case you are using the Cloud version
>
OPIK_PROJECT_NAME="<your-project-name>"
>
OPIK_WORKSPACE="<your-workspace-name>"

1
import { Opik } from "opik";
2
3
const client = new Opik();

Logging traces and spans

Once the Opik client is set up, you can log your LLM calls by adding spans to the trace:

1
// Log a trace with an LLM span
2
const trace = client.trace({
3
  name: `Trace`,
4
  input: {
5
    prompt: `Hello!`,
6
  },
7
  output: {
8
    response: `Hello, world!`,
9
  },
10
});
11
12
const span = trace.span({
13
  name: `Span`,
14
  type: "llm",
15
  input: {
16
    prompt: `Hello, world!`,
17
  },
18
  output: {
19
    response: `Hello, world!`,
20
  },
21
});
22
23
// Flush the client to send all traces and spans
24
await client.flush();

Decorators

1
import { track } from "opik";
2
3
const generateText = track({ name: "generateText", type: "llm" }, async () => {
4
  return "Generate text";
5
});
6
7
const translate = track({ name: "translate" }, async (text: string) => {
8
  return `Translated: ${text}`;
9
});
10
11
const process = track(
12
  { name: "process", projectName: "translation-service" },
13
  async () => {
14
    const text = await generateText();
15
    return translate(text);
16
  }
17
);

On calling process() it’ll create a trace and create all the spans for each tracked function called in this function

Class method decorators (TypeScript)

TypeScript started supporting decorators from version 5 but it’s use is still not widespread. The Opik typescript SDK also supports decorators but it’s currently considered experimental.

1
import { track } from "opik";
2
3
class TranslationService {
4
  @track({ type: "llm" })
5
  async generateText() {
6
    // Your LLM call here
7
    return "Generated text";
8
  }
9
10
  @track({ name: "translate" })
11
  async translate(text: string) {
12
    // Your translation logic here
13
    return `Translated: ${text}`;
14
  }
15
16
  @track({ name: "process", projectName: "translation-service" })
17
  async process() {
18
    const text = await this.generateText();
19
    return this.translate(text);
20
  }
21
}

Using the REST API

The documentation for the Opik REST API is available here.