Log Traces

LLM applications are complex systems that do more than just call an LLM API, they will often involve retrieval, pre-processing and post-processing steps. Tracing is a tool that helps you understand the flow of your application and identify specific points in your application that may be causing issues.

Opik's tracing functionality allows you to track not just all the LLM calls made by your application but also any of the other steps involved.

Tracing in Opik

Opik provides different ways to log your LLM calls and traces to the platform:

Using one of our integrations: This is the easiest way to get started.
Using the @track decorator: This allows you to track not just LLM calls but any function call in your application, it is often used in conjunction with the integrations.
Using the Python SDK: This allows for the most flexibility and customizability and is recommended if you want to have full control over the logging process.
Using the Opik REST API: If you are not using Python, you can use the REST API to log traces to the platform. The REST API is currently in beta and subject to change.

Logging with the Python SDK

In order to use the Opik Python SDK, you will need to install it and configure it:

Command Line
Jupyter Notebook

# Install the SDK
pip install opik

# Configure the SDK
opik configure

%pip install --quiet --upgrade opik

# Configure the SDK
import opik
opik.configure(use_local=False)

tip

Opik is open-source and can be hosted locally using Docker, please refer to the self-hosting guide to get started. Alternatively, you can use our hosted platform by creating an account on Comet.

Using an integration

When using one of Opik's integration you will simply need to add a couple of lines of code to your existing application to track your LLM calls and traces. There are integrations available for many of the most popular LLM frameworks and libraries.

Here is a short overview of our most popular integrations:

OpenAI
LangChain
LlamaIndex

By wrapping the OpenAI client in the track_openai function, all calls to the OpenAI API will be logged to the Opik platform:

from opik.integrations.openai import track_openai
from openai import OpenAI

client = OpenAI()
client = track_openai(client)

# Every call to the OpenAI API will be logged to the platform
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
      {"role":"user", "content": "Hello, world!"}
    ]
)

from opik.integrations.langchain import OpikTracer

# Initialize the Opik tracing callback
opik_tracer = OpikTracer()

# Create the LLM Chain using LangChain
llm = OpenAI(temperature=0)

prompt_template = PromptTemplate(
    input_variables=["input"],
    template="Translate the following text to French: {input}"
)

llm_chain = LLMChain(llm=llm, prompt=prompt_template)

# Generate the translations
translation = llm_chain.run("Hello, how are you?", callbacks=[opik_tracer])
print(translation)

from llama_index.core import Document, VectorStoreIndex
from llama_index.core import global_handler, set_global_handler

# Configure the Opik integration
set_global_handler("opik")

# Generate the response
documents = [
    Document(text="LlamaIndex is a tool for creating indices over your documents to query them using LLMs."),
    Document(text="It supports various types of indices, including vector-based indices for efficient querying."),
    Document(text="You can query the index to extract relevant information from large datasets of text.")
]

index = VectorStoreIndex(documents)
query_engine = index.as_query_engine()

query_engine.query("What is LlamaIndex used for?")

tip

If you are using a framework that Opik does not integrate with, you can raise a feature request on our Github repository.

If you are using a framework that Opik does not integrate with, we recommed you use the opik.track function decorator.

Using function decorators

Using the opik.track decorator is a great way to add Opik logging to your existing LLM application. We recommend using this method in conjuntion with one of our integrations for the most seamless experience.

When you add the @track decorator to a function, Opik will create a span for that function call and log the input parameters and function output for that function. If we detect that a decorated function is being called within another decorated function, we will create a nested span for the inner function.

Decorating your code

You can add the @track decorator to any function in your application and track not just LLM calls but also any other steps in your application:

from opik import track
import openai

client = openai.OpenAI()

@track
def retrieve_context(input_text):
    # Your retrieval logic here, here we are just returning a hardcoded list of strings
    context =[
        "What specific information are you looking for?",
        "How can I assist you with your interests today?",
        "Are there any topics you'd like to explore or learn more about?",
    ]
    return context

@track
def generate_response(input_text, context):
    full_prompt = (
        f" If the users asks a question that is not specific, use the context to provide a relevant response.\n"
        f"Context: {', '.join(context)}\n"
        f"User: {input_text}\n"
        f"AI:"
    )

    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": full_prompt}]
    )
    return response.choices[0].message.content

@track(name="my_llm_application")
def llm_chain(input_text):
    context = retrieve_context(input_text)
    response = generate_response(input_text, context)

    return response

# Use the LLM chain
result = llm_chain("Hello, how are you?")
print(result)

info

The @track decorator will only track the input and output of the decorated function. If you are using OpenAI, we recommend you also use the track_openai function to track the LLM call as well as token usage:

from opik.integrations.openai import track_openai
from openai import OpenAI

client = OpenAI()
client = track_openai(client)

Configuring the project name

You can configure the project you want the trace to be logged to using the project_name parameter of the @track decorator:

@track(project_name="my_project")
def my_function():
    # Function code
    # ...

If you want to configure this globally for all traces, you can also use the environment variable:

import os

os.environ["OPIK_PROJECT_NAME"] = "my_project"

Logging additional data

As mentioned above, the @track decorator only logs the input and output of the decorated function. If you want to log additional data, you can use the update_current_span function and update_current_trace function to manually update the span and trace:

from opik import track, opik_context

@track
def llm_chain(input_text):
    # LLM chain code
    # ...

    # Update the trace
    opik_context.update_current_trace(
        tags=["llm_chatbot"],
        feedback_scores=[
            {"name": "user_feedback", "value": 1.0, "reason": "The response was helpful and accurate."}
        ]
    )

    # Update the span
    opik_context.update_current_span(
        name="llm_chain"
    )

You can learn more about the opik_context module in the opik_context reference docs.

Disabling automatic logging

You can use the capture_input and capture_output parameters of the @track decorator to disable the automatic logging of the function input and output:

@track(capture_input=False, capture_output=False)
def llm_chain(input_text):
    # LLM chain code
    # ...

You can then use the opik_context module to manually log the trace and span attributes.

Using the low-level Opik client

If you want full control over the data logged to Opik, you can use the Opik client to log traces, spans, feedback scores and more.

Logging traces and spans

Logging traces and spans can be achieved by first creating a trace using Opik.trace and then adding spans to the trace using the Trace.span method:

from opik import Opik

client = Opik(project_name="Opik client demo")

# Create a trace
trace = client.trace(
    name="my_trace",
    input={"user_question": "Hello, how are you?"},
    output={"response": "Comment ça va?"}
)

# Add a span
trace.span(
    name="Add prompt template",
    input={"text": "Hello, how are you?", "prompt_template": "Translate the following text to French: {text}"},
    output={"text": "Translate the following text to French: hello, how are you?"}
)

# Add an LLM call
trace.span(
    name="llm_call",
    type="llm",
    input={"prompt": "Translate the following text to French: hello, how are you?"},
    output={"response": "Comment ça va?"}
)

# End the trace
trace.end()

note

It is recommended to call trace.end() and span.end() when you are finished with the trace and span to ensure that the end time is logged correctly.

Logging feedback scores

You can log scores to traces and spans using the log_traces_feedback_scores and log_spans_feedback_scores methods:

from opik import Opik

client = Opik()

trace = client.trace(name="my_trace")

client.log_traces_feedback_scores(
    scores=[
        {"id": trace.id, "name": "overall_quality", "value": 0.85, "reason": "The response was helpful and accurate."},
        {"id": trace.id, "name": "coherence", "value": 0.75}
    ]
)

span = trace.span(name="my_span")
client.log_spans_feedback_scores(
    scores=[
        {"id": span.id, "name": "overall_quality", "value": 0.85, "reason": "The response was helpful and accurate."},
        {"id": span.id, "name": "coherence", "value": 0.75}
    ]
)

tip

If you want to log scores to traces or spans from within a decorated function, you can use the update_current_trace and update_current_span methods instead.

Ensuring all traces are logged

Opik's logging functionality is designed with production environments in mind. To optimize performance, all logging operations are executed in a background thread.

If you want to ensure all traces are logged to Opik before exiting your program, you can use the opik.Opik.flush method:

from opik import Opik

client = Opik()

# Log some traces
client.flush()

Logging traces with the REST API

warning

The Opik REST API is currently in beta and subject to change, if you encounter any issues please report them to the Github.

The documentation for the Opik REST API is available here.

Logging with the Python SDK​

Using an integration​

Using function decorators​

Decorating your code​

Configuring the project name​

Logging additional data​

Disabling automatic logging​

Using the low-level Opik client​

Logging traces and spans​

Logging feedback scores​

Ensuring all traces are logged​

Logging traces with the REST API​