Log distributed traces

When working with complex LLM applications, it is common to need to track a traces across multiple services. Opik supports distributed tracing out of the box when integrating using function decorators using a mechanism that is similar to how OpenTelemetry implements distributed tracing.

For the purposes of this guide, we will assume that you have a simple LLM application that is made up of two services: a client and a server. We will assume that the client will create the trace and span, while the server will add a nested span. In order to do this, the trace_id and span_id will be passed in the headers of the request from the client to the server.

Distributed Tracing

The Python SDK includes some helper functions to make it easier to fetch headers in the client and ingest them in the server:

client.py
1from opik import track, opik_context
2
3@track()
4def my_client_function(prompt: str) -> str:
5    headers = {}
6
7    # Update the headers to include Opik Trace ID and Span ID
8    headers.update(opik_context.get_distributed_trace_headers())
9
10    # Make call to backend service
11    response = requests.post("http://.../generate_response", headers=headers, json={"prompt": prompt})
12    return response.json()

On the server side, you can pass the headers to your decorated function:

server.py
1from opik import track
2from fastapi import FastAPI, Request
3
4@track()
5def my_llm_application():
6    pass
7
8app = FastAPI()  # Or Flask, Django, or any other framework
9
10
11@app.post("/generate_response")
12def generate_llm_response(request: Request) -> str:
13    return my_llm_application(opik_distributed_trace_headers=request.headers)

The opik_distributed_trace_headers parameter is added by the track decorator to each function that is decorated and is a dictionary with the keys opik_trace_id and opik_parent_span_id.

Built with