Using Opik with LlamaIndex

This notebook showcases how to use Opik with LlamaIndex. LlamaIndex is a flexible data framework for building LLM applications:

LlamaIndex is a “data framework” to help you build LLM apps. It provides the following tools:

• Offers data connectors to ingest your existing data sources and data formats (APIs, PDFs, docs, SQL, etc.).
• Provides ways to structure your data (indices, graphs) so that this data can be easily used with LLMs.
• Provides an advanced retrieval/query interface over your data: Feed in any LLM input prompt, get back retrieved context and knowledge-augmented output.
• Allows easy integrations with your outer application framework (e.g. with LangChain, Flask, Docker, ChatGPT, anything else).

For this guide we will be downloading the essays from Paul Graham and use them as our data source. We will then start querying these essays with LlamaIndex.

Creating an account on Comet.com

Comet provides a hosted version of the Opik platform, simply create an account and grab you API Key.

You can also run the Opik platform locally, see the installation guide for more information.

1
%pip install opik llama-index llama-index-agent-openai llama-index-llms-openai --upgrade --quiet

1
import opik
2
3
opik.configure(use_local=False)

Preparing our environment

First, we will download the Chinook database and set up our different API keys.

And configure the required environment variables:

1
import os
2
import getpass
3
4
if "OPENAI_API_KEY" not in os.environ:
5
    os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")

In addition, we will download the Paul Graham essays:

1
import os
2
import requests
3
4
# Create directory if it doesn't exist
5
os.makedirs("./data/paul_graham/", exist_ok=True)
6
7
# Download the file using requests
8
url = "https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt"
9
response = requests.get(url)
10
with open("./data/paul_graham/paul_graham_essay.txt", "wb") as f:
11
    f.write(response.content)

Using LlamaIndex

Configuring the Opik integration

You can use the Opik callback directly by calling:

1
from llama_index.core import Settings
2
from llama_index.core.callbacks import CallbackManager
3
from opik.integrations.llama_index import LlamaIndexCallbackHandler
4
5
opik_callback_handler = LlamaIndexCallbackHandler()
6
Settings.callback_manager = CallbackManager([opik_callback_handler])

Now that the callback handler is configured, all traces will automatically be logged to Opik.

Using LLamaIndex

The first step is to load the data into LlamaIndex. We will use the SimpleDirectoryReader to load the data from the data/paul_graham directory. We will also create the vector store to index all the loaded documents.

1
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
2
3
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
4
index = VectorStoreIndex.from_documents(documents)
5
query_engine = index.as_query_engine()

We can now query the index using the query_engine object:

1
response = query_engine.query("What did the author do growing up?")
2
print(response)

You can now go to the Opik app to see the trace: