Groq

Groq is Fast AI Inference.

You can check out the Colab Notebook if you’d like to jump straight to the code:

Getting Started

Configuring Opik

To start tracking your Groq LLM calls, you can use our LiteLLM integration. You’ll need to have both the opik and litellm packages installed. You can install them using pip:

$ pip install opik litellm

In addition, you can configure Opik using the opik configure command which will prompt you for the correct local server address or if you are using the Cloud platform your API key:

$ opik configure

If you’re unable to use our LiteLLM integration with Groq, please open an issue

Configuring Groq

In order to configure Groq, you will need to have:

Your Groq API Key: You can create and manage your Groq API Keys on this page.

Once you have these, you can set them as environment variables:

1 import os
2 
3 os.environ["GROQ_API_KEY"] = "" # Your Google AI Studio Groq API Key

Logging LLM calls

In order to log the LLM calls to Opik, you will need to create the OpikLogger callback. Once the OpikLogger callback is created and added to LiteLLM, you can make calls to LiteLLM as you normally would:

1 from litellm.integrations.opik.opik import OpikLogger
2 import litellm
3 
4 opik_logger = OpikLogger()
5 litellm.callbacks = [opik_logger]
6 
7 response = litellm.completion(
8     model="groq/llama3-8b-8192",
9     messages=[
10         {"role": "user", "content": "Why is tracking and evaluation of LLMs important?"}
11     ]
12 )

Logging LLM calls within a tracked function

If you are using LiteLLM within a function tracked with the @track decorator, you will need to pass the current_span_data as metadata to the litellm.completion call:

1 from opik import track, opik_context
2 import litellm
3 
4 @track
5 def generate_story(prompt):
6     response = litellm.completion(
7         model="groq/llama3-8b-8192",
8         messages=[{"role": "user", "content": prompt}],
9         metadata={
10             "opik": {
11                 "current_span_data": opik_context.get_current_span_data(),
12             },
13         },
14     )
15     return response.choices[0].message.content
16 
17 
18 @track
19 def generate_topic():
20     prompt = "Generate a topic for a story about Opik."
21     response = litellm.completion(
22         model="groq/llama-3.3-70b-versatile",
23         messages=[{"role": "user", "content": prompt}],
24         metadata={
25             "opik": {
26                 "current_span_data": opik_context.get_current_span_data(),
27             },
28         },
29     )
30     return response.choices[0].message.content
31 
32 
33 @track
34 def generate_opik_story():
35     topic = generate_topic()
36     story = generate_story(topic)
37     return story
38 
39 
40 generate_opik_story()

1	import os
2
3	os.environ["GROQ_API_KEY"] = "" # Your Google AI Studio Groq API Key

1	from litellm.integrations.opik.opik import OpikLogger
2	import litellm
3
4	opik_logger = OpikLogger()
5	litellm.callbacks = [opik_logger]
6
7	response = litellm.completion(
8	model="groq/llama3-8b-8192",
9	messages=[
10	{"role": "user", "content": "Why is tracking and evaluation of LLMs important?"}
11	]
12	)

1	from opik import track, opik_context
2	import litellm
3
4	@track
5	def generate_story(prompt):
6	response = litellm.completion(
7	model="groq/llama3-8b-8192",
8	messages=[{"role": "user", "content": prompt}],
9	metadata={
10	"opik": {
11	"current_span_data": opik_context.get_current_span_data(),
12	},
13	},
14	)
15	return response.choices[0].message.content
16
17
18	@track
19	def generate_topic():
20	prompt = "Generate a topic for a story about Opik."
21	response = litellm.completion(
22	model="groq/llama-3.3-70b-versatile",
23	messages=[{"role": "user", "content": prompt}],
24	metadata={
25	"opik": {
26	"current_span_data": opik_context.get_current_span_data(),
27	},
28	},
29	)
30	return response.choices[0].message.content
31
32
33	@track
34	def generate_opik_story():
35	topic = generate_topic()
36	story = generate_story(topic)
37	return story
38
39
40	generate_opik_story()