Run an Experiment in Jupyter Notebooks¶

Comet integrates seamlessly with hosted Jupyter Notebook services, including Google's Colab, Microsoft's Azure Notebooks, Amazon SageMaker Studio Lab, Amazon SageMaker notebook instance, and others.

Start logging data to Comet¶

You can pip install and configure the comet_ml library from any notebook using:

1
2
3
4

%pip install comet_ml --quiet

import comet_ml
comet_ml.login()

You can then create a new experiment using the Experiment class which will start the Comet auto-loggers and allow you to log custom metrics:

1
2
3
4
5
6
7
8

exp = comet_ml.start()

exp.log_metrics({"loss": 0.002, "batch_loss": 0.0019})

exp.log_parameters({"batch_size": 256, "momentum": 0.95})

# Once training is complete, you can mark the experiment as finished using
exp.end()

Please refer to Logging Experiment data for information on how to log different data types, integrations for an overview of the frameworks that Comet integrates with and corresponding functionalities, and Python SDK for a complete reference to all the methods provided by comet_ml.

Display the Comet UI directly in a Notebook¶

Another key feature of Comet in Notebooks is the ability to display the Comet UI within a cell using the Experiment.display() method:

1
2
3
4
5
6

import comet_ml

comet_ml.login()
exp = comet_ml.start()

exp.display(tab='panels')

The following tab names are available:

Managing Comet API Key in Google Colab¶

When using Comet in Google Colab, you can securely store your API key using Colab's built-in secrets management system. This feature ensures your API key remains private even when sharing notebooks.

Setting up the Secret¶

1. In your Google Colab notebook, click on the "🔑" icon in the left sidebar to open the "Secrets" panel
2. Click on "Add new secret"
3. Set the following:
   • Name: COMET_API_KEY
   • Value: Your Comet API key
4. Click "Add new secret"

Using the Secret in Your Code¶

You can access your API key in your notebook using the userdata module. Here's how to use it with Comet:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

from google.colab import userdata
import comet_ml

# Get the API key from Colab secrets
api_key = userdata.get("COMET_API_KEY")

# Initialize Comet with the API key
comet_ml.login(api_key=api_key)

# Start an experiment
experiment = comet_ml.start(project_name="my-project")

Alternatively, you can pass the API key directly to comet_ml.start:

1
2
3
4
5
6
7
8

from google.colab import userdata
import comet_ml

# Get the API key from Colab secrets and start an experiment
experiment = comet_ml.start(
    api_key=userdata.get("COMET_API_KEY"),
    project_name="my-project"
)

End an Experiment¶

When running Comet in a Notebook, we must end your Experiment explicitly when the training is finished using the comet.end() method:

1

comet.end()

When as experiment is ended, Comet will finish uploading all remaining data as well as all the code cells that where executed before returning.