Search & Export data¶

Comet provides an extensive REST API and a Python API object to help you search and export your Comet data. Our goal is to provide a method to perform programmatically any action you can perform in the UI as well as access any logged data.

There are two steps to using the Python API method:

Find the Experiment you wish to access using the API.query() or API.get() methods, both of these methods will return an APIExperiment().
Use the APIExperiment() methods to access the data logged to this experiment.

Get Experiments¶

Get Experiment by Key¶

If you would like to find a single Experiment and you know it's Experiment Key (it can be found in the Single Experiment Page), the easiest way to access the Experiment is using the API.get() method:

import comet_ml

# Create an experiment that we can fetch at a later date
comet_ml.login()
exp = comet_ml.start()
for i in range(100):
    exp.log_metric("accuracy", i, step=i)

exp_key = exp.get_key()
exp.end()

# Fetch the experiment
api = comet_ml.API()
api_experiment = api.get_experiment_by_key(exp_key)

# You can now access the logged data
api_experiment.get_metrics()

Get all Experiments in a project¶

To get all the Experiments in a workspace or project, you can use the API.get_experiments() method which will return a list of APIExperiment():

import comet_ml

PROJECT_NAME="test-project"
WORKSPACE_NAME="your-workspace"

# Create an experiment that we can fetch at a later date
comet_ml.login()
exp = comet_ml.start()
for i in range(100):
    exp.log_metric("accuracy", i, step=i)

exp_key = exp.get_key()
exp.end()

# Get all Experiemnts in workspace
api = comet_ml.API()

all_experiments_in_workspace = api.get_experiments(WORKSPACE_NAME)
all_experiments_in_project = api.get_experiments(WORKSPACE_NAME, PROJECT_NAME)

Search for Experiments¶

If you would like to find a set of experiments or find a specific experiments but don't have access to the Experiment Key, you can use the API.query() method:

import comet_ml
from comet_ml.query import Metric
import random

PROJECT_NAME = "test-api-query"
WORKSPACE_NAME = "your-workspace"

# Create 10 experiments that we can then search for
comet_ml.login()

for i in range(10):
    exp = comet_ml.start(project_name=PROJECT_NAME)
    for j in range(100):
        exp.log_metric("accuracy", random.random(), step=j)
    exp.end()

# Search experiments
api = comet_ml.API()
query_condition = (Metric("accuracy") > 0.5) & (Metric("accuracy") < 0.8)
matching_api_experiments = api.query(WORKSPACE_NAME, PROJECT_NAME, query_condition)

The query condition follows the format ((QUERY-VARIABLE OPERATOR VALUE) & ...) or (QUERY-VARIABLE.METHOD(VALUE) & ...) with the following possible value:

QUERY-VARIABLE: Environment(NAME), Metric(NAME), Parameter(NAME), Other(NAME), Metadata(NAME), or Tag(VALUE).
OPERATOR: ==, <=, >=, !=, < or >
METHOD: between(), contains(), startswith() or endswith
Value: Can be string, boolean, double or datetime. When using datetime, be aware that the backend is using UTC datetimes.

You can use the following query strings as inspiration for your own queries:

from comet_ml.query import (
    Environment,
    Metric,
    Parameter,
    Other,
    Metadata,
    Tag
)

# Find all experiments that have an acc metric value > .98:
api.query("workspace", "project", Metric("acc") > .98)

# Find all experiments that have a loss metric < .1 and
# a learning_rate parameter value >= 0.3:
loss = Metric("loss")
lr = Parameter("learning_rate")
query = ((loss < .1) & (lr >= 0.3))
api.query("workspace", "project", query)

# Find all of the experiments tagged "My simple tag":
tagged = Tag("My simple tag")
api.query("workspace", "project", tagged)

# Find all experiments started before Sept 24, 2019 at 5:00am:
q = Metadata("start_server_timestamp") < datetime(2019, 9, 24, 5)
api.query("workspace", "project", q)

# Find all experiments lasting more that 2 minutes (in seconds):
q = Metadata("duration") > (2 * 60)
api.query("workspace", "project", q)

Export Experiment data¶

Once you have an APIExperiment object using one of the methods described above, you can use one of the many APIExperiment methods to both access and edit the data logged to an Experiment.

The most commonly used methods include:

APIExperiment.get_parameters_summary(): Used to get a summary of the parameters logged.
APIExperiment.get_metrics_summary(): Used to get a summary of the metrics logged.
APIExperiment.get_metrics(): Used to get Experiment metrics.
APIExperiment.get_model_asset_list(): Used to get a list of the model files logged.
APIExperiment.get_asset(): Used to download an asset using the asset_id that can be retrieved using APIExperiment.get_model_asset_list() or APIExperiment.get_asset_list().

You can find a full list of available methods in the APIExperiment reference documentation.

Nov. 18, 2024