comet_ml.API ¶

The API class is used as a Python interface to the Comet.ml Python API.

You can use an instance of the API() class to quickly and easily access all of your logged information at comet, including metrics, parameters, tags, and assets.

Example calls to get workspace, project, and experiment data:

API.get(): gets all of your personal workspaces
API.get(WORKSPACE): gets all of your projects from WORKSPACE
API.get(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
API.get_experiment(WORKSPACE, PROJECT_NAME, EXPERIMENT_KEY): get an APIExperiment
API.get_experiment("WORKSPACE/PROJECT_NAME/EXPERIMENT_KEY"): get an APIExperiment
API.get_experiments(WORKSPACE): get all APIExperiments in WORKSPACE
API.get_experiments(WORKSPACE, PROJECT_NAME): get all APIExperiments in WORKSPACE/PROJECT
API.get_experiments(WORKSPACE, PROJECT_NAME, PATTERN): get all APIExperiments in WORKSPACE/PROJECT/PATTERN

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()

## Return all of my workspace names in a list:
api.get()

## Get an APIExperiment:
experiment = api.get("cometpublic/comet-notebooks/example 001")

## Get metrics:
experiment.get_metrics("train_accuracy")

The API instance also gives you access to the low-level Python API function calls:

1	`api.delete_experiment(experiment_key)`

For more usage examples, see Comet Python API examples.

Functions¶

init ¶

__init__(api_key=None, cache=True, version='v2')

Application Programming Interface to the Comet Python interface.

Parameters:

api_key (str, default: None ) –
Your private COMET_API_KEY.
cache (bool, default: True ) –
Whether to cache on values or not.
version (str, default: 'v2' ) –
The version of the REST API to use.

Note

api_key may be defined in environment (COMET_API_KEY) or in a .comet.config file.

Example

import comet_ml
from comet_ml.api import API

comet_ml.login()
api = API()
api.get("my-workspace")

add_registry_model_version_stage ¶

add_registry_model_version_stage(workspace, registry_name, version, stage)

Adds a stage to a registered model version.

This method has been deprecated as stage has been replaced by status and a new comet_ml.Model object was introduced. We recommend using the comet_ml.API.get_model method to get the Model object and then using model.set_status(version='<version>', status='<status>')

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.
version (str) –
The version of model to update.
stage (str) –
"production", or "staging", etc.

archive_experiment ¶

archive_experiment(experiment_key)

Archive one experiment.

Parameters:

experiment_key (str) –
the experiment key to archive

archive_experiments ¶

archive_experiments(experiment_keys)

Archive list of experiments.

Parameters:

experiment_keys (list) –
the experiment keys to archive

clear_cache ¶

clear_cache()

Used when cache is on, but you have added/changed data outside of this API instance.

Note

You could also just start with no cache.

Example

import comet_ml

comet_ml.login()
api = comet_ml.API(cache=False)

Or, if you had started with cache, turn it off:

>>> api = API(cache=True)
>>> api.do_cache(False)

create_project ¶

create_project(workspace, project_name, project_description=None, public=False)

Create a project.

Parameters:

project_name (str) –
The project name.
project_description (str, default: None ) –
The description for this project.
public (bool, default: False ) –
If True the project will be public.

create_project_share_key ¶

create_project_share_key(project_id)

Get the share keys for a private project ID.

Parameters:

project_id (str) –
The ID of the project

Example

import comet_ml

comet_ml.login()
api = comet_mlAPI()
SHARE_KEY = api.create_project_share_key(PROJECT_ID)

See also: API.get_project_share_keys(), and API.delete_project_share_key().

delete_experiment ¶

delete_experiment(experiment_key)

Delete one experiment.

Parameters:

experiment_key (str) –
The experiment key for the experiment to delete.

delete_experiments ¶

delete_experiments(experiment_keys)

Delete list of experiments.

Parameters:

experiment_keys (list) –
a list of experiment keys to delete.

delete_project ¶

delete_project(
    workspace=None, project_name=None, project_id=None, delete_experiments=False
)

Delete a project.

Parameters:

workspace (str, default: None ) –
the name of the workspace (required if project_id not given)
project_name (str, default: None ) –
the name of the project (required if project_id not given)
project_id (str, default: None ) –
the project id (required, if workspace and project name not given)
delete_experiments (bool, default: False ) –
if True, delete all of the experiments, too

delete_project_share_key ¶

delete_project_share_key(project_id, share_key)

Delete a share key for a private project ID.

Parameters:

project_id (str) –
The ID of the project
share_key (str) –
The share key to delete

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()
SHARE_KEYS = api.get_project_share_keys(PROJECT_ID)

api.delete_project_share_key(PROJECT_ID, SHARE_KEYS[0])

See also: API.get_project_share_keys(), and API.create_project_share_key().

delete_registry_model ¶

delete_registry_model(workspace, registry_name)

Deletes a registered model.

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.

delete_registry_model_version ¶

delete_registry_model_version(workspace, registry_name, version)

Deletes a registered model version.

Parameters:

workspace (str) –
The name of the workspace
registry_name (str) –
The name of the model
version (str) –
The version of model to update

delete_registry_model_version_stage ¶

delete_registry_model_version_stage(workspace, registry_name, version, stage)

Removes a stage from a registered model version.

This method has been deprecated as stage has been replaced by status and a new comet_ml.Model object was introduced. We recommend using the comet_ml.API.get_model method to get the Model object and then using model.set_status(version='<version>', status='<status>')

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.
version (str) –
The version of model to update.
stage (str) –
"production", or "staging", etc.

do_cache ¶

do_cache(*endpoints)

Cache the given endpoints.

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()
api.do_cache("experiments", "projects")

do_not_cache ¶

do_not_cache(*endpoints)

Do not cache the given endpoints.

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()

api.do_not_cache("experiments", "projects")

download_experiment_asset ¶

download_experiment_asset(
    experiment_key: str, asset_id: str, output_path: str
) -> None

Download an experiment (or a model registry) asset to the specified output_path.

Parameters:

experiment_key (str) –
The experiment unique key to download from.
asset_id (str) –
The asset ID.
output_path (str) –
Where to download the asset.

Raises:

CometRestApiException –
if the asset or experiment_key is not found.
OSError –
if the asset cannot be written to the output_path.

download_registry_model ¶

download_registry_model(
    workspace,
    registry_name,
    version=None,
    output_path="./",
    expand=True,
    stage=None,
)

Download and save all files from the registered model.

This method has been deprecated as stage has been replaced by status and a new comet_ml.Model object was introduced. We recommend using the comet_ml.API.get_model method to get the Model object and then using comet_ml.Model.download

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.
version (str, default: None ) –
The version string of the model.
output_path (str, default: './' ) –
The output directory
expand (bool, default: True ) –
If True, the downloaded zipfile is unzipped; if False, then the zipfile is copied to the output_path.
stage (str, default: None ) –
A textual tag such as "production" or "staging".

gen_experiments ¶

gen_experiments(workspace, project_name=None, pattern=None)

Get APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.

get ¶

get(
    workspace: Optional[str] = None,
    project_name: Optional[str] = None,
    experiment: Optional[str] = None,
)

Get the following items:

list of workspace names, given no arguments
list of project names, given a workspace name
list of experiment names/keys, given workspace and project names
an experiment, given workspace, project, and experiment name/key

Parameters:

workspace (str, default: None ) –
Workspace name.
project_name (str, default: None ) –
Project name.
experiment (str, default: None ) –
Experiment key.

Note

workspace, project_name, and experiment can also be given as a single string, delimited with a slash.

get_account_details ¶

get_account_details()

Return the username and the default workspace name for the authorized user.

Returns:

dict –

Returns dictionary object of the format:

{
'userName': 'USERNAME',
'defaultWorkspaceName': 'WORKSPACE',
}

get_archived_experiment ¶

get_archived_experiment(workspace, project_name, experiment)

Get a single archived APIExperiment by workspace, project, experiment.

Parameters:

workspace (str) –
Workspace name
project_name (str) –
Project name
experiment (str) –
Experiment key

get_archived_experiments ¶

get_archived_experiments(workspace, project_name=None, pattern=None)

Get archived APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.

Parameters:

workspace (str) –
Workspace name
project_name (str, default: None ) –
The project name, if ommitted all projects will be searched
pattern (str, default: None ) –
Regex pattern to apply on the experiment name or the experiment key

get_artifact_details ¶

get_artifact_details(workspace=None, artifact_name=None, artifact_id=None)

Returns the details of a single artifact identified either by the workspace name + the artifact name or by its unique artifact ID.

Parameters:

workspace (str, default: None ) –
The name of the workspace
artifact_name (str, default: None ) –
The name of the artifact
artifact_id (str, default: None ) –
The unique ID of the artifact, for example 6194e719-f596-48e7-8cca-8530c16dd007

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()

artifact_details = api.get_artifact_details("demo", "demo-artifact")
print(artifact_details)

The print statement above prints the following output:

{'artifactId': '6194e719-f596-48e7-8cca-8530c16dd007',
    'project': 'demo-artifacts',
    'type': 'dataset',
    'name': 'demo-artifact',
    'description': None,
    'latestVersion': '2.0.0',
    'tags': [],
    'isPublic': False,
    'emoji': None,
    'sizeInBytes': 21113,
    'versions': [{'artifactVersionId': 'a8286090-c637-4270-99ab-25b18676a035',
            'version': '1.0.0',
            'owner': 'lothiraldan',
            'metadata': None,
            'createdFrom': None,
            'sizeInBytes': 0,
            'state': None,
            'added': 1621948911721,
            'alias': ['current-production'],
            'tags': ['production']},
        {'artifactVersionId': 'bf778c64-a97c-4bff-9752-7fa6bfebbe2e',
            'version': '2.0.0',
            'owner': 'lothiraldan',
            'metadata': None,
            'createdFrom': None,
            'sizeInBytes': 21113,
            'state': None,
            'added': 1621948972987,
            'alias': ['Latest'],
            'tags': ['staging']}]}

get_artifact_files ¶

get_artifact_files(
    workspace=None,
    artifact_name=None,
    artifact_id=None,
    version=None,
    alias=None,
)

Returns the files of a single artifact version. The artifact is identified either by the workspace name + the artifact name or by its unique artifact ID. The artifact version is identified either by an explicit version or by an explicit alias.

Parameters:

workspace (str, default: None ) –
The name of the workspace
artifact_name (str, default: None ) –
The name of the artifact
artifact_id (str, default: None ) –
The unique ID of the artifact, for example 6194e719-f596-48e7-8cca-8530c16dd007
version (str, default: None ) –
The version number of the artifact version you want
alias (str, default: None ) –
The alias of the artifact version you want

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()

artifact_files = api.get_artifact_files("demo", artifact_name="demo-artifact", version="2.0.0")
print(artifact_files)

The print statement above return the following:

{'files': [{'artifactId': '6194e719-f596-48e7-8cca-8530c16dd007',
    'artifactVersionId': 'bf778c64-a97c-4bff-9752-7fa6bfebbe2e',
    'assetId': '6aa914ffbee94e11b69445383d7732f4',
    'fileName': 'logo.png',
    'fileSize': 21113,
    'link': None,
    'dir': None,
    'type': 'unknown',
    'metadata': None}]}

To query by alias you can run:

import comet_ml

comet_ml.login()
api = comet_ml.API()
artifact_files = api.get_artifact_files("demo", artifact_name="demo-artifact", alias="current-production")
print(artifact_files)

which will print the following:

{'files': [{'artifactId': '6194e719-f596-48e7-8cca-8530c16dd007',
    'artifactVersionId': 'a8286090-c637-4270-99ab-25b18676a035',
    'assetId': 'dea243de41714a48961f725dbbe4d214',
    'fileName': 'file',
    'fileSize': 0,
    'link': 's3://bucket/dir/file',
    'dir': None,
    'type': 'unknown',
    'metadata': None}]}

get_artifact_list ¶

get_artifact_list(workspace, artifact_type=None)

Return the list of artifacts in a given workspace. Could be optionally filtered by a specific type.

Parameters:

workspace (str) –
The name of the workspace
artifact_type (str, default: None ) –
If provided only returns Artifacts with the given type

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()
artifacts = api.get_artifact_list("demo")

print(artifacts)

get_default_workspace ¶

get_default_workspace()

Get the default workspace name.

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()
workspace = api.get_default_workspace()

get_experiment ¶

get_experiment(workspace, project_name, experiment)

Get a single APIExperiment by workspace, project, experiment.

Parameters:

workspace (str) –
Workspace name
project_name (str) –
Project name
experiment (str) –
Experiment key

get_experiment_by_id ¶

get_experiment_by_id(experiment)

get_experiment_by_key ¶

get_experiment_by_key(experiment_key)

Get an APIExperiment by experiment key.

Parameters:

experiment_key (str) –
Experiment key

get_experiments ¶

get_experiments(workspace, project_name=None, pattern=None)

Get APIExperiments by workspace, workspace + project, or workspace + project + regular expression pattern.

Parameters:

workspace (str) –
Workspace name
project_name (str, default: None ) –
The project name, if ommitted all projects will be searched
pattern (str, default: None ) –
Regex pattern to apply on the experiment name or the experiment key

get_latest_registry_model_version_details ¶

get_latest_registry_model_version_details(
    workspace: str,
    registry_name: str,
    stage: Optional[str] = None,
    version_major: Optional[int] = None,
    version_minor: Optional[int] = None,
) -> Dict[str, Any]

Return details about the latest model registry version, including its asset list.

This method has been deprecated as stage has been replaced by status and a new comet_ml.Model object was introduced. We recommend using the comet_ml.API.get_model method to get the Model object and then using comet_ml.Model.find_versions

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.
version_major (int, default: None ) –
The major part of version string of the model.
version_minor (int, default: None ) –
The minor part of version string of the model.
stage (str, default: None ) –
A textual tag such as "production" or "staging".

Example

Running the following code:

import comet_ml

comet_ml.login()
api = comet_ml.API()
res = api.get_latest_registry_model_version_details("myworkspace", "model-name")

print(res)

will print the following dictionary:

{
    "registryModelId": "someRegistryModelId",
    "modelName": "someModelName",
    "description": "someDescription",
    "isPublic": "[Boolean]",
    "createdAt": "[long, when this model was created in the DB]",
    "lastUpdated": "[long, last time this model was updated in the DB]",
    "userName": "someUserName",
    "versions": [
        {
            "registryModelItemId": "someRegistryModelItemId",
            "experimentModel": {
                "experimentModelId": "someExperimentModelId",
                "experimentModelName": "someExperimentModelName",
                "experimentKey": "someExperimentKey"
            },
            "version": "someVersion",
            "comment": "someComment",
            "stages": [
                "production",
                "staging"
            ],
            "userName": "someUserName",
            "createdAt": "[long, when this model item was created in the DB]",
            "lastUpdated": "[long, last time this model item was updated in the DB]",
            "assets": [
                {
                    "fileName": "someFileName",
                    "fileSize": "[Long, file size]",
                    "runContext": "someRunContext",
                    "step": "[Integer, step asset was logged during]",
                    "link": "link to download asset file",
                    "createdAt": "[Long, timestamp asset was created in DB]",
                    "dir": "someDirectory",
                    "canView": "[Boolean, whether the asset is viewable as an image]",
                    "audio": "[Boolean, whether the asset is an audio file]",
                    "histogram": "[Boolean, whether the asset is a histogram file]",
                    "image": "[Boolean, whether the asset was stored as an image]",
                    "type": "the type of asset",
                    "metadata": "Metadata associated with the asset",
                    "assetId": "someAssetId"
                }
            ]
        }
    ]
}

get_metrics_df ¶

get_metrics_df(
    experiment_keys: List[str],
    metrics: List[str],
    x_axis: str = "step",
    interpolate: bool = True,
) -> Optional[pd.DataFrame]

Get a DataFrame of multiple metrics from a set of experiments across a specified x-axis.

Parameters:

experiment_keys (List[str]) –
A list of experiment keys.
metrics (List[str]) –
a list of metric names.
x_axis (step | epoch | duration, default: 'step' ) –
a specifield field that metric values will be based on.
interpolate (bool, default: True ) –
whether to apply linear interpolation to numerical columns or not.

Returns:

Optional[DataFrame] –
pd.DataFrame: A Pandas DataFrame of metrics from a set of experiments across a specified x-axis

get_metrics_for_chart ¶

get_metrics_for_chart(
    experiment_keys, metrics=None, parameters=None, independent=True, full=False
)

Get multiple metrics and parameters from a set of experiments. This method is designed to make custom charting easier.

Parameters:

experiment_keys (list) –
a list of experiment keys
metrics (list, default: None ) –
List of metric names (e.g., "loss")
parameters (list, default: None ) –
List of parameter names (e.g., "learning-rate")
independent (bool, default: True ) –
get independent results?
full (bool, default: False ) –
Fetch the full result?

Returns:

dict –
A dictionary of experiment keys with the following structure. {EXPERIMENT_KEY: {'params'} will be None if there are no parameters passed in.

Note

You should pass in a list of metric names, or a list of parameter names, or both.

Example

import comet_ml

comet_ml.login()
api = comet_ml.API()
res = api.get_metrics_for_chart([experiment_key1, experiment_key2, ...],
                                    ["loss"], ["hidden_layer_size"])
print(res)

The print statement above would return an object like:

{EXPERIMENT_KEY: {
'experiment_key': EXPERIMENT_KEY,
'steps': STEPS,
'epochs': None,
'metrics': [
    {'metricName': 'loss',
    'values': [VALUE, ...],
    'steps': [STEP, ...],
    'epochs': [EPOCH, ...],
    'timestamps': [TIMESTAMP, ...],
    'durations': [DURATION, ...],
    }],
'params': {'hidden_layer_size': VALUE, ...},
}, ...}

get_model ¶

get_model(workspace: str, model_name: str) -> model.Model

Get a Model API object corresponding to a given model

Parameters:

workspace (str) –
The name of workspace.
model_name (str) –
The name of registered model.

get_model_registry_version_assets ¶

get_model_registry_version_assets(
    workspace, registry_name, version=None, stage=None
)

Return details about a single model registry version, including its asset list.

This method has been deprecated as stage has been replaced by status and a new comet_ml.Model object was introduced. We recommend using the comet_ml.API.get_model method to get the Model object and then using comet_ml.Model.get_assets

Parameters:

workspace (str) –
The name of the workspace.
registry_name (str) –
The name of the model.
version (str, default: None ) –
The version string of the model.
stage (str, default: None ) –
A textual tag such as "production" or "staging".

Example

Running the following code

import comet_ml

comet_ml.login()
api = API()
res = api.get_model_registry_version_assets("myworkspace", "model-name")

print(res)

will print the dictionary:

{
    "registryModelItemId": "someRegistryModelItemId",
    "experimentModel": {
        "experimentModelId": "someExperimentModelId",
        "experimentModelName": "someExperimentModelName",
        "experimentKey": "someExperimentKey"
    },
    "version": "someVersion",
    "comment": "someComment",
    "stages": ["production", "staging"],
    "userName": "someUserName",
    "createdAt": "[long, when this model item was created in the DB]",
    "lastUpdated": "[long, last time this model item was updated in the DB]",
    "assets": [
        {
            "fileName": "someFileName",
            "fileSize": "[Long, file size]",
            "runContext": "someRunContext",
            "step": "[Integer, step asset was logged during]",
            "link": "link to download asset file",
            "createdAt": "[Long, timestamp asset was created in DB]",
            "dir": "someDirectory",
            "canView": "[Boolean, whether the asset is viewable as an image]",
            "audio": "[Boolean, whether the asset is an audio file]",
            "histogram": "[Boolean, whether the asset is a histogram file]",
            "image": "[Boolean, whether the asset was stored as an image]",
            "type": "the type of asset",
            "metadata": "Metadata associated with the asset",
            "assetId": "someAssetId",
        }
    ],
}