comet_ml.Experiment ¶

Creates a new experiment on Comet.

Parameters:

• api_key (str, default: None ) –

  Your API key obtained from comet.com

• project_name (str, default: None ) –

  Send your experiment to a specific project. Otherwise will be sent to Uncategorized Experiments. If project name does not already exists Comet will create a new project.

• workspace (str, default: None ) –

  Attach an experiment to a project that belongs to this workspace

• log_code (bool, default: True ) –

  Allows you to enable/disable code logging

• log_graph (bool, default: True ) –

  Allows you to enable/disable automatic computation graph logging.

• auto_param_logging (bool, default: True ) –

  Allows you to enable/disable hyper parameters logging

• auto_metric_logging (bool, default: True ) –

  Allows you to enable/disable metrics logging

• auto_metric_step_rate (int, default: 10 ) –

  Controls how often batch metrics are logged

• auto_histogram_tensorboard_logging (bool, default: False ) –

  Allows you to enable/disable automatic tensorboard histogram logging

• auto_histogram_epoch_rate (int, default: 1 ) –

  Controls how often histograms are logged

• auto_histogram_weight_logging (bool, default: False ) –

  Allows you to enable/disable histogram logging for biases and weights

• auto_histogram_gradient_logging (bool, default: False ) –

  Allows you to enable/disable automatic histogram logging of gradients

• auto_histogram_activation_logging (bool, default: False ) –

  Allows you to enable/disable automatic histogram logging of activations

• auto_output_logging (str, default: AUTO_OUTPUT_LOGGING_DEFAULT_VALUE ) –

  Allows you to select which output logging mode to use. You can pass "native" which will log all output even when it originated from a C native library. You can also pass "simple" which will work only for output made by Python code. If you want to disable automatic output logging, you can pass False. The default is "simple".

• auto_log_co2 (bool, default: True ) –

  Automatically tracks the CO2 emission of this experiment if codecarbon package is installed in the environment

• parse_args (bool, default: True ) –

  Allows you to enable/disable automatic parsing of CLI arguments

• log_env_details (bool, default: True ) –

  Log various environment information in order to identify where the script is running

• log_env_gpu (bool, default: True ) –

  Allow you to enable/disable the automatic collection of gpu details and metrics (utilization, memory usage etc..). log_env_details must also be true.

• log_env_cpu (bool, default: True ) –

  Allow you to enable/disable the automatic collection of cpu details and metrics (utilization, memory usage etc..). log_env_details must also be true.

• log_env_network (bool, default: True ) –

  Allow you to enable/disable the automatic collection of network details and metrics (sent, receive rates, etc..). log_env_details must also be true.

• log_env_disk (bool, default: True ) –

  Allow you to enable/disable the automatic collection of disk utilization metrics (usage, IO rates, etc.). log_env_details must also be true.

• log_env_host (bool, default: True ) –

  Allow you to enable/disable the automatic collection of host information (ip, hostname, python version, user etc...). log_env_details must also be true.

• log_git_metadata (bool, default: True ) –

  Allow you to enable/disable the automatic collection of git details

• log_git_patch (bool, default: True ) –

  Allow you to enable/disable the automatic collection of git patch

• display_summary_level (int, default: None ) –

  Control the summary detail that is displayed on the console at end of experiment. If 0, the summary notification is still sent. Valid values are 0 to 2.

• disabled (bool, default: False ) –

  Allows you to disable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything.

• experiment_key (str, default: None ) –

  If provided, will be used as the experiment key. If an experiment with the same key already exists, it will raises an Exception. Could be set through configuration as well. Must be an alphanumeric string whose length is between 32 and 50 characters.

Add a tag to the experiment. Tags will be shown in the dashboard.

Parameters:

• tag (str) –

  A tag to add to the experiment.

Add several tags to the experiment. Tags will be shown in the dashboard.

Parameters:

• tags (List[str]) –

  Tags list to add to the experiment.

Clean the experiment loggers, useful in case you want to debug your scripts with IPDB.

Returns:

• bool –

  Returns a Boolean indicating whether the cleaning was successful or not.

A context manager to mark the beginning and the end of the training phase. This allows you to provide a namespace for metrics/params. For example:

Parameters:

• context (str) –

  Name of the context manager.

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Log validation_accuracy using the context manager
with exp.context_manager("validation"):
    exp.log_metric("accuracy", 0.12)

Create a confusion matrix for use over multiple epochs.

Parameters:

• y_true (list, default: None ) –

  List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.

• y_predicted (list, default: None ) –

  List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.

• labels (list, default: None ) –

  A list of strings that name of the columns and rows, in order.

• matrix (list, default: None ) –

  The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.

• title (str, default: 'Confusion Matrix' ) –

  A custom name to be displayed. By default, it is "Confusion Matrix".

• row_label (str, default: 'Actual Category' ) –

  Label for rows.

• column_label (str, default: 'Predicted Category' ) –

  Label for columns.

• max_examples_per_cell (int, default: 25 ) –

  Maximum number of examples per cell.

• max_categories (int, default: 25 ) –

  Max number of columns and rows to use.

• winner_function (callable, default: None ) –

  A function that takes in an entire list of rows of patterns, and returns the winning category for each row.

• index_to_example_function (callable, default: None ) –

  A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.

• cache (bool, default: True ) –

  Should the results of index_to_example_function be cached and reused?

• selected (list, default: None ) –

  A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.

• images (list, default: None ) –

  A list of data that can be passed to Experiment.log_image().

• kwargs (optional, default: {} ) –

  Any extra keywords and their values will be passed onto the index_to_example_function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import comet_ml
from sklearn.metrics import confusion_matrix

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

y_true = [2, 0, 2, 2, 0, 1]
y_pred = [0, 0, 2, 2, 0, 2]

confusion_matrix = exp.create_confusion_matrix(y_true, y_pred)

exp.end()

Create an embedding image (a sprite sheet). Returns the image and the url to the image.

Parameters:

• image_data (Any) –

  List of arrays or Images.

• image_size (list) –

  The size of each image.

• image_preprocess_function (callable, default: None ) –

  If image_data is an array, apply this function to each element first.

• image_transparent_color (list, default: None ) –

  A (red, green, blue) tuple.

• image_background_color_function (callable, default: None ) –

  A function that takes an index, and returns a (red, green, blue) color tuple.

Returns:

• tuple ( Optional[Tuple[Any, str]] ) –

  Returns the image and url

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

num_images = 10
inputs = np.random.randint(0, 256, (num_images, 28, 28))
label = np.random.randint(0, 10, num_images)

def label_to_color(index):
    label_colors = [
        (255, 0, 0), (0, 255, 0), (0, 0, 255),
        (255, 255, 0), (0, 255, 255), (128, 128, 0),
        (0, 128, 128), (128, 0, 128), (255, 0, 255),
        (255, 255, 255)
    ]
    return label_colors[label[index] % len(label_colors)]

image, image_url = exp.create_embedding_image(
    image_data=inputs,
    image_size=(28, 28),
    image_preprocess_function=lambda matrix: np.round(matrix / 255, 0) * 2,
    image_transparent_color=(0, 0, 0),
    image_background_color_function=label_to_color
)

print("Generated Image URL:", image_url)

exp.end()

Disabling the auto-collection of metrics and monkey-patching of the Machine Learning frameworks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.disable_mp()

exp.log_parameter("example_parameter", 123)

exp.end()

Show the Comet experiment page in an IFrame in a Jupyter notebook or Jupyter lab, OR open a browser window or tab.

This is only supported for online experiment at the moment.

Parameters:

• clear (bool, default: False ) –

  To clear the output area, use clear=True. This is only used in Notebook environments.

• wait (bool, default: True ) –

  To wait for the next displayed item, use wait=True (cuts down on flashing). This is only used in Notebook environments.

• new (int, default: 0 ) –

  Open a new browser window if new=1, otherwise re-use existing window/tab. This is only used in non-Notebook environments.

• autoraise (bool, default: True ) –

  Make the browser tab/window active. This is only used in non-Notebook environments.

• tab (str, default: None ) –

  Name of the Tab on Experiment View

Raises:

• NotImplementedError –

  This is not yet supported for offline experiments.

Show the Comet project page in an IFrame in either (1) a Jupyter notebook or Jupyter lab or (2) open a browser window or tab.

This is only supported for online experiment at the moment.

Parameters:

• view_id (str, default: None ) –

  The id of the view to show.

• clear (bool, default: False ) –

  To clear the output area, use clear=True.

• wait (bool, default: True ) –

  To wait for the next displayed item, use wait=True (cuts down on flashing).

• new (int, default: 0 ) –

  Open a new browser window if new=1, otherwise re-use existing window/tab.

• autoraise (bool, default: True ) –

  Make the browser tab/window active.

Raises:

• NotImplementedError –

  This is not yet supported for offline experiments.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Optionally log some metrics or parameters (not required for displaying the project)
exp.log_metric("accuracy", 0.95)
exp.log_parameter("learning_rate", 0.01)

exp.display_project()

exp.end()

The end() method finalizes an experiment by ensuring all data (i.e., parameters, metrics, asset, etc.) are uploaded to Comet before it returns. Unless you are running in a Jupyter Notebook, using the end() method is optional as it will be called automatically by Comet when the script ends.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import comet_ml
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Initialize Comet
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Run an experiment
data = load_iris()
X = data.data
y = data.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)

y_pred_train = model.predict(X_train)

acc = accuracy_score(y_train, y_pred_train)

# Log accuracy metric to Comet
exp.log_metric("accuracy", acc)

# End the experiment
exp.end()

Flush all pending data to the Comet server. It works similar to end but without ending the run. This is a blocking operation that will wait for all the data logged so far to be delivered to the Comet server.

Returns:

• bool –

  Returns a Boolean indicating whether the flush was successful or not.

Returns a logged artifact object that can be used to access the artifact version assets and download them locally.

If no version or alias is provided, the latest version for that artifact is returned.

This is only supported for online experiment at the moment.

Parameters:

• artifact_name (str) –

  Retrieve an artifact with that name. This could either be a fully qualified artifact name like workspace/artifact-name:versionOrAlias or just the name of the artifact like artifact-name.

• workspace (str, default: None ) –

  Retrieve an artifact belonging to that workspace

• version_or_alias (str, default: None ) –

  Retrieve the artifact by the given alias or version.

Raises:

• NotImplementedError –

  This is not yet supported for offline experiments.

Returns:

• LoggedArtifact ( LoggedArtifact ) –

  The artifact requested

logged_artifact = experiment.get_artifact(
    "workspace/artifact-name:version_or_alias"
)

logged_artifact = experiment.get_artifact(
    artifact_name="artifact-name",
    workspace="workspace",
    version_or_alias="version_or_alias")

Get a callback for a particular framework.

Parameters:

• framework (str) –

  Specifies the machine learning framework for which you want a callback.

Returns:

• callable ( Callable ) –

  Callback object that is compatible with the framework specified in the call.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import comet_ml
import tensorflow as tf
from tensorflow import keras

# Initialize the Comet experiment with your API key and project information
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Load a dataset (using MNIST as an example)
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Build a simple model
model = keras.models.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(10)
])

# Compile the model
model.compile(optimizer='adam',
            loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
            metrics=['accuracy'])

# Get the Comet ML callback for Keras
comet_callback = exp.get_callback("keras")

# Train the model
model.fit(x_train, y_train, epochs=5, validation_data=(x_test, y_test), callbacks=[comet_callback])

# End the experiment
exp.end()

This method is deprecated. See Experiment.get_callback("keras"). Get a callback for the Keras framework.

Returns the experiment key, useful for using with the ExistingExperiment class

Returns:

• str ( str ) –

  Experiment Key (String)

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.end()

prev_exp = comet_ml.ExistingExperiment(experiment_key=exp.get_key())

Get the last value for a metric from those logged.

Parameters:

• name (str) –

  The name of the metric to get. This method will return the last value logged to this metric.

Get the name of the experiment, if one.

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.set_name("My Name")

print(exp.get_name())

Get the latest value logged for an other parameter from those previously logged.

Parameters:

• name (str) –

  The name of the other parameter to get.

Get a parameter that was logged previously in an Experiment.

Parameters:

• name (str) –

  The name of the parameter to get.

Returns:

• str ( Any ) –

  Parameter value

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_parameter("learning_rate", 0.0001)
exp.log_parameter("batch_size", 64)

learning_rate = exp.get_parameter("learning_rate")
print(f"Learning Rate: {learning_rate}")
assert float(learning_rate) == 0.0001

batch_size = exp.get_parameter("batch_size")
print(f"Batch Size: {batch_size}")
assert int(batch_size) == 64

exp.end()

Return the tags of this experiment.

Returns:

• List[str] –

  List[str]: The list of tags.

Log an Artifact object, synchronously create a new Artifact Version and upload asynchronously all local and remote assets attached to the Artifact object.

This is only supported for online experiment at the moment.

Parameters:

• artifact (Artifact) –

  An Artifact object.

Raises:

• NotImplementedError –

  This is not yet supported for offline experiments.

Returns:

• LoggedArtifact ( LoggedArtifact ) –

  The artifact that was logged

Logs the Asset determined by file_data.

Parameters:

• file_data (Any) –

  Either the file path of the file you want to log, or a file-like asset.

• file_name (str, default: None ) –

  A custom file name to be displayed. If not provided the filename from the file_data argument will be used.

• overwrite (bool, default: False ) –

  If True will overwrite all existing assets with the same name.

• copy_to_tmp (bool, default: True ) –

  (Deprecated) If file_data is a file-like object, then this flag determines if the file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

• step (int, default: None ) –

  Used to associate the asset to a specific step.

• metadata (Any, default: None ) –

  Some additional data to attach to the audio asset. Must be a JSON-encodable dict.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import comet_ml
import pandas as pd

data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'David'],
    'Age': [25, 30, 35, 40],
    'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']
}
df = pd.DataFrame(data)
df.to_csv('sample_data.csv', index=False)

comet_ml.login()
exp = comet_ml.Experiment(comet_project="comet-docs")

exp.log_asset(file_data="sample_data.csv", file_name="sample_data.csv")

exp.end()

Logs the data given (str, binary, or JSON).

Parameters:

• data (Any) –

  data to be saved as asset.

• name (Optional[str], default: None ) –

  A custom file name to be displayed. If not provided the filename from the temporary saved file will be used.

• overwrite (bool, default: False ) –

  If True will overwrite all existing assets with the same name.

• step (Optional[int], default: None ) –

  Used to associate the asset to a specific step.

• epoch (Optional[int], default: None ) –

  Used to associate the asset to a specific epoch.

• metadata (Optional[Dict[str, Any]], default: None ) –

  Some additional data to attach to the asset data. Must be a JSON-encodable dict.

• file_name (Optional[str], default: None ) –

  A custom file name to be displayed [Deprecated: use name instead].

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

data = {"key": "value", "number": 123}

exp.log_asset_data(data, file_name = "sample_data.json")

exp.end()

Logs all the files located in the given folder as assets.

Parameters:

• folder (str) –

  the path to the folder you want to log.

• step (int, default: None ) –

  used to associate the asset to a specific step.

• log_file_name (bool, default: None ) –

  if True, log the file path with each file.

• recursive (bool, default: False ) –

  if True, recurse folder and save file names.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import comet_ml

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Path to the folder containing assets you want to log
folder_path = "./"

# Log the entire folder as an asset
exp.log_asset_folder(folder_path)

# End the experiment
exp.end()

Logs the audio Asset determined by audio data.

Parameters:

• audio_data (str | array) –

  Either the file path of the file you want to log, or a numpy array given to scipy.io.wavfile.write for wav conversion. Note that scipy assumes that stereo audio is in (n_samples, n_channels) order.

• sample_rate (int, default: None ) –

  The sampling rate given to scipy.io.wavfile.write for creating the wav file.

• normalize_data (str, default: 'PCM_16_BIT' ) –

  The parameter that determines whether audio_data should be normalized to fit within the specific numerical bounds (e.g. 16-bit PCM). If you set this parameter to 'OFF', it is your responsibility to ensure that audio_data is compatible with the format supported by scipy.io.wavfile.write. Supported values: 'PCM_16_BIT', 'OFF'.

• file_name (str, default: None ) –

  A custom file name to be displayed. If not provided, the filename from the audio_data argument will be used.

• metadata (dict, default: None ) –

  Some additional data to attach to the audio asset. Must be a JSON-encodable dict.

• overwrite (bool, default: False ) –

  If True will overwrite all existing assets with the same name.

• copy_to_tmp (bool, default: True ) –

  (Deprecated) If audio_data is a numpy array, then this flag determines if the WAV file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

• step (int, default: None ) –

  Used to associate the audio asset to a specific step.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import comet_ml
import numpy as np

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Create an audio sample
fs = 100
rate = 44100
t = np.linspace(0., 1., rate)
amplitude = np.iinfo(np.int16).max
audio_sample = amplitude * np.sin(2. * np.pi * fs * t)

# Log the audio to Comet ML
exp.log_audio(
    audio_data=audio_sample,
    sample_rate=rate,
    file_name="Example Audio",
    step=0,
)

# End the experiment
exp.end()

Logs additional source code files. This method allows logging code in different ways:

• Without parameters, logs the code from the file calling this method.
• With file_name, logs the code from the specified file.
• Using folder, logs all files' code in that folder.
• Using code, logs the content as source code.

Parameters:

• file_name (str, default: None ) –

  File path to log.

• folder (str, default: None ) –

  Folder path where the code files are stored.

• code (str, default: None ) –

  Source code, either as text or a file-like object (such as StringIO). If passed, code_name is mandatory.

• code_name (str, default: None ) –

  Name of the source code file.

• overwrite (bool, default: False ) –

  If True, overwrites existing code with the same name.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Logs code from the file using this method.
exp.log_code()

# Logs code from the file 'dataset.py'.
exp.log_code(file_name="dataset.py")

# Logs all files code in the 'training/' folder.
exp.log_code(folder="training/")

# Logs any other code
exp.log_code(code=DYNAMIC_CODE_SOURCE, code_name="user_training_code.py")

Logs a confusion matrix.

Parameters:

• y_true (list, default: None ) –

  List of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.

• y_predicted (list, default: None ) –

  List of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.

• labels (list, default: None ) –

  A list of strings that name of the columns and rows, in order.

• matrix (list, default: None ) –

  The confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.

• title (str, default: 'Confusion Matrix' ) –

  A custom name to be displayed. By default, it is "Confusion Matrix".

• row_label (str, default: 'Actual Category' ) –

  Label for rows.

• column_label (str, default: 'Predicted Category' ) –

  Label for columns.

• max_examples_per_cell (int, default: 25 ) –

  Maximum number of examples per cell.

• max_categories (int, default: 25 ) –

  Max number of columns and rows to use.

• winner_function (callable, default: None ) –

  A function that takes in an entire list of rows of patterns, and returns the winning category for each row.

• index_to_example_function (callable, default: None ) –

  A function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. If left blank, the function returns a number representing the index of the example.

• cache (bool, default: True ) –

  Should the results of index_to_example_function be cached and reused?

• file_name (str, default: 'confusion-matrix.json' ) –

  The name of the file that the confusion matrix will be saved as when logged.

• overwrite (bool, default: False ) –

  If set to True, the existing confusion matrix with the same file name will be overwritten. If False, a new entry is created, preserving the previous versions.

• step (int, default: None ) –

  The step parameter can be used to specify the particular step or iteration in the training process at which the confusion matrix is logged.

• epoch (int, default: None ) –

  Similar to the step parameter, epoch specifically denotes the training epoch.

• images (list, default: None ) –

  A list of data that can be passed to Experiment.log_image().

• selected (list, default: None ) –

  A list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.

• kwargs (Any, default: {} ) –

  Any extra keywords and their values will be passed onto the index_to_example_function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import comet_ml
from sklearn.metrics import confusion_matrix

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

y_true = [2, 0, 2, 2, 0, 1]
y_pred = [0, 0, 2, 2, 0, 2]

confusion_matrix = exp.create_confusion_matrix(y_true, y_pred)
exp.log_confusion_matrix(matrix=confusion_matrix, step=0, file_name="confusion_matrix.json")

exp.end()

Log timeseries data.

Parameters:

• name (str) –

  Name of data.

• x (Any) –

  list of x-axis values.

• y (Any) –

  list of y-axis values.

• overwrite (bool, default: False ) –

  If True, overwrite previous log.

• step (Optional[int], default: None ) –

  The step value.

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

exp.log_curve("my curve", x=[1, 2, 3, 4, 5],
                          y=[10, 20, 30, 40, 50])
exp.end()

Log a pandas DataFrame profile as an asset. Optionally, can also log the dataframe.

Parameters:

• dataframe (Any) –

  the dataframe to profile and/or log

• name (str, default: 'dataframe' ) –

  the basename (without extension) of the dataframe assets.

• minimal (bool, default: False ) –

  if True, create a minimal profile. Useful for large datasets.

• log_raw_dataframe (bool, default: True ) –

  Log the dataframe as an asset (same as calling log_table())

• dataframe_format (str, default: 'json' ) –

  The format. for optionally logging the dataframe.

• format_kwargs (Any, default: {} ) –

  Keyword args for dataframe logging as an asset.

1
2
3
4
5
6
7
8
9

import comet_ml
import pandas as pd

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
df = pd.read_csv("https://people.sc.fsu.edu/~jburkardt/data/csv/airtravel.csv",
                 parse_dates=['year'], encoding='UTF-8')
exp.log_dataframe_profile(df)
exp.end()

See also: Experiment.log_table(pandas_dataframe)

Used to log the hash of the provided object.

Parameters:

• data (Any) –

  Any object that when casted to string (e.g. str(data)) returns a value that represents the underlying data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import comet_ml
import pandas as pd

# Initialize Comet ML Experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name = "comet-docs")

# Example: Load a dataset (could be from a file, database, etc.)
data = pd.read_csv('path/to/your/dataset.csv')

# Function to create a consistent hash of a pandas DataFrame
def hash_dataframe(df):
    return pd.util.hash_pandas_object(df).sum()

# Compute the hash of the dataset
dataset_hash = hash_dataframe(data)

# Log the dataset hash to Comet ML
exp.log_dataset_hash(dataset_hash)

# End the experiment
exp.end()

Used to log information about your dataset.

Parameters:

• name (str, default: None ) –

  A string representing the name of the dataset.

• version (str, default: None ) –

  A string representing a version identifier.

• path (str, default: None ) –

  A string that represents the path to the dataset. Potential values could be a file system path, S3 path, or Database query.

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_dataset_info(name="Your dataset", version="1.0", path="./data.csv")

exp.end()

Reports name,version to the Installed Packages tab on Comet. Useful to track dependencies.

Parameters:

• name (Any) –

  Any type of key (str,int,float...)

• version (Any) –

  Any type of value (str,int,float...)

Log a multidimensional dataset and metadata for viewing with Comet's Embedding Projector (experimental).

Parameters:

• vectors –

  the tensors to visualize in 3D.

• labels –

  labels for each tensor, or a table for each tensor

• image_data (optional, default: None ) –

  list of arrays or Images

• image_size (optional, default: None ) –

  The size of each image, required if image_data is given.

• image_preprocess_function (optional, default: None ) –

  If image_data is an array, apply this function to each element first

• image_transparent_color –

  A (red, green, blue) tuple.

• image_background_color_function –

  a function that takes an index, and returns a (red, green, blue) color tuple.

• title –

  Name of tensor.

• template_filename –

  Name of template JSON file.

• group –

  Name of group of embeddings.

See also: comet_ml.Embedding

Note: labels must be a list of strings, or a table where table is a list of lists of data, and the first row is a header followed by a list for each vector. See example below.

Examples:

from comet_ml import Experiment

import numpy as np
from keras.datasets import mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()

def label_to_color(index):
    label = y_test[index]
    if label == 0:
        return (255, 0, 0)
    elif label == 1:
        return (0, 255, 0)
    elif label == 2:
        return (0, 0, 255)
    elif label == 3:
        return (255, 255, 0)
    elif label == 4:
        return (0, 255, 255)
    elif label == 5:
        return (128, 128, 0)
    elif label == 6:
        return (0, 128, 128)
    elif label == 7:
        return (128, 0, 128)
    elif label == 8:
        return (255, 0, 255)
    elif label == 9:
        return (255, 255, 255)

experiment = Experiment(project_name="projector-embedding")

experiment.log_embedding(
    vectors=x_test,
    labels=y_test,
    image_data=x_test,
    image_preprocess_function=lambda matrix: np.round(matrix/255,0) * 2,
    image_transparent_color=(0, 0, 0),
    image_size=(28, 28),
    image_background_color_function=label_to_color,
)

# With a table:
experiment.log_embedding(
    vectors=[[3, 6, 2], [6, 1, 3], [9, 1, 1]],
    labels=[
        ["index", "label"],
        [      5, "apple"],
        [     16, "car"],
        [      2, "person"],
    ]
)

Logs that the epoch finished. Required for progress bars.

Parameters:

• epoch_cnt (int) –

  The current count of completed epochs, indicating how many epochs have finished in the training cycle.

• step (int, default: None ) –

  The step count at which the epoch ends.

1
2
3
4
5
6
7
8
9

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

num_epochs = 5
exp.log_epoch_end(num_epochs)

exp.end()

Log a visual representation of the provided figure to Comet in SVG format.

Parameters:

• figure_name (str, default: None ) –

  A descriptive name for the figure.

• figure (Any, default: None ) –

  The figure to be logged. Accepts Plotly Figures, Matplotlib Figures, or Seaborn simple plots. If not provided, the function will log the current global Matplotlib Pyplot figure.

• overwrite (bool, default: False ) –

  Determines whether to overwrite an existing figure with the same name.

• step (int, default: None ) –

  Associates the figure asset with a specific step in the Comet experiment.

Logs a histogram of values for a 3D chart as an asset for this experiment. Calling this method multiple times with the same name and incremented steps will add additional histograms to the 3D chart on Comet.ml.

Parameters:

• values (Any) –

  A list, tuple, array (any shape) to summarize, or a Histogram object.

• name (Optional[str], default: None ) –

  Name of summary.

• step (Optional[int], default: None ) –

  Used as the Z axis when plotting on Comet.

• epoch (Optional[int], default: None ) –

  Used as the Z axis when plotting on Comet.

• metadata (Optional[Dict[str, Any]], default: None ) –

  Used for items like prefix for histogram name.

• kwargs (Any, default: {} ) –

  Additional keyword arguments for histogram.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

values = np.random.normal(0, 1, (1000, 3))

exp.log_histogram_3d(
    values=values,
    name="Histogram of Randomly Distributed Data",
    step=1,
    epoch=1,
    metadata={'description': 'This is a 3D histogram of three normally distributed random variables'}
)

exp.end()

Reports any HTML blob to the HTML tab on Comet.ml. Useful for creating your own rich reports. The HTML will be rendered as an Iframe. Inline CSS/JS supported.

Parameters:

• html (str) –

  Any html string.

• clear (bool, default: False ) –

  When setting clear=True it will remove all previous html.

1
2
3
4
5
6
7

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html('<a href="www.comet.com"> I love Comet</a>')
exp.end()

Easy to use method to add a link to a URL in the HTML tab on Comet.

Parameters:

• url (str) –

  a link to a file or notebook, for example

• text (Optional[str], default: None ) –

  text to use a clickable word or phrase (optional; uses url if not given)

• label (Optional[str], default: None ) –

  text that precedes the link

1
2
3
4
5
6
7

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html_url("https://my-company.com/file.txt")
exp.end()

<a href="https://my-company.com/file.txt">
https://my-company.com/file.txt
</a>

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_html_url("https://my-company.com/file.txt",
                            "File")
exp.end()

<a href="https://my-company.com/file.txt">File</a>

1
2
3
4
5

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")
exp.log_html_url("https://my-company.com/file.txt", "File", "Label")

Label: <a href="https://my-company.com/file.txt">File</a>

Logs the image. Images are displayed on the Graphics tab in Comet.

Parameters:

• image_data (Any) –

  Is one of the following: - a path (string) to an image - a file-like object containing an image - a numpy matrix - a TensorFlow tensor - a PyTorch tensor - a list or tuple of values - a PIL Image

• name (str, default: None ) –

  A custom name to be displayed on the dashboard. If not provided the filename from the image_data argument will be used if it is a path.

• overwrite (bool, default: False ) –

  If another image with the same name exists, it will be overwritten if overwrite is set to True.

• image_format (str, default: 'png' ) –

  If the image_data is actually something that can be turned into an image, this is the format used. Typical values include 'png' and 'jpg'.

• image_scale (float, default: 1.0 ) –

  If the image_data is actually something that can be turned into an image, this will be the new scale of the image.

• image_shape (tuple, default: None ) –

  If the image_data is actually something that can be turned into an image, this is the new shape of the array. Dimensions are (width, height) or (width, height, colors) where colors is 3 (RGB) or 1 (grayscale).

• image_colormap (str, default: None ) –

  If the image_data is actually something that can be turned into an image, this is the colormap used to colorize the matrix.

• image_minmax (tuple, default: None ) –

  If the image_data is actually something that can be turned into an image, this is the (min, max) used to scale the values. Otherwise, the image is autoscaled between (array.min, array.max).

• image_channels (str, default: 'last' ) –

  If the image_data is actually something that can be turned into an image, this is the setting that indicates where the color information is in the format of the 2D data. 'last' indicates that the data is in (rows, columns, channels) where 'first' indicates (channels, rows, columns).

• copy_to_tmp (bool, default: True ) –

  (Deprecated) If image_data is not a file path, then this flag determines if the image is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud.

• step (int, default: None ) –

  Used to associate the image asset to a specific step.

• annotations (list, default: None ) –

  Annotations contains a list of annotation layers. Each annotation layer can contain bounding boxes and/or polygons. See example below.

• metadata (dict, default: None ) –

  Additional metadata to be associated with logged image.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

import comet_ml
from comet_ml.annotations import Layer, Box, Polygon
import numpy as np
from PIL import Image

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")


# Create a dummy image using numpy
width, height = 256, 256
array = np.tile(np.linspace(0, 255, width, dtype=np.uint8), (height, 1))
image = Image.fromarray(array)

# Convert image to grayscale
image = image.convert("L")

# One layer with a box and polygon:
annotations = [
    Layer(
        boxes=[Box([x, y, w, h], label="dog")],
        polygons=[Polygon([(x1, y1), ...], label="cat")],
        name="Predictions",
    )
]

# Log the image
exp.log_image(
    image_data=image,
    name="Gradient.png",
    image_format="png",
    image_scale=0.5,
    image_shape=(128, 128),
    image_colormap='gray',
    step=1,
    annotations=annotations,
    metadata={"description": "A simple gradient image from black to white"}
)

# End the experiment
exp.end()

Logs a general metric (i.e. accuracy, f1).

See also log_metrics.

Parameters:

• name (Any) –

  Name of your metric.

• value (Any) –

  The value of your metric, we recommend that this is a numerical value.

• step (Optional[int], default: None ) –

  Used as the X axis when plotting on Comet.

• epoch (Optional[int], default: None ) –

  Used as the X axis when plotting on Comet.

• include_context (bool, default: True ) –

  If set to True, the current context will be logged along the metric.

1
2
3

y_pred_train = model.predict(X_train)
acc = compute_accuracy(y_pred_train, y_train)
experiment.log_metric("accuracy", acc)

Logs a key,value dictionary of metrics. See also log_metric

Logs the model data under the name. Data can be a file path, a folder path or a file-like object.

Parameters:

• name (str) –

  The name of the model

• file_or_folder (str | dict) –

  The model data; can be a file path, a folder path or a file-like object.

• file_name (str, default: None ) –

  The name of the model data. Used with file-like objects or files only.

• overwrite (bool, default: False ) –

  If True, then overwrite previous versions Does not apply to folders.

• metadata (dict, default: None ) –

  Some additional data to attach to the the data. Must be a JSON-encodable dict.

• copy_to_tmp (bool, default: True ) –

  Deprecated. For file name or file-like; if True copy to temporary location before uploading; if False, then upload from current location

• prepend_folder_name (bool, default: True ) –

  If True and logging a folder, prepend file path by the folder name.

Returns:

• dict ( Optional[Dict[str, str]] ) –

  Dictionary of model URLs

Log a Jupyter Notebook file as an asset.

Parameters:

• filename (str) –

  The path and name of the notebook to be logged.

• overwrite (bool, default: False ) –

  If True, overwrites the previously logged notebook asset.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import comet_ml
import nbformat
from nbformat.v4 import new_notebook

# Create a dummy notebook to log
nb = new_notebook()

with open('example_notebook.ipynb', 'w', encoding='utf-8') as f:
    nbformat.write(nb, f)

# Log the new notebook to Comet
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

exp.log_notebook("./example_notebook.ipynb")

exp.end()

Logs an existing optimization result.

Parameters:

• optimization_id (str, default: None ) –

  The id of the optimization result.

• metric_name (str, default: None ) –

  The name of your metric

• metric_value (Any, default: None ) –

  The value of the given metric.

• parameters (dict, default: None ) –

  Additional parameters to be logged.

• objective (str, default: None ) –

  The objective of the optimization, could be either maximum/minimum.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Define a hypothetical scenario for optimization results
optimization_id = "opt123"
metric_name = "accuracy"
metric_value = 0.95
parameters = {
    'C': 1,
    'kernel': 'linear',
    'gamma': 'scale'
}
objective = "maximum"

# Log the optimization
exp.log_optimization(
    optimization_id=optimization_id,
    metric_name=metric_name,
    metric_value=metric_value,
    parameters=parameters,
    objective=objective
)

# End the experiment
exp.end()

Reports a key and value to the Other tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.

Other metadata can also be defined as environment variables. All environment variables that starts with COMET_LOG_OTHER_ will be logged automatically to the Experiment.

Parameters:

• key (Any) –

  Any type of key (str,int,float..)

• value (Any) –

  Any type of value (str,int,float..)

Reports dictionary of key/values to the Other tab on Comet.ml. Useful for reporting datasets attributes, datasets path, unique identifiers etc.

See log_other

Parameters:

• dictionary (dict) –

  dict of key/values where value is any type of value (str,int,float..)

Logs a parameter.

Parameters:

• name (Any) –

  Name of your parameter.

• value (Any) –

  The value of your parameter.

• step (Optional[int], default: None ) –

  Used as the x-axis when plotting on Comet.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import comet_ml

# Initialize an experiment
comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Log a parameter
exp.log_parameter("batch_size", 64)

# End the experiment
exp.end()

Logs a dictionary (or dictionary-like object) of multiple parameters.

Parameters:

• parameters (dict) –

  This is a dictionary where each key is a string representing the name of the parameter, and the value is the parameter value itself, which can be of any data type that is serializable. The method logs each key-value pair as a parameter in the experiment.

• prefix (str, default: None ) –

  This optional parameter allows you to add a prefix to the keys in the parameters dictionary when they are logged. This can be helpful for organizing or grouping parameters under a common namespace, making them easier to identify and analyze in the Comet UI.

• step (int, default: None ) –

  This optional parameter can be used to associate the logged parameters with a particular step or iteration in an experiment. This is useful for tracking how parameters change over time, particularly in iterative processes like training a machine learning model over multiple epochs.

• nested_support (bool, default: True ) –

  This parameter controls whether the method should support nested dictionaries. If set to True, the method can handle parameters that are dictionaries themselves and will flatten these into a format suitable for logging. Each key in a nested dictionary will be combined with its parent keys to create a single, flat key.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import comet_ml

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

params = {
    "batch_size": 64,
    "layer1": "LSTM(128)",
    "layer2": "LSTM(128)",
    "MAX_LEN":200
}

exp.log_parameters(params)

exp.end()

Log 3d points and bounding boxes as an asset.

Parameters:

• scene_name (str) –

  A string identifying the 3d scene to render. A same scene name could be logged across different steps.

• points (list, default: None ) –

  A list of points, each point being a list (or equivalent like Numpy array). Each point length should be either 3, if only the position is given: [X, Y, Z]. The length could also be 6, if color is passed as well: [X, Y, Z, R, G, B]. Red, Green and Blue should be a number between 0 and 1. Either points or boxes are required.

• boxes (list, default: None ) –

  List of box definition Dict.

• step (int, default: None ) –

  Used to associate the asset to a specific step.

• metadata (dict, default: None ) –

  Additional custom metadata can be associated with the logged asset.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

import comet_ml
import numpy as np

comet_ml.login()
exp = comet_ml.Experiment(project_name="comet-docs")

# Generate some synthetic 3D points
np.random.seed(42)
points = np.random.rand(10, 6)  # 10 points, with XYZ and RGB values

# Normalize the color values to be between 0 and 1
points[:, 3:] = points[:, 3:] / np.max(points[:, 3:])

# Define a bounding box as a list of segments (lines)
# Each segment is a pair of points (x, y, z)
box_segments = [
    # Base of the box
    [[0, 0, 0], [1, 0, 0]],
    [[1, 0, 0], [1, 1, 0]],
    [[1, 1, 0], [0, 1, 0]],
    [[0, 1, 0], [0, 0, 0]],
    # Top of the box
    [[0, 0, 1], [1, 0, 1]],
    [[1, 0, 1], [1, 1, 1]],
    [[1, 1, 1], [0, 1, 1]],
    [[0, 1, 1], [0, 0, 1]],
    # Vertical lines
    [[0, 0, 0], [0, 0, 1]],
    [[1, 0, 0], [1, 0, 1]],
    [[1, 1, 0], [1, 1, 1]],
    [[0, 1, 0], [0, 1, 1]],
]

# Create a box dictionary with the segments
box = {
    "segments": box_segments,
    "color": [255, 0, 0],  # Red color
    "name": "example_box",
}

# Log 3D points and the bounding box to Comet
exp.log_points_3d(
    scene_name="Simple3DExample",
    points=points.tolist(),
    boxes=[box],
)

# End the experiment
exp.end()