Configure the Comet SDK¶
When getting started with Comet experiment tracking, you need to make sure your API key is correctly configured and you may also want to make sure that you have customized Comet exactly for the kind of experiment logging you want.
Set the API key¶
You should use the login()
method to provide your API key to Comet, either from command line or in your Python code:
1 |
|
1 2 3 |
|
With either approach, the login()
method interactively prompts you for your API key which, once provided, is automatically accessed from your ~/.comet.config
configuration file.
Tip
If you would like to log data to the Comet platform but don't want to create an account, you can use comet_ml.login(anonymous=True)
. The data will be logged to a public Comet project which you can then claim if you wish.
Please don't use this modality with proprietary or other sensitive data.
Warning
While it is possible to pass your API key using comet_ml.login(api_key="your_api_key")
, it is not recommended since code is automatically logged by Comet and you could accidentally commit your API key to Git too.
Configure Comet logging¶
Comet has a wide range of configuration options allowing you to enable and disable features in a very granular way.
You can configure the comet_ml
Python SDK in three ways:
- From the
.comet.config
file: You can directly edit the configuration file, which is automatically created for you when runninglogin()
as detailed in the section above.
This is the recommended approach. - With Comet environment variables: You can define the Comet configuration as environment variables from terminal or in your Python code directly with the
os
functionalities.
Consider this approach when it is not convenient to create and store a configuration file. - In your Python code: You can specify the configuration when creating a Comet Experiment through the
comet_ml.start
method.
Consider this approach when wanting to enable or disable features in-place for the single Experiment only.
From the .comet.config file¶
The ~/.comet.config
configuration file follows the format:
[section_name]
config_name=value
So if you wanted to add your API key to your configuration file, you could set it using:
[comet]
api_key="your_api_key"
Find the list of Comet configuration names in the Explore the configuration options section.
With Comet environment variables¶
You can set environment variables to update the Comet configuration temporarily for the current session or permanently for all sessions.
Run export YOUR_VARIABLE_NAME="your_value"
from your terminal.
1. Open one of .bashrc
, .bash_profile
, or .profile
configuration files depending on your set-up.
2. Add this line to the file: export YOUR_VARIABLE_NAME="your_value"
.
3. Save and execute the file, e.g. with source ~/.bashrc
.
For more information and instructions, read more about setting environment variables in Python.
Comet environment variables are capitalized, and follow the format:
COMET_<parameter-name>
For example, you could set the Comet workspace by running export COMET_WORKSPACE="your_workspace"
.
Find the list of Comet environment parameter names in the Explore the configuration options section.
In your Python code¶
The comet_ml.start function supports a wide set of arguments that allow you to customize the automated logging performed by Comet by setting experiment configuration using ExperimentConfig
:
1 2 3 4 5 6 7 8 9 10 |
|
Many of the configuration options described in the Explore Comet configuration options section below can be set with this function. Please refer to the linked reference for a complete list of supported configuration options.
Explore Comet configuration options¶
When first setting up Comet for your project / workspace / team, you may find yourself choosing how Comet automatically logs information for you and what information it should log.
The options you are likely to update more frequently are listed at the top and highlighted in bold.
Comet Config Section/Name | Environment Parameter Name | Description |
---|---|---|
[comet] api_key | COMET_API_KEY | The API key used for creating Experiments and using the Python API |
[comet] workspace | COMET_WORKSPACE | Use this workspace when creating new experiments |
[comet] project_name | COMET_PROJECT_NAME | Use this project when creating new experiments |
[comet] url_override | COMET_URL_OVERRIDE | URL to upload the experiments to, this needs to be updated for on-premise deployments. Defaults to https://www.comet.com/clientlib/ . |
[comet_logging] file | COMET_LOGGING_FILE | Use the given file for storing log messages. This can also contain patterns such as "comet-{project}.log" (See below) |
[comet_logging] file_level | COMET_LOGGING_FILE_LEVEL | By default, the log outputs will contains INFO and higher level (WARNING and ERROR) log messages. This configuration item can be used to change the level of the logged messages. Defaults to INFO. |
[comet] offline_directory | COMET_OFFLINE_DIRECTORY | Set the offline directory for OfflineExperiment() . Defaults to .cometml-runs . |
[comet] git_directory | COMET_GIT_DIRECTORY | Set the directory that contains your git repo, if you are not under it |
[comet] display_summary_level | COMET_DISPLAY_SUMMARY_LEVEL | Controls the level of detail displayed in the experiment summary. If set to 0, the summary is hidden, but the summary notification is still sent. If set to 2, the experiment summary shows everything that has been logged. Defaults to 1. |
[comet] rich_output | COMET_RICH_OUTPUT | When enabled, Comet console log messages will be rendered using formatted methods from the Rich library. Defaults to True. |
[comet] system_cpu_interval | COMET_SYSTEM_CPU_INTERVAL | Controls the interval at which system CPU metrics (CPU utilization and memory utilization) are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
[comet] system_gpu_interval | COMET_SYSTEM_GPU_INTERVAL | Controls the interval at which system GPU metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
[comet] system_network_interval | COMET_SYSTEM_NETWORK_INTERVAL | Controls the interval at which system network metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
[comet] system_disk_interval | COMET_SYSTEM_DISK_INTERVAL | Controls the interval at which system disk metrics are logged. The value must be in seconds and cannot be less than 1s. Defaults to 60. |
[comet] http_session.retry_total | COMET_HTTP_SESSION_RETRY_TOTAL | Number of times the SDK will try to upload a message before triggering the fallback to offline mechanism. Defaults to 3. |
[comet] http_session.retry_backoff_factor | COMET_HTTP_SESSION_RETRY_BACKOFF_FACTOR | Controls how much time is applied between two different attempts to send data to Comet. Comet will wait for backoff_factor * (2 ** (attempt_number - 1)) seconds between attemps. Defaults to 2. |
[comet_fallback_streamer] keep_offline_zip | COMET_FALLBACK_STREAMER_KEEP_OFFLINE_ZIP | If True, save a local copy of experiment data for all training runs, not the just the ones that encountered connectivity issues. Defaults to False. |
[comet_logging] hide_api_key | COMET_LOGGING_HIDE_API_KEY | If True, redact the Comet API key in logs. Defaults to True. |
[comet_logging] file_overwrite | COMET_LOGGING_FILE_OVERWRITE | Overwrites the log file on each run, if True. Defaults to False. |
[comet_logging] console | COMET_LOGGING_CONSOLE | Set the logging level for console messages (for example, INFO, DEBUG). |
[comet_logging] metrics_ignore | COMET_LOGGING_METRICS_IGNORE | List of framework:metric names to not log automatically. Defaults to keras:batch_size,keras:batch_batch |
[comet_logging] parameters_ignore | COMET_LOGGING_PARAMETERS_IGNORE | List of framework:parameter names to not log automatically. Defaults to keras:verbose,keras:do_validation,keras:validation_steps |
[comet_logging] others_ignore | COMET_LOGGING_OTHERS_IGNORE | List of framework:other names do not log automatically. |
[comet_timeout] cleaning | COMET_TIMEOUT_CLEANING | How many times (in seconds) does the Python SDK wait until all experiment data has been uploaded. Default is 3600 seconds / 1 hour. |
[comet_auto_log] code | COMET_AUTO_LOG_CODE | Auto Log source code for the current Experiment Run. Defaults to True. |
[comet_auto_log] git_metadata | COMET_AUTO_LOG_GIT_METADATA | Auto Log git metadata for the current Experiment Run. Defaults to True. |
[comet_auto_log] git_patch | COMET_AUTO_LOG_GIT_PATCH | Auto Log the git patch for current Experiment Run. Set to True to enable. |
[comet_auto_log] graph | COMET_AUTO_LOG_GRAPH | Auto Log the model graph. This will depend on the framework being used. Set to True to enable. |
[comet_auto_log] parameters | COMET_AUTO_LOG_PARAMETERS | Auto Log the model parameters. This will depend on the framework being used. Set to True to enable. |
[comet_auto_log] weights | COMET_AUTO_LOG_WEIGHTS | Auto Log the model weights. This will depend on the framework being used. Set to True to enable. |
[comet_auto_log] metrics | COMET_AUTO_LOG_METRICS | Auto Log the model metrics. This will depend on the framework being used. Set to True to enable. |
[comet_auto_log] cli_arguments | COMET_AUTO_LOG_CLI_ARGUMENTS | Auto Log CLI arguments passed to the training script. Set to True to enable. |
[comet_auto_log] output_logger | COMET_AUTO_LOG_OUTPUT_LOGGER | Auto Log console and Jupyter Notebook standard output and error. Set to True to enable. |
[comet_auto_log] co2 | COMET_AUTO_LOG_CO2 | Auto Log CO2 emissions from the run using the code-carbon package. Set to True to enable. |
[comet_auto_log] env_details | COMET_AUTO_LOG_ENV_DETAILS | Auto Log details about the compute environment. Set to True to enable, set to False to quickly disable all environment details logging. |
[comet_auto_log] env_gpu | COMET_AUTO_LOG_ENV_GPU | Auto Log the collection of GPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True. |
[comet_auto_log] env_host | COMET_AUTO_LOG_ENV_HOST | Auto Log the host information (ip, host name, python version, user etc...). Logging Environment Details must also be set to True. |
[comet_auto_log] env_cpu | COMET_AUTO_LOG_ENV_CPU | Auto Log the collection of CPU details and system metrics (utilization, memory usage etc..). Logging Environment Details must also be set to True. |
[comet_auto_log] env_cpu_per_core | COMET_AUTO_LOG_ENV_CPU_PER_CORE | Auto Log the collection of CPU details and system metrics (utilization etc..) for each CPU core. Logging Environment Details must also be set to True. Defaults to False. |
[comet_auto_log] env_network | COMET_AUTO_LOG_ENV_NETWORK | Auto Log the collection of network system metrics. Logging Environment Details must also be set to True. Defaults to True. |
[comet_auto_log] env_disk | COMET_AUTO_LOG_ENV_DISK | Auto Log the collection of disk system metrics. Logging Environment Details must also be set to True. Defaults to True. |
[comet_auto_log] disable | COMET_AUTO_LOG_DISABLE | Disable all network communication with the Comet backend. It is useful when you want to test to make sure everything is working, without actually logging anything. Defaults to False. |
In addition, the following configurations are available only as environment variables:
Environment Parameter | Description |
---|---|
COMET_INI | A path to another .comet.config file to take into account |
COMET_OPTIMIZER_ID | Current optimizer ID; for use with the comet optimize CLI command |
COMET_OPTIMIZER_PROCESS_ID | Current job number; set when using comet optimize CLI command |
COMET_OPTIMIZER_PROCESS_JOBS | Total number of parallel jobs; set when using comet optimize CLI command |
COMET_DISABLE_AUTO_LOGGING | Disable all automatic logging (see next section) |