Getting the most out of Comet

Getting the most out of Comet

The Comet.ml UI is very powerful. Here are some items to highlight to help you get the most out of it.

Tags

Tags offer an easy way to group Experiments. Remember that through the UI you can apply multiple tags to each experiment, and easily add a tag to a group of experiments. Using the Query Builder you can filter the Experiment Table and data in charts to only see the Experiments with that tag. See instructions on how to programmatically tag Experiments here.

Info

Remember that you can use complex filters to select experiments, then tag them. Later, you can use the tag to easily select those same experiments with the filter.

Filters

Using the Query Builder you can select which Experiments to expose in the charts and Experiment Table. The Query Builder allows combining selection filters in complex ways. For example, you can select any Experiment that has a training_accuracy greater than 0.90 and a test_accuracy less than 0.90. See how to add Filters through the Query Builder here.

Info

Suggestion: if you are going to use the filters to create an interesting chart, add the chart first before creating the query. That way, you can visually inspect what you are selecting.

Assets

Every experiment can have its own set of assets (associated files). For example, you can add datasets, programs, configuration files, images, etc. as assets to an Experiment. You can also share assets among Experiments. See instructions on how to log Experiment assets here.

Hyperparameter Optimization

How can you find the right set of hyperparameters (such as learning rate, hidden layer size, optimizer, etc.) for any given problem? The answer is to use the Hyperparameter Optimizer to help you search through the entire cross-product of possibilities. See instructions on how to use the Optimizer here.

Reproduce an Experiment

After running an Experiment, you may wish to return to the exact details (including code, dataset, and hyperparameters) that created the original results. You can go back to the original state by going to the Experiment View, and clicking the Reproduce button. See more details about the Reproduce button here.

Info

If your code base is in Github, then Comet.ml will detect that, and store all of the details necessary to reconstruct the exact state of your repo. Simply click the Reproduce button on the Experiment's page and paste the displayed commands into the console. Comet.ml can even detect uncommitted changes, and provide a patch to get your code base in sync to run the experiment.