We’re excited to announce another excellent integration with Comet — Ray! This integration allows data scientists to leverage Comet’s experiment tracking and visualization tools with Ray’s incredible library for scaling compute intensive Machine Learning workloads.

About Ray

Ray is an open source project that makes it simple to scale any compute-intensive Python workload — from deep learning to production model serving.

With a rich set of libraries for Hyperparameter Tuning, Reinforcement Learning, Model Serving and integrations built on a flexible distributed execution framework, Ray makes distributed computing easy and accessible to every engineer.

About the Integration

Ray now ships with a CometLoggerCallback. You can now log metrics, hyperparameters, source code, and much more from your Ray Trials to the Comet UI.

Once you’ve set up your account and configured your Comet API Key within your project, you install Ray Tune and Comet using pip

pip install comet_ml
pip install "ray[tune]"

Configure your project with your Comet credentials.

export COMET_API_KEY="Your API Key"
export COMET_WORKSPACE="Your Workspace Name"
export COMET_PROJECT_NAME ="Your Project Name"

Once you have set up your credentials, simply import the CometLoggerCallback from Ray!

The following code snippet shows how you can easily log Ray Trials to the Comet UI.

"""Examples logging Tune runs to comet.ml"""
import comet_ml
import argparse
import numpy as np

from ray import tune
from ray.tune.integration.comet import CometLoggerCallback


def train_function(config, checkpoint_dir=None):
    for i in range(30):
        loss = config["mean"] + config["sd"] * np.random.randn()
        tune.report(loss=loss)


def tune_function():
    analysis = tune.run(
        train_function,
        name="comet-ray-demo",
        metric="loss",
        mode="min",
        callbacks=[
            CometLoggerCallback(
                tags=["my-trial"]
            )
        ],
        config={"mean": tune.grid_search([1, 2, 3]), "sd": tune.uniform(0.2, 0.8)},
    )
    return analysis.best_config

best_config = tune_function()

Your runs will now be reported by Comet. You’ll then be able to take advantage of Comet’s rich visualization and analysis capabilities to gain insights from your Ray experiments.

Ray Tune Runs Logged to Comet

Getting Started

Getting started with this integration is really easy and the following resources should help you start logging Ray Trials to Comet in no time!

• A free Comet account: Building with Comet is absolutely free — unlimited public and private projects, 100GB of storage, hyperparameter search, and more.
• Colab Notebook: Our Notebook is ready to run, but you can also create a copy if you’d like to modify it.
• An example Comet Project showcasing logged runs from a Ray Tune Hyperparameter Sweep
• A full example using Comet and Ray to run and log a Hyperparameter search on MNIST
• Ray GitHub Repo: Need a crash course on Ray? Check out this GitHub repo for the basics, runnable examples, and much more.

You’re all set to track experiment data at scale with Ray. Companies like Uber, Amazon, LinkedIn, and OpenAI all use Ray to run their ML workflows. We are excited to see what advances in Machine Learning can be built with this integration!