Integrate with Gymnasium¶

Gymnasium defines a standard API for defining Reinforcement Learning environments. Comet provides a gymnasium.Wrapper which makes it easy to log the environment performance to the Comet Platform.

Start logging¶

Wrap your gymnasium Enviornment with the CometLogger.

from comet_ml import Experiment, start, login
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

login()

experiment = start(project_name="comet-example-gymnasium-doc")

env = gym.make('Acrobot-v1')
env = CometLogger(env, experiment)

for x in range(20):
    observation, info = env.reset()
    truncated = False
    terminated = False
    while not (truncated or terminated):
        observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
        env.render()

Log automatically¶

Metrics:
- episode_reward: cumulative reward for an episode
- episode_length: number of steps in an episode

Visualize your RL training videos¶

The RecordVideo wrapper records videos of the environment. If this wrapper is used before CometLogger, CometLogger will log all video files to Comet via env.close().

from comet_ml import start, login
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

login()

experiment = start(project_name="comet-example-gymnasium-doc")

env = gym.make('Acrobot-v1', render_mode="rgb_array")
env = gym.wrappers.RecordVideo(env, 'test')
env = CometLogger(env, experiment)

for x in range(20):
    observation, info = env.reset()
    truncated = False
    terminated = False
    while not (truncated or terminated):
        observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
        env.render()

env.close() #Uploads video folder 'test' to Comet

Use the Comet Video Viewer Panel, found in the Public Panels Section, to view the mp4 files within the Comet Platform.

Continue your RL training¶

You can leverage ExistingExperiment to continue a training run. The experiment key of a previous run can be found in the Tabular view of the Experiments Pane of the Comet UI.

from comet_ml import ExistingExperiment
from comet_ml.integration.gymnasium import CometLogger
import gymnasium as gym

exp = ExistingExperiment(previous_experiment='YOUR_PREV_EXP_KEY')
env = gym.make('Acrobot-v1')
env = CometLogger(env, exp)

Try it out!¶

Don't just take our word for it, try it out for yourself.

For more examples using Gymnasium, see our examples GitHub repository.
Run the end-to-end example above in Colab:

Nov. 18, 2024