Javascript Panels SDK¶

When you log metrics, hyperparameters, or other values in Comet, you can easily plot those values using the built-in charts. However, you may want to alter the chart in a subtle way. For example, perhaps you want the lines slightly thicker in the display. Or perhaps you want to display your data in a manner that is very different from any of the built-in charting options.

You can easily create your own visualizations with your Comet data and reuse them on any of your Projects or Experiments. While we recommend you use our Python Panels, you can also use Javascript Panels to create highly interactive panels.

This page demonstrates how to do this:

• First, learn how to create a simple Panel from scratch.
• Next, deconstruct a pre-written Panel.

For more information on Panels, see:

• Comet's JavaScript SDK
• Comet Panels User Interface
• Browse the Comet Panels Gallery

Panel code¶

To make a snappy user experience in the browser, write a Panel in JavaScript.

Create a new Panel from the Panels Gallery as described in Javascript Panel UI.

1. Go to the Panels Gallery on the Project view and click Create New. A screen similar to the following is displayed:

2. Write the code. There is some default code provided on the creation of a new Panel. If you prefer to remove it, simply select it all using Ctrl+A, then press Delete.

   Your new Panel code needs only two things:

   • A JavaScript class named MyPanel.
   • A draw(experimentKeys, projectId){} method that takes an array of Experiment keys and project ID.

That's it!

Comet has also provided a Comet.Panel class that lets you implement additional functionality. You use it like this:

class MyPanel extends Comet.Panel {
  draw(experimentKeys, projectId) {
    // Your code here
  }
}

In the tradition of introductory programming exercises, explore a "Hello world" Panel. Take the basic template from above, and add instructions to print "Hello, world!" in the panel. Use this.print() to display the message.

Also, use the HTML <h1> tag to make the message large and bold.

class MyPanel extends Comet.Panel {
  draw(experimentKeys, projectId) {
    this.print("<h1>Hello, world!</h1>");
  }
}

If you write (or copy and paste) the above code into the Panel Code section, and click the green arrow button (or press Ctrl+Enter) you should see the following:

Now you can save your Panel by clicking Save (or pressing Ctrl+S):

Name your Panel something like "Hello World", and click Save. The first time you save a new Panel, you'll be asked to capture a thumbnail of your Panel. You can either select a portion of the screen, or upload your own image:

If you click Done, you go back to the Panels Gallery. Now you can add a "Hello World" Panel to your project by clicking the Add button in the Panels Gallery. A window appears in which you can set some options (see the next section). For now, just press Done.

Your Project view should now include your "Hello World" Panel:

The next section describes further options.

Panel options¶

Imagine you wish to plot the metric "acc" in one chart, but the metric "loss" in another. To let you make slight variations like these, without changing code, Comet has made the Panels quite abstract, and provided you with Options to manipulate them.

To provide these options that can be changed when creating a Panel and without changing code, all you need is to define a JavaScript object in the MyPanel instance named options. Typically, you will define your options in the setup method, as shown below. These are default values that can be overridden by the user when creating a new instance of the Panel.

class MyPanel extends Comet.Panel {
  setup() {
    // Your setup code here
    this.options = {
      // Your default options here
    }
  }

  draw(experimentKeys, projectId) {
    // Your code here
  }
}

As a concrete example, change the "Hello World" Panel to include a setup() method with the option "message":

class MyPanel extends Comet.Panel {
  setup() {
    this.options = {
      message: "Hello, world!"
    }
  }

  draw(experimentKeys, projectId) {
    this.print(`<h1>${this.options.message}</h1>`);
  }
}

If you try this Panel, you'll see exactly the same result as before. However, there is a big difference: you can now add two instances of the "Hello World" Panel to your Project view, and make each one say something different -- merely by changing the options when you add the Panel to your Project:

Specifically, the steps are:

1. Create the Hellow World Panel, using the option "message".
2. Add one of these Panels to your Project.
3. Add another of these Panels to your Project, this time changing the message to something else.

Comet.Panel class¶

When you extend from Comet.Panel you get additional functions, including:

• this.print(object, newLine) - print text in main Panel area; newLine == true means follow with newline.
• this.getOption("name", default) - get a name from the options (returns null if not defined); return default if not found.
• this.clear(divId) - clear all of the print()ed items in the panel area, or give it the ID of a div to clear.
• this.drawOne(experimentKey) - conventional method for single Experiment drawing code.
• this.select(label, options, parentId) - create an HTML select widget for selecting widgets; calls this.drawOne(selectedItem).
• this.getColor(experimentKey) - get the color associated with this Experiment.

In addition, you can also define the following methods to gain additional functionality:

• update(experimentKeys, projectId) {} - method to define code that will not clear the panel, but can be used to update existing UI components (see below).

The Panel.draw() method¶

A Panel's draw() method receives two things:

• An array of Experiment keys. Each Panel can pick what Experiments to operate on.
• A Project ID

The array of Experiment keys is one of three things, in this order:

• If a Panel filter is set, the Experiment keys are all those matching the Panel filter.
• If a Project View filter is set, the Experiment keys are all those matching the Project filter.
• If no filter is set, the Experiment keys are those visible in the Project Experiment table.

Note that there are two ways in which an Experiment becomes visible in the Experiment table on the Project view:

• The Experiment must be on the current table page (the table shows a limited number of Experiments per page).
• The Experiment must not be hidden (the "eye" icon must not be grayed out).

If you want to always process all Experiments (or programmatically select a subset from all of them), you can instead use the Project ID to get all of the Experiments in a Project, like this:

draw(experimentKeys, projectId) {
  this.api.experimentsByProjectId(projectId).then(experimentKeys => {
    // process all Project experimentKeys here
  });
}

If you are processing the array of Experiment keys (experimentKeys) then be aware that these can change and a refresh will occur on any of these events:

• You change the Experiment table page limit (number of Experiments per page).
• You change, add, or remove the Experiment filter.
• You change any Experiment's hide or show status (either by clicking the "eye" icon in the Experiment table, or by selecting Experiments to hide through another Panel).

If you are instead processing the Experiments by using the projectId key, then you will probably want to flip the toggle on the Panel instance editor to make it "static":

URL links to Experiments

You can create URL links to Experiments from a Panel.

You can include any HTML, and URL links will work fine in a Panel. There is one thing to keep in mind: always provide a target for your href like this:

a href="" target="_blank">TEXT<a>

or

a href="" target="_top">TEXT<a>

You should use "_top" (replaces the current page) or "_blank" (opens a new tab) for the target attribute.

Note that Panels run in HTML iframes, and do not have access to your Comet account. For example, you aren't logged in in the iframe, and so your data is safe and secure.

The Panel.update() method¶

Here is a template using the Panel.update() method. Restructure the code so that the Panel.draw() creates a Plotly plot instance, and a place to cache a set of Experiment keys. Panel.update() is called whenever new data is received, but the update method only processes Experiments that it hasn't seen before.

class MyPanel extends Comet.Panel {
  ...

  draw(experimentKeys, projectId) {
    Plotly.newPlot(this.id, [], this.options.layout);
    this.cached_experiments = [];
    this.draw(experimentkeys, projectId);
  }

  update(experimentKeys, projectId) {
    experimentKeys = experimentKeys.filter(key => ! this.cached_experiments.includes(key));
    this.api
      .experimentMetricsForChart(
        experimentKeys,
        this.options.metrics,
        this.options.parameters
      )
      .then(results => {
        Object.keys(results).forEach(experimentKey => {
          ...
          this.cached_experiments.push(experimentKey);
          Plotly.addTraces(this.id, data);
        });
      });
  }
}

Note that if you don't have a Panel.update() method that the entire Panel HTML area will be cleared and Panel.draw() will be called each time new data is available. You can use the Panel.update() method to do more intelligent processing for a snappier user interface experience.

JavaScript API¶

The Comet.Panel class also creates an interface to Comet's JavaScript SDK through this.api. This has all of the methods needed to access your data, plus some that are especially useful in creating a Panel. There are a few methods of special interest here:

• this.api.store(name, data) - save to this Panel instance's persistent memory
• this.api.getFromStore(name) - get an item from this Panels' persistent memory

For more details on the complete library, please see Comet's JavaScript SDK.

In addition, there are a variety of Comet open source JavaScript libraries that you can use (and build upon) via the Resources Tab.

Single Experiment workaround¶

this.select() is designed to allow you to select a single Experiment from the Project, and works as follows:

class MyPanel extends Comet.Panel {
  // Define your draw method like this
  draw(experimentKeys, projectId) {
    if (experimentKeys.length > 0) {
      this.select("Select an experiment: ", experimentKeys);
      this.drawOne(experimentKeys[0]);
    }
  }

  drawOne(experimentKey) {
    // Insert code here to do something with one experiment key
  }
}

Debug¶

Note that console.log() displays the items in the Console portion of the user interface.

Full example¶

Here is a basic line chart example using Plotly:

class MyPanel extends Comet.Panel {
  setup() {
    // Your setup code here
    this.options = {
      // Your default options here
      metrics: ["loss", "acc"],
      parameters: [],
      layout: {
        showlegend: true,
        legend: {
          orientation: 'h',
        },
        title: {
          text: "My Panel Title",
        }
      },
    };
  }

  draw(experimentKeys, projectId) {
    // Your code here
    Plotly.newPlot(this.id, [], this.options.layout);
    this.api.experimentMetricsForChart(
      experimentKeys, this.options.metrics, this.options.parameters)
      .then(results => {
        Object.keys(results).forEach(experimentKey => {
          const name = this.api.experimentName(experimentKey).then(name => {
            results[experimentKey].metrics.forEach(result => {
              const data = {
                y: result.values,
                x: result.steps,
                name: `${name} ${result.metricName}`,
              };
              Plotly.addTraces(this.id, data);
            });
          });
        });
      });
  }
}

This example defines this.options that defines a list of metrics and parameters to chart, and a layout object. Also, it uses Plotly, so you can put anything there that Plotly can use. See Plotly for more information.

You can override any option when you instantiate the Panel on your own page. Therefore, you can easily use the same code for plotting multiple items by simply changing the options.

For more details on using Comet Panels, see Panels.

JavaScript tips¶

JavaScript is designed to give a performant experience on the web. If you adapt your coding style to 'the way of the JavaScript,' you will be rewarded with a fast-loading, snappy experience. But it does take a little bit of extra work.

Comet's JavaScript SDK is written using Promises. These are JavaScript objects that represent a promise to complete some code. The code might be finished already, or you might have to await for it to complete. You can do whatever you would like, but JavaScript has some simple syntax for running multiple functions in parallel.

Take a look at some examples. Often, you may want to get some data, and then process it, and finally display it. Ideally, you can do this all in parallel. The example from above does exactly that. Consider that code again:

class MyPanel extends Comet.Panel {
  ...
  draw(experimentKeys, projectId) {
    // Your code here
    Plotly.newPlot(this.id, [], this.options.layout);
    this.api.experimentMetricsForChart(
      experimentKeys, this.options.metrics, this.options.parameters)
      .then(results => {
        Object.keys(results).forEach(experimentKey => {
        const name = this.api.experimentName(experimentKey).then(name => {
          results[experimentKey].metrics.forEach(result => {
            const data = {
              y: result.values,
              x: result.steps,
              name: `${name} ${result.metricName}`,
            };
            Plotly.addTraces(this.id, data);
          });
        });
      });
    });
}
}

Some important things to note:

• Line 5 displays an empty Plotly chart, which will get added to in parallel.
• Lines 6 and 10 represent Promises that are followed by a .then() method that says what to do after the promise is resolved.
• Line 17 adds the result of each individual chain of processing to the chart.

This is the ideal method of JavaScript processing for the web. However, you can't always work in such a style. For example, if you wanted to find the average of all of a particular metric, then you must collect all of those metrics first. Here is an example of how to do serial processing:

class MyPanel extends Comet.Panel {
  ...
  async draw(experimentKeys, projectId) {
    const data = await this.collectData(experimentKeys);
    this.print(`<p>Average: ${avg(data)}</p>`);
  }
}

Things to note:

• Line 3 has added the async keyword in front of the draw() method. This is necessary for any function or method that needs to await on a result.
• Line 4 has an await keyword in front of a method call.

That's it, for the structure. Now, take a look at how collectData() could be written. Consider that you have logged an "other" value (not a metric or parameter) using experiment.log_other("x", value) for all of the Experiments in this project. Now, fetch all of those values, and average them.

async collectData(experimentKeys) {
  const values = await Promise.all(
    experimentKeys.map(experimentKey => {
      return this.api.experimentOther(experimentKey, "x");
    })
  return values.filter(v => typeof v == "string");
  );

Things to note:

• Line 1, again, add async.
• Line 2, perform an await on a Promise.all(item.map())- a very common pattern.
• Line 6, a bit of technical detail, to filter out NaN and other bad values.

The last example blocks to get all of the values, before proceeding. This is different from the first example. If your code takes a while to process, you can put a this.print("<h2>Processing...</h2>") before the collection, and this.clear() after it.

For more examples, see the shared code examples in the Gallery.

Work with Plotly¶

Plotly is a free and open source JavaScript library built on d3.js and stack.gl.

As shown above, for performance reasons, you typically want to create the Plot and incrementally add traces to it.

Plotly has many options, such as limiting the X and Y axis using the layout.xaxis.range and layout.yaxis.range values, like this:

class MyClass extends Comet.Panel {
  draw(experimentKeys, projectId) {
    const data = ... // get some data here
    const layout = {
      xaxis: {range: [1, 10]},
      yaxis: {range: [5, 7]}
    };
    Plotly.newPlot(this.id, data, layout);
  }
}

Here are some of the most useful links for working with Plotly:

• Chart Fundamentals
• Basic Charts
• Statistical Charts
• Scientific Charts
• 3D Charts
• Subcharts
• Chart Events
• Animations

Work with other JavaScript libraries¶

There really is no limit to what you can build with Comet Panels. Here are some additional links to JavaScript libraries that you might find useful:

• chartjs.org
• d3.js
• Google Charts
• highcharts.com
• zingchart.com

Standalone JavaScript example¶

To use Comet’s JavaScript SDK programming standalone interface, you must first get a Comet API key.

To use the JavaScript SDK outside of Comet Panels, you must load the library. You can do that in HTML, so:

<script src="https://cdn.comet.com/js/comet-javascript-sdk-latest.js">
</script>

Once the Comet JavaScript library is loaded, then you can use it as described in the previous section. Here is a brief example (substituting your Comet API key in the following):

const comet_api = new Comet.API("COMET_API_KEY");

comet_api.experimentMetricsForChart(
  experimentKeys, ["metric1", "metric2"], ["param1", "param2"])
      .then(results => {
        Object.keys(results).forEach(experimentKey => {
          const name = comet_api.experimentName(experimentKey).then(name => {
            results[experimentKey].metrics.forEach(result => {
              const data = {
                y: result.values,
                x: result.steps,
                name: `${name} ${result.metricName}`,
              };
              // do something with data
            });
          });
        });
      });

Learn more¶

• Comet's JavaScript SDK
• Comet Panels User Interface