7 Simple Steps to Standardizing the ML Experiment (Jan. 19)

Welcome to another recap of the Comet ML Office Hours, powered by The Artists of Data Science! This week we’re covering Session 3 of our new series. This session took place Jan. 19th, 2022 and we were joined by researcher and professor Dr. Angelica Lo Duca.

As a reminder, we’d love to see any and all of you at these fifty minute sessions—so feel free to register for upcoming Office Hours sessions here! As always, there’s a lot more in the full session (which you can find on our YouTube channel), so be sure to check it out, alongside clips from roundtables, webinars, and previous Office Hours.

Defining The Baseline

The way to tell if you’ve created a good experiment is by using baselines. Baselines tell you if your model is performing accurately and how much improvement you’re brining to the problem with your ML model.

A good baseline is three things: Simple, fast, and repeatable. Learn more in the clip below:

The Most Important Thing? Tracking!

Our special guest Dr. Angelica Lo Duca has written a number of articles on workflow and highlights the importance of a standardized workflow and consistently tracked data. Just as we discussed last week, logging each and every step is the difference in creating a reproducible (and therefore standardized) experiment.

Reproducibility was one thing that drew Dr. Lo Duca to Comet. Check out the clip below to hear her reasoning.

Experimentation and Community

Throughout the discussion Harpreet and Angelica mentioned experimentation: Asking questions, iterating on models, trying out new tools and techniques. Constantly experimenting and playing is what makes models (and people) successful. Try different models, try different algorithms, investigate!

“We are a community,” Dr. Lo Duca explained, and communities share their knowledge. Though different people have different focuses, such as data journalism for her, it is important to learn from each other and constantly be evolving your practice and yourself.

This learning process can be difficult but both Harpreet and Angelica highlighted the importance of taking your time and taking breaks when studying. Hear more of their tips in the clip below.

Resources

Make sure to read Angelica’s articles on Heartbeat and other Medium publications. The work of Andrew Ng, Jason Brownlee, Emmanuel Ameisen, and Jose Berengueres was also mentioned.

As always, there’s more to be discussed and discovered. Check out Comet’s contributor-led publication Heartbeat as well as our YouTube, Twitter, and LinkedIn for more great information.

• Comet Office Hours Bookshelf
• Office Hours YouTube Playlist
• Follow Comet ML on Twitter
• Connect with Comet on LinkedIn

Curious to Learn more? Join Us!

We run these virtual Office Hours every Wednesday at 11am EST (New York, NY). Completely free to attend and participate, and we’d love to see any and all of you there! We’ve got a great series planned and welcome questions for Harpreet or any of our guests via email to emilie@comet.ml.

But most importantly:

Register for Comet Office Hours