7 Simple Steps to Standardizing the ML Experiment (Feb. 9)

Welcome to another recap of the Comet ML Office Hours, powered by The Artists of Data Science! This week we’re covering Session 6 of our new series. This session took place Feb. 9th, 2022 and we were joined by Dr. Doug Blank from Comet and Tiffany Fabianac of AstraZeneca.

As a reminder, we’d love to see any and all of you at these fifty minute sessions—so feel free to register for upcoming Office Hours sessions here! As always, there’s a lot more in the full session (which you can find on our YouTube channel), so be sure to check it out, alongside clips from roundtables, webinars, and previous Office Hours.

Reproducibility, Reproducibility, Reproducibility

This week we focused on how experiment management helps you iterate, build, and deploy models faster. The key takeaway from this week’s panel was reproducibility. Knowing the exact hyperparameters, tracking each change, and generally building reproducibility into your standard practice is vital.

When asked about building reproducibility, both panelists suggested melding it into the culture of the company or workflow. Doug suggested focusing on protecting against irreproducibility as true reproducibility is a myth. Hear more in the clips below:

Understanding the Model

Tiffany also suggested that rather than focusing on just being able to copy an experiment, practitioners should focus on maintaining their understanding of the model and the problems that it solves. As data will change and models will adjust, keeping the end-goal in mind can be a means of creating an iterative machine learning lifecycle.

Rogue Robots

As an example of how impossible it is to prepare for all outcomes, Doug told a story from his time teaching developmental robotics. The robot, which didn’t have memory, was supposed to travel down a hall and turn based on the color of the light.

What happened was a memory-less robot teaching itself on the job! Hear the full story in the clip below:

Resources

During the session, Doug mentioned the book The Extended Mind by Annie Murphy Paul.

As always, there’s more to be discussed and discovered. Check out Comet’s contributor-led publication Heartbeat as well as our YouTube, Twitter, and LinkedIn for more great information.

• Comet Office Hours Bookshelf
• Office Hours YouTube Playlist
• Follow Comet ML on Twitter
• Connect with Comet on LinkedIn

Curious to Learn more? Join Us!

We run these virtual Office Hours every Wednesday at 11am EST (New York, NY). Completely free to attend and participate, and we’d love to see any and all of you there! We’ve got a great series planned and welcome questions for Harpreet or any of our guests via email to emilie@comet.ml.

But most importantly:

Register for Comet Office Hours