7 Simple Steps to Standardizing the ML Experiment (Feb. 2)

Welcome to another recap of the Comet ML Office Hours, powered by The Artists of Data Science! This week we’re covering Session 5 of our new series. This session took place Feb. 2nd, 2022 and we were joined by Dr. Santona Tuli of Astronomer, Susan Shu Chang of Clearco, and Dr. W. Ronny Huang of Google AI.

As a reminder, we’d love to see any and all of you at these fifty minute sessions—so feel free to register for upcoming Office Hours sessions here! As always, there’s a lot more in the full session (which you can find on our YouTube channel), so be sure to check it out, alongside clips from roundtables, webinars, and previous Office Hours.

ML Like the Pros

This week we focused on what it takes to manage ML projects like the experts—specifically the experts who have worked across the entire ML lifecycle from research and startup projects to huge enterprise companies deploying models into production at scale.

Host Harpreet Sahota asked each of the panelists what their day-to-day workflow looked like. While it was unanimously agreed that there’s no such thing as ‘typical,’ there were a few common threads as seen in this clip of Dr. Santona Tuli:

Team Structure and Responsibilities

All of the panelists agreed that a key step in managing ML pipelines meant managing responsibilities and outlining clear ownership of each project and step of the process. Susan Shu Chang also mentioned that an effective team structure means that each team member gets to show off their “superpower.”

Hear more in the clip below:

Correct Behavior in Production

Once the team has deployed a model into production, it’s important to ensure that that model is behaving as expected.

To address these concerns, Dr. Tuli suggested responding to alerts quickly and efficiently and constantly checking up on the model. From a researcher’s prospective, Dr. Huang mentioned using tools before deployment to test the model on near-to-life synthetic data to work out any issues prior to production.

Check out the clip below to hear from both panelists.

Resources

Check out Susan’s website for more of her work. As always, there’s more to be discussed and discovered. Check out Comet’s contributor-led publication Heartbeat as well as our YouTube, Twitter, and LinkedIn for more great information.

• Comet Office Hours Bookshelf
• Office Hours YouTube Playlist
• Follow Comet ML on Twitter
• Connect with Comet on LinkedIn

Curious to Learn more? Join Us!

We run these virtual Office Hours every Wednesday at 11am EST (New York, NY). Completely free to attend and participate, and we’d love to see any and all of you there! We’ve got a great series planned and welcome questions for Harpreet or any of our guests via email to emilie@comet.ml.

But most importantly:

Register for Comet Office Hours