State of the Art MLOps: Efficient Model Management with a Model Registry

The Power of a Model Registry 

A comprehensive Model Registry is one of the most overlooked components when machine learning teams are building out their MLOps toolset. A Model Registry serves as the bridge between the training and production phases of the model lifecycle. 

In Training, researchers are iterating across parameters and data, trying to find their champion model. Once found, the model is moved to a model registry. Only models stored in the registry see the light of production. 

While it may feel as if the Model Registry is just a file and versioning store, a comprehensive Model Registry offers a lot more. 

Comet’s Model Registry helps Machine Learning teams implement best practices by: 

1. Preserving the model context
2. Providing specific usage instructions
3. Capturing model history
4. Automating deployment.

1. Preserving the Model Context

The Model Registry is used by different personas at different stages of the Model Lifecycle. Oftentimes a model trained by a Data Scientist is handed off to a ML Engineer who is responsible for deploying and maintaining a model. Typically ML Engineers are  not involved in the model training process, giving room for miscommunication and confusion. To ensure a clean handoff, it is important that the Model Registry provides the complete context of how a model was trained. The following screenshot is a sample of how Machine Learning teams of all shapes and sizes utilize Comet’s Model registry today 

From the Model Registry above it’s very easy to answer the following questions

1. Who created this model? 
2. How long did it take to train this model?
3. Is this model in staging or production?
4. The link to the experiment that created the Model (Model Lineage)
5. What version of the data did I train this model with? 
6. What are the KPIs of this model?

Having a one-stop shop of all the relevant information for a model allows Machine Learning organizations to operate at high efficiency.

2. Providing Specific Usage Instructions

A model’s journey doesn’t stop at the Model Registry. Model’s are meant to be eventually downloaded and deployed to make predictions on real-time data. Oftentimes, the process for retrieving a production-grade model however is disjointed or undocumented . Comet reduces such friction by laying out explicit instructions on how to download a specific model version in different flavors (Command Line, REST, Python). 

3. Capturing Model History

It is inevitable that models will change over time. Data and Concepts drifts are some of the biggest reasons why models start to under-perform in production. To tackle this, teams often re-train their models on new data-sets to create newer and more accurate models. Model Versioning starts to become incredibly important in such situations. The History tab in Comet’s Model Registry allows users to see all the changes for a particular model.

Not only does the History Tab capture when a new model version was added or when a model was moved to staging, but it also can track who downloaded the model. This visibility is extremely useful if things go wrong. Say it’s known that a particular model version has a bug. The model maintainer can come to the History tab, see all the individuals who downloaded the model, and alert them of the issue.

4. Automating Deployment

Having a Model Registry that is integrated with your other tooling allows the team to automate their internal process. Comet’s Model Registry can be integrated with your deployment infrastructure through Webhooks. By simply changing a model’s stage, users can trigger their CI/CD pipelines automatically, reducing the need for manual human intervention.

Use a Model Registry Today!

Overall, a Model Registry can provide that single point of control for managing Machine Learning models, which helps to ensure that models are used in a consistent, compliant, and efficient manner. It’s imperative for organizations to treat their models as first class citizens.  It is as simple as adding a few lines of code with your model info and you are ready to get started.

Check out the example below and if you’re ready to start logging, check out our documentation!