Integrate with New Relic¶
Comet integrates with New Relic.
New Relic is a full stack monitoring platform that allows you to visualize, analyze, and optimize your entire software stack from one place. With over 300 agents and integrations, you can ingest and store all your operational data, including logs, in one place.
Comet partnered with New Relic to extend full stack observability to include your machine learning models in production and their performance baselines established in training, so you can build better models, faster.
Data available in New Relic¶
Once the integration has been configured by your Comet customer success manager, you will get access in New Relic to the following metrics for each model:
| Metric | Description |
|---|---|
comet.hourly.nb_predictions | Number of predictions made by the model |
comet.hourly.input.data_drift | Data drift broken down by input feature |
comet.hourly.prediction.value | Distribution of the model’s prediction |
comet.hourly.prediction.probability | Distribution of the model’s prediction probability. Available percentiles appear in the following rows. |
comet.hourly.prediction.probability.min | Minimum |
comet.hourly.prediction.probability.p10 | 10th percentile |
comet.hourly.prediction.probability.p25 | 25th percentile |
comet.hourly.prediction.probability.p50 | 50th percentile or median |
comet.hourly.prediction.probability.p75 | 75th percentile |
comet.hourly.prediction.probability.p90 | 90th percentile |
comet.hourly.prediction.probability.max | Maximum |
Use Comet metrics in New Relic¶
Comet metrics can be used to create new dashboards or augment existing ones. To get started, you can use the default Comet dashboard to start tracking the performance of your machine learning models in New Relic.
In addition, you can use Comet metrics with New Relic infrastructure metrics to correlate the performance of your machine learning models with the performance of their underlying infrastructure.
Debug machine learning models with New Relic and Comet¶
Comet metrics can be used as part of New Relic Applied Intelligence to find, troubleshoot and resolve infrastructure issues that could be impacting the performance of your machine learning models.
If the issue with the performance of your model is not related to underlying infrastructure, you can use the Comet MPM dashboard to identify performance issues on subsets of your data using MPM Segments. You can also use MPM Log Search to query individual predictions and identify outliers.
Get started¶
Already a Comet customer monitoring models in production? Setting up the integration with New Relic One can be completed quickly. Simply reach out to your Comet customer success manager to complete the onboarding.
Not a Comet customer yet? Contact Comet today to set up an account and get started.