LLM Juries for Evaluation
Evaluating the correctness of generated responses is an inherently challenging task. LLM-as-a-Judge evaluators have gained popularity for their ability to…
G-Eval for LLM Evaluation
LLM-as-a-judge evaluators have gained widespread adoption due to their flexibility, scalability, and close alignment with human judgment. They excel at…
BERTScore For LLM Evaluation
Introduction BERTScore represents a pivotal shift in LLM evaluation, moving beyond traditional heuristic-based metrics like BLEU and ROUGE to a…
Intro to LLM Observability: What to Monitor & How to Get Started
While LLM usage is soaring, productionizing an LLM-powered application or software product presents new and different challenges compared to traditional…
Building ClaireBot, an AI Personal Stylist Chatbot
Follow the evolution of my personal AI project and discover how to integrate image analysis, LLM models, and LLM-as-a-judge evaluation…
Perplexity for LLM Evaluation
Perplexity is, historically speaking, one of the "standard" evaluation metrics for language models. And while recent years have seen a…
Building a Low-Cost Local LLM Server to Run 70 Billion Parameter Models
A guest post from Fabrício Ceolin, DevOps Engineer at Comet. Inspired by the growing demand for large-scale language models, Fabrício…
