Evaluating Large Language Models presents unique challenges due to their generative nature and lack of ground truth data. Traditional evaluation metrics used for discriminative models are often insufficient for assessing the quality, coherence, diversity, and usefulness of LLM-generated text. In this session, we will discuss several key considerations for evaluating LLMs, including qualitative analysis by human assessors, quantitative metrics such as perplexity and diversity scores, and domain-specific evaluation through downstream tasks. Additionally, we will discuss the importance of benchmark datasets, reproducibility, and the need for standardized evaluation protocols to facilitate fair comparison and advancement in LLM research.