Topics

Browse everything Evals ML publishes by category and tag: every topic covered here, with the most recent guides listed under each one.

Tags

#llm-evaluation 6
#benchmarks 5
#pass-at-k 3
#llm-as-a-judge 2
#model-grading 2
#ai-agents 1
#ci-cd 1
#code-generation 1
#eval-frameworks 1
#eval-metrics 1
#gsm8k 1
#gui-automation 1
#humaneval 1
#mmlu 1
#regression-testing 1
#vision-language-models 1

Categories

Benchmarks 1 post

LLM Benchmarks Explained: MMLU, HumanEval, GSM8K

What MMLU, HumanEval and GSM8K actually contain, how each score is computed, and the point where a headline benchmark number stops being useful.

Eval Design 1 post

LLM Evaluation Design: From Task to Regression Suite

How to design an LLM evaluation: task-grounded eval sets, grader choice, sampling variance, and the contamination traps that mislead benchmark scores.

Eval Tooling 1 post

LLM Evaluation Frameworks Compared: Task Fit and CI Gates

Compare LLM evaluation tools for CI gates, reproducible runs, judge control, trace linkage and licensing, with promptfoo and lm-eval configs.

Metrics 1 post

Pass@k Metrics Explained: Formula, Estimator, Pitfalls

Pass@k is the chance at least one of k samples is correct. How the unbiased estimator works, why pass@1 is the honest number, and when pass^k wins.

Model Grading 1 post

LLM-as-a-Judge Bias: How to Detect and Correct It

Position, verbosity and self-preference bias in model graders, how to measure judge agreement against humans, and the calibration steps that reduce each.

Research 1 post

GUI-Primitives: Spatial Reasoning Failures in Vision Models

GUI-Primitives tests spatial reasoning in vision-language models for GUI grounding, finding weak localization and 32% peak strict point-in-box accuracy.