LLM-as-a-Judge Bias: How to Detect and Correct It

A model grader is the only practical way to score open-ended output at volume, and it is also a measurement instrument with known, documented, reproducible faults. The faults are not subtle. They are large enough to reverse a comparison. The good news is that each one has a published detection method and a mitigation that costs little more than a second API call.

The symptom

The pattern that should trigger suspicion: a judge reports a clear preference between two candidates, the preference does not survive small changes to how the comparison was presented, and nobody looked because the aggregate number seemed plausible. An eval that quietly reverses when the candidates are swapped is not producing a measurement at all.

Four failure modes account for most of it.

Position bias

Presenting two candidate answers to a judge and asking which is better introduces a preference for one of the two slots that has nothing to do with the content. Wang and co-authors documented this directly in Large Language Models are not Fair Evaluators: presenting the same pair of responses in the opposite order can flip the verdict, and the effect is strong enough that a weaker candidate placed in the favoured position can be scored above a stronger one. Zheng and co-authors report the same effect in the MT-Bench work and treat it as one of the three headline limitations of the approach.

Detection. Run every pairwise comparison twice, once in each order. Count how often the two runs agree. That consistency rate is a direct measurement of how much position bias your judge and prompt are carrying.

Correction. Score both orders and count only the consistent verdicts, treating inconsistent pairs as ties. The MT-Bench work proposes exactly this, and the Fair Evaluators paper proposes balanced position calibration together with multiple-evidence calibration as a more systematic version. The cost is a doubling of judge calls, which is almost always worth paying because the alternative is a number that means nothing.

The related and cheaper option is to avoid pairwise comparison entirely. Grading each candidate independently against a fixed rubric removes the slot problem by construction, at the cost of a coarser signal.

Verbosity bias

Judges prefer longer answers. Zheng and co-authors identify verbosity bias as a distinct failure mode, in which a longer response is rated higher without being more correct or more useful. This one is dangerous in a specific way: if the eval is being used to select a prompt or a model, and the judge rewards length, the optimisation loop will produce a system that pads.

Detection. Log response length alongside every judge score and check the correlation. A strong positive correlation between token count and score, on a task where length should not matter, is the whole diagnosis.

Correction. State length neutrality explicitly in the rubric, and grade the specific properties you care about rather than overall quality. “Does the answer contain the account number from the source document?” cannot be won by padding. “Is this answer good?” can. Where length genuinely varies for legitimate reasons, controlling for it in the analysis is more honest than pretending the correlation is not there.

Self-preference bias

A judge tends to favour outputs produced by itself or by models in its own family, an effect Zheng and co-authors name self-enhancement bias. The practical consequence is straightforward: never use the same model as both the generator under test and the grader, and be careful comparing two models when the judge is one of them or shares a lineage with one of them.

Detection. Include a small set of items with known-correct human labels covering output from every model in the comparison, and check whether the judge’s error rate differs by which model produced the text.

Correction. Grade with a model outside the families being compared. Where that is impossible, run the same suite through two independent judges and report the disagreement rate as part of the result rather than picking whichever one is more convenient.

Score clustering

Asking a model for an integer score out of ten produces a distribution piled on a few values, typically 7 and 8, with almost nothing at the extremes. That compresses the range you are trying to measure across.

The G-Eval work of Liu and co-authors addresses this in two steps: the judge first generates chain-of-thought evaluation steps from the criteria, then fills in a form, and the final score is computed as a probability-weighted sum over the possible score tokens rather than the single sampled token. The weighted version produces a continuous score and recovers resolution that the discrete sample throws away.

Where token probabilities are not available, the practical substitutes are a shorter scale with concrete anchors, for example a three-point scale where each point is described in terms of an observable property, or decomposition of the judgement into several binary checks that are then summed. Binary checks are almost always more reliable than a holistic score, because the judge is being asked a question with a fact-shaped answer.

Calibrating against humans

None of the above tells you whether the judge is right, only whether it is stable. That requires human labels.

The procedure is a fixed cost paid once per rubric. Draw a stratified sample of items, ideally a couple of hundred spanning the range of quality the judge will see. Have humans label them against the same rubric the judge receives. Then compute agreement between judge and human, using a chance-corrected statistic such as Cohen’s kappa rather than raw percentage agreement, because raw agreement looks impressive on any skewed distribution.

The benchmark to compare against is human-to-human agreement on the same items, not perfection. The MT-Bench work reports GPT-4 agreeing with human preferences at above 80 percent, which is the level of agreement humans reach with each other, and that framing is the right one: a judge that agrees with people as often as people agree with each other is doing the job.

If judge and human agreement is well below the human-human ceiling, the rubric is usually the problem rather than the model. Rubrics fail when they ask for a global impression, when their criteria overlap, or when they use words such as “helpful” and “clear” without saying what would make an answer fail.

Repeatability and error bars

A judge run at nonzero temperature is a sampler, so the same item scored twice can produce two scores. Two things follow.

Fix decoding parameters for the judge and pin the judge model version. A provider silently updating the model behind an alias changes the instrument mid-experiment, and the resulting score movement will be attributed to whatever else changed that week.

Report uncertainty. Miller’s Adding Error Bars to Evals sets out the statistical treatment for language model evaluations: an eval score is an estimate from a finite sample, it carries a standard error, and comparisons between two systems need to account for both sampling variance and the fact that the same items are being reused across systems. A difference smaller than the confidence interval is not a result. This applies to model-graded scores exactly as it applies to the benchmark scores discussed in LLM benchmarks explained.

For a first pass on whether an eval set is even large enough to resolve the difference you care about, the Pass@k and evaluation cost calculator reports the worst-case margin of error for a given item count alongside the token cost of the run.

When not to use a judge at all

A model grader is the fallback for things no assertion can check. Where an assertion can check it, the assertion wins on every axis: cheaper, deterministic, reproducible, and free of all four biases above. Valid JSON, a schema match, code that compiles and passes tests, an extracted field equal to a reference string, a citation that resolves to a real document. Any of these should be a programmatic check.

The useful arrangement is layered. Programmatic checks handle everything they can, a calibrated model judge handles the open-ended remainder, and a small human-labelled set recalibrates the judge whenever the rubric or the judge model changes. That layering is part of the broader design covered in designing an LLM evaluation that actually tells you something, and the frameworks that implement model-graded metrics are compared in LLM eval frameworks compared.

Checklist

• Judge model and version pinned, decoding parameters fixed.
• Judge is not the model under test and not from its family.
• Pairwise comparisons run in both orders; inconsistent verdicts counted as ties.
• Response length logged and checked against score.
• Rubric decomposed into concrete, testable criteria rather than a global impression.
• Agreement with human labels measured with a chance-corrected statistic on a held-out sample.
• Per-item outputs logged, not just the aggregate, so regressions are diffable.
• Confidence intervals reported on every comparison.

A judge that survives all eight is an instrument. One that survives none is a random number generator with good prose style.