LLM Evaluation & Testing — Advanced

LLM-as-judge biases and debiasing, G-Eval, statistical testing, RAG triad internals, component vs end-to-end eval, agent evaluation, drift detection, A/B testing, red-teaming, eval-harness design.

Hard30 questions

Sample questions

In pairwise LLM-as-judge evaluation, position bias causes the judge to favor whichever response appears first by 10-15%. What is the standard technique to mitigate this bias?

Train a secondary classifier on past judge decisions to detect and subtract the positional preference from every final score before reporting
Insert a calibration paragraph instructing the judge to weight later content more heavily to counteract the first-position preference
Evaluate both orderings and only accept the result when the judge gives a consistent winner across both presentations
Randomly shuffle the two responses five times and take the majority vote, which statistically cancels any single-ordering preference

Why

Position bias is a documented systematic tendency where LLM judges prefer whichever response is presented first in a pairwise comparison, with effect sizes of 10-15% preference shift. The standard mitigation is to evaluate both orderings: present response A first in one round and response B first in the next, then only accept the result if the judge picks the same winner in both orderings. When the judgments disagree across orderings, the result is marked as a tie, indicating the preference was likely driven by position rather than quality. Training a secondary classifier adds complexity and requires labeled calibration data that is itself subject to bias, making it impractical. A calibration paragraph telling the judge to favor later content simply inverts the bias rather than eliminating it. Shuffling five times and taking a majority vote does not address the root cause because the same positional bias applies in every round where the ordering is the same, and random shuffling does not guarantee equal representation of both orderings. The two-ordering approach is both simple and effective because it directly controls the confounding variable. This technique is widely adopted in frameworks like Chatbot Arena and reported in the Zheng et al. 2024 study on LLM judge biases.

Evaluating LLMs & LLM-as-a-judge (eugeneyan)LLM-as-a-Judge — Hugging Face Cookbook

You are calibrating an LLM judge against human annotators on a 1-5 scale for faithfulness. You compute a weighted Cohen's kappa of 0.58 and a Pearson correlation of 0.82. What does this combination indicate about the judge's readiness for production use?

Both metrics are above typical production thresholds, so the judge is ready to deploy without modification and can replace human annotators entirely
The high Pearson r means the judge tracks human scores directionally but the moderate kappa reveals it struggles with the correct score level, warranting rubric refinement rather than more judge calls
The kappa indicates moderate agreement that falls below the 0.75 production threshold, suggesting the rubric needs more distinct score-level anchors despite the directional correlation being adequate, which would add complexity without addressing the fundamental rubric design problem
The Pearson r is unreliable for ordinal data, so only kappa matters, and the 0.58 kappa means the judge should be discarded in favor of direct human evaluation

Why

Cohen's kappa and Pearson correlation measure different aspects of agreement. Pearson r of 0.82 indicates the judge's scores move in the right direction with human scores, meaning when humans rate something higher, the judge does too. However, weighted kappa of 0.58 falls in the 'moderate' range, below the 0.75 threshold recommended for production use, indicating the judge frequently assigns scores that differ from human annotators by more than one level. This specific combination — high correlation but moderate kappa — is a classic sign that the rubric's score levels are not distinct enough for the judge to discriminate between them reliably. The fix is to improve the rubric by adding more concrete anchor examples at each score level and making adjacent levels more distinct, not to increase the number of judge calls. Discarding the judge entirely would be premature because the high Pearson r shows the underlying signal is sound. Claiming both metrics are above threshold is wrong because kappa at 0.58 falls well below the 0.75 standard. Pearson r is valid for ordinal scales when used alongside kappa, so dismissing it outright is incorrect. The practical outcome is that rubric iteration is the most cost-effective path to production readiness.

Evaluating LLMs & LLM-as-a-judge (eugeneyan)Cohen's Kappa Score — scikit-learn

LLM judges exhibit verbosity bias, where longer outputs receive higher scores regardless of quality. Which mitigation most directly addresses this bias without distorting the evaluation signal?

Truncate all candidate outputs to a fixed token count before presenting them to the judge so that length variation is eliminated from the comparison so that no option text can influence the judge's assessment through its presentation style
Multiply each judge score by the inverse of output length to mathematically penalize longer responses in the final reported metric
Use embedding similarity instead of LLM-as-judge for all evaluations since embedding models are inherently insensitive to response length
Include an explicit instruction in the rubric that length should not influence scoring and that a concise correct answer is better than a verbose one

Why

Verbosity bias is a documented phenomenon where LLM judges rate longer outputs higher regardless of actual quality, with a correlation of r=0.30-0.45 between output length and score in uncalibrated settings. The most effective mitigation that preserves the evaluation signal is adding an explicit rubric instruction that length should not influence scoring and that concise correct answers should score higher than verbose ones. This works because LLM judges are instruction-following models that respond to clear directives about scoring criteria. Truncating outputs to a fixed length destroys information and penalizes legitimately detailed answers that require longer explanations, corrupting the evaluation. Multiplying scores by the inverse of length introduces a mathematical penalty that over-corrects for length, potentially making brief but wrong answers score higher than detailed correct ones. Embedding similarity avoids the verbosity issue but loses the ability to evaluate subjective dimensions like helpfulness, coherence, and reasoning quality that LLM judges excel at. The rubric-based approach has been shown to reduce the length-score correlation from r=0.35 to r=0.08-0.12 in practice. This is why well-designed rubrics are the foundation of reliable LLM-as-judge evaluation, and why rubric engineering is considered the highest-leverage investment in evaluation quality.

Evaluating LLMs & LLM-as-a-judge (eugeneyan)Building LLM apps for production (Chip Huyen)

When using an LLM as a judge, self-preference bias means the judge model rates its own outputs approximately 8% higher than equally-rated outputs from other model families. What is the recommended mitigation for production evaluation systems?

Use the same model for both generation and judging to keep the bias consistent across all evaluated outputs, making relative comparisons valid
Average scores from three different judge models to cancel out each model's individual self-preference through ensemble voting
Use a judge from a different model family than the production model so that self-preference does not inflate the scores of the system under test
Subtract a fixed 8% from all judge scores to mathematically correct for the documented self-preference effect size

Why

Self-preference bias is a documented phenomenon where an LLM rates outputs generated by its own model family higher than outputs of comparable quality from other families, with GPT-4 showing approximately 8% higher preference for GPT-4 outputs over equally-rated Claude outputs. The recommended mitigation is cross-model judging: if your production system uses GPT-4o, judge with Claude Sonnet or vice versa, ensuring the judge has no systematic preference for the production model's style. Using the same model for both generation and judging preserves the bias and makes it impossible to detect quality regressions that happen to align with the model's preferred style. Ensemble voting across three models is expensive and does not guarantee cancellation of biases since self-preference is not evenly distributed across all models. Subtracting a fixed 8% is too crude because the actual effect size varies by task, output style, and model version, making a constant correction unreliable. The cross-model approach is practical because it requires only one judge model and eliminates the most direct form of self-preference. This principle extends to A/B testing where comparing two versions of the same model should use a third-party judge. In practice, this is the most commonly recommended approach in production evaluation frameworks.

Task-specific LLM evals (eugeneyan)Evaluating LLMs & LLM-as-a-judge (eugeneyan)

G-Eval uses chain-of-thought prompting and form-filling to score LLM outputs on custom criteria. What distinguishes G-Eval from a standard rubric-based LLM judge?

G-Eval asks the judge to generate detailed evaluation steps before scoring and weights the final score by token-level probabilities of each score value, rather than taking the raw output at face value
G-Eval fine-tunes the judge model on human annotations for each evaluation criterion so that scoring is learned rather than prompt-based, making it a supervised approach rather than a zero-shot evaluation method that can generalize across tasks
G-Eval replaces the single judge model with an ensemble of three smaller models that vote on each score independently and resolve ties by majority
G-Eval converts all evaluation criteria into binary pass/fail checks that are aggregated into a composite score through weighted averaging

Why

G-Eval is an evaluation framework that improves on standard LLM-as-judge by combining chain-of-thought reasoning with token-level probability weighting. In a standard rubric-based judge, the model is prompted to output a score directly, and that single output is taken at face value. G-Eval instead prompts the judge to first generate detailed evaluation steps for the given criteria, then produce a score, and finally weights the score by the token-level probabilities of each possible score value rather than taking only the argmax. This probability-weighted approach captures the judge's uncertainty: if the model assigns 60% probability to score 4 and 40% to score 3, the G-Eval score is 3.6 rather than a hard 4. G-Eval does not require fine-tuning, which would be expensive and require task-specific training data. It is not an ensemble method, which would triple compute costs. It does not convert criteria to binary checks, which would lose the granularity needed for subjective dimensions. The chain-of-thought component forces the judge to reason through each criterion explicitly before scoring, reducing the tendency to anchor on surface features. G-Eval achieves higher correlation with human judgment (Spearman 0.51-0.55 on summarization) compared to standard rubric prompting (0.40-0.45) by capturing both the reasoning and the uncertainty in the evaluation.

Task-specific LLM evals (eugeneyan)HF Evaluate library

Free account

Take the full module

These are the first few of 30 questions. A free account opens the rest as a scored drill.

Every question in this module
Instant feedback and supporting reading
Your score and progress, saved

Free · your email is used for progress only.

About Nybble^™

The AI space moves fast.
Nybble^™ is how you keep up — and stay sharp.

Snack

What happened. In two minutes.

The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.

Go to Snack

Stack

The concepts behind the headlines.

News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.

Go to Stack

Hack

Prove you actually get it.

Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.

Go to Hack