LLM-as-a-judge - a model grades another model’s output (or full trajectory) against a written rubric, instead of exact-match metrics like BLEU/ROUGE that ignore semantics.
Which judge shape
| Need | Prefer |
|---|---|
| Clear success condition | DAG (deterministic yes/no tree) |
| Subjective quality | G-Eval (steps then score) |
| Fact-check against sources | QAG (closed yes/no questions) |
Self-evaluation (same model scoring its own answer) is calibration, not introspection - Ren et al. showed it beats sequence-likelihood baselines on TruthfulQA and TL;DR.
On this site
The Claude Code /goal path is production LLM-as-a-judge with a hard gate: the evaluator only reads the transcript and a verifiable completion condition (transcript-only rule). Full pipeline sizing: Five metrics for an agent eval pipeline.