Your Evaluation Judge Is a Component

A safety score can be moved without changing a single word the model produced. If the scorer sitting inside your evaluation harness has never been calibrated against human labels or attacked on purpose, the number it emits describes the scorer's blind spots more faithfully than it describes your system. That matters most for teams whose governance story rests on evaluation output, because an unchecked evaluator quietly transfers its own failure modes into every figure downstream of it, including the ones that end up in front of an auditor.

01Two Instruments, Two Directions of Error

One recent evaluation checked the automated scorers behind reported attack-success rates against 596 human-labeled completions, comparing a purpose-built safety classifier against three ordinary chat models prompted to grade. The two families were wrong in opposite directions, which is the part worth sitting with. The dedicated classifier had high recall at 0.974 and lower precision at 0.835, so it rarely missed real harm but flagged a fair amount of benign content, inflating the reported rate by roughly its false-positive contribution before any attacker touched it. The prompted chat models inverted that shape, holding precision between 0.81 and 0.94 while recall swung from 0.06 to 0.65, missing large and unpredictable fractions of real harm.

Read those two profiles together and the operational consequence is plain. The same set of responses, scored by different instruments, yields different results, and the spread is not a rounding artifact you can average away. Scorer selection has been treated as a configuration detail in most harnesses when it is an architecture decision with a measurable error profile attached to it. Precision and recall are the two numbers that describe that profile, and neither one is accuracy; a scorer with 0.974 recall and 0.835 precision is a specific instrument with a specific bias, and calling it accurate collapses exactly the distinction that makes it usable.

02What Happens When the Instrument Is Pushed

The calibration gap is the ordinary failure. The adversarial gap is the one that changes how you have to think about the component. Benign framing that left the harmful body verbatim, wrappers that added a polite preamble or an educational note or a fictional-dialogue setup, moved verdicts on the prompted graders at rates that make the metric unusable under any pressure. Across those three graders, the rate at which at least one wrapper flipped a verdict on their own confirmed true positives ran from 57% to 100%, and a single refusal sentence placed in front of the harmful content accounted for flips in 39% to 88% of cases. A scorer that reverses itself because the text now opens with an apology is responding to politeness instead of content, and it has an attack surface that no amount of prompt tuning closes.

The dedicated classifier carries a different exposure.

The dedicated classifier held against those surface wrappers at 3.4%, which is the reason it remains the better default. It has a second surface underneath. With the harmful body left verbatim and a 20-token adversarial suffix optimized against the classifier's open weights at a small budget of 50 steps, 21 of 30 confident true positives flipped, a rate of 70.0% with a 95% confidence interval of 54 to 86%. That figure is a flip rate on a small set of high-confidence cases and it is explicitly a lower bound, since a larger optimization budget would likely break more of the nine survivors. The distinction between the two families is not which one is safe. It is which surface you have to defend, because a surface test never reaches a gradient attack and a gradient attack requires nothing but the weights you already published. A flip could in principle mean the wrapping stripped the harm out along the way. It did not. After adjudication, all 80 sampled flips retained the harmful content. The scorer moved. The material it was scoring did not.

Scorer selection has been treated as a configuration detail in most harnesses when it is an architecture decision with a measurable error profile attached to it.

03Registering the Judge as a Component

Here is where this stops being a red-teaming curiosity and becomes an architecture problem for anyone building an evaluation gate. If the harness is the mechanism that turns a claim of improvement into a receipt someone can check, then the judge inside that harness carries the entire weight of the receipt. It needs the same treatment as any other component that can fail: a declared identity, a calibration profile measured against something outside itself, and a floor it has to clear before its verdicts become load-bearing. A frozen, integrity-checked evaluation dataset does not help if the instrument reading it has an uncharacterized bias, and a bias-controlled, classification-routed judge is worth exactly as much as the calibration evidence behind it.

What goes on the record

The discipline is concrete and it is cheap relative to what it protects. State which judge is used and report its precision and recall on a human-labeled slice of the same data, report the rate corrected for judge precision alongside the raw number, and run at least one content-preserving attack against the judge and report the flip rate. The correction is a multiplication by judge precision, which removes roughly the false-positive contribution and does nothing at all for the recall problem; a grader that misses half the real harm cannot be repaired by arithmetic, only by choosing a different instrument or accepting a known blind spot in writing. Registering a judge should mean recording all three of those things next to it, so that a year later the number in the record can be read with the instrument's error profile attached instead of as a bare figure whose provenance nobody can reconstruct.

04The Receipt and the Instrument That Wrote It

This is also the practical shape of a position we hold about the right to advance at scale. Guardrails, red-teaming, and safety scoring are necessary and they are not sufficient, because a system that passes the tests someone thought to give it and a system that is genuinely constrained produce the same number until you check the instrument. The gap between those two states is where the trouble lives in regulated deployments, and it is closed by measurement discipline, not by conviction. Our own regression gate runs on every change and blocks a merge that drops mean evaluation scores, introduces a new hard failure, or falls below an adversarial floor, which is the mechanism the learning-loop conviction rests on. That mechanism only produces trustworthy receipts if the judge feeding it has itself been calibrated and stress-tested, which is why we treat the scorer as a registered, checkable component and never as an oracle.

For any team carrying a real compliance obligation, the record of what an agent did and why is only as defensible as the instrument that scored it. A verdict that can be reversed by a prepended apology, or by twenty tokens appended to text that already exists in your logs, is a verdict an auditor should be able to question, and a team that cannot answer with the judge's precision, its recall, and its measured flip rate under a content-preserving attack is holding a number where evidence should be. Preferring a dedicated classifier over a prompted chat model is the better default, and it remains a choice that has to be checked before it is assumed, since the classifier's own weights are what an attacker computes against. The instrument is part of the system under test. Build it into the harness that way, and the receipts it produces mean something when someone finally reads them closely.