All twelve graded cases, wired to the golden set. Every press of RUN draws ten
samples per case FROM THE RECORDED RUNS, with replacement, which is the point: green
is a sample, not a property, and the observed rate wobbles around the true one.
Click a case for its full record. Switch eval versions to watch the fix loop work,
and watch what it cannot fix.
EVAL VERSION
·pass@1 · share of all runs that succeed
·pass^10 · cases clean in ten of ten
·worst case this sample
CASE
WHAT IT IS · RUTH'S CALL
RUNS (THIS SAMPLE)
Press RUN. The bench reports nothing it has not sampled.
GRADED CASE
THE CASE
THE DATA SAMPLE · WHAT THE AGENT CAN AND CANNOT READ
INSIDE RETRIEVAL SCOPE
OUTSIDE RETRIEVAL SCOPE
RUTH'S ADJUDICATION
RECORDED PERFORMANCE · TEN RUNS PER VERSION
VERSION
SCORE
WHAT CHANGED
CF-031Wired to canon · sampling draws from the recorded runs