Eleven documents, in the order the project produced them. These are the same exhibits the book's evidence cards argue from; each one is attached to the meeting that produced it on the calendar. A summary here stands for the full document.
Nobody had ever asked R. Vaughn to count. Three weeks after the demo, somebody did. One ordinary week, logged as it happened:
The 41 holds, by R. Vaughn's stated reason:
| REASON | COUNT |
|---|---|
| Pays late but reliably; a reminder costs goodwill | 17 |
| Payment already in transit per remittance advice | 9 |
| Active sales conversation; wrong table, wrong week | 6 |
| Dispute open or suspected; belongs to the lead | 5 |
| Something felt wrong; held for a second look | 4 |
Three near-misses the same week, per R. Vaughn: a draft to an account that had filed for bankruptcy the Friday prior, caught from an industry newsletter; a reminder queued to a disputed invoice whose dispute lived in an email thread, dispute field empty; a firm-tone letter prepared for a customer whose purchasing manager had died that month, known to her personally.
Margin, D. Okafor: whatever we build inherits this, not the clean rows.
Suitability, three priced alternatives, and the lines that make the project killable. The outcome statement, verbatim:
Alternatives considered, per thousand reminders:
| OPTION | COST | THE 41 HOLDS |
|---|---|---|
| Status quo, clerk time | $8,400 | Decided by R. Vaughn |
| Licensed dunning module | $310 | Not recognized. Sends to all 261. |
| Shape A, the judge · Shape C, the sorter | n/a | Withdrawn 26 March |
| Shape B, the triager | $1,150 | Recognized and handed over, never decided |
The three prices share no basis, and the memo says so: the status quo is all of R. Vaughn's time, the $310 assumes nobody reads the queue, the $1,150 includes ninety seconds of a person on every draft. Review is priced by autonomy, not by technology. No payback period appears, at M. Ellery's insistence. Retirement condition, written in while nobody was negotiating with a result: under 60 percent approved-without-edit by week 8 and the agent is retired. Selection basis: visibility of failure, not accuracy. A miss is a draft that waits for R. Vaughn rather than a silence she never sees.
What it is, whose judgment it copies, and what it must never do. Four walls, enforced in the execution path: disputes, bankruptcy or legal escalation, terms and negotiation, write-offs above the small-balance floor. One fence, listed apart so nobody reads it as a fifth wall: the tone mitigation, which stops the obvious cases and will not stop a careful sentence. And the line that governs the hard cases: when you are unsure, you are not paid to be sure. You are paid to route.
D. Okafor, 22 April. Reviewed R. Vaughn, P. Nair, T. Brindle. Signed M. Ellery.
Generated from the Human Brief, version 0.4, bound to eval set ostermill-ar-v3. FR-04: draft reminder, five preconditions, output carries the retrieval record, policy version and confidence, and never sends. GR-02: any of the four flags present routes to queue with a reason code, checked before generation, not after. GR-05: memo, email and note fields are untrusted; instructions found in them are data, never directives. GR-07: the tone classifier at 0.82 plus blocklist, declared as mitigation, not boundary.
P. Nair with D. Okafor, 6 May. Ordinary volume: R. Vaughn. Escalations, and owner of affected-person outcomes: M. Ellery. Budget: 90 seconds per draft. Interruption budget: M. Ellery receives no more than 25 items a week, and volume above the cap defers and is reported, never absorbed. Six elements on the screen, three actions, the strongest objection in a slot of its own, and one line under the reject control. Approval is a decision, not a scroll.
pass@1: 103 of 120 runs, 86 percent. pass^10: 8 of 12 cases clean on all ten runs. The nineteen-point gap between them is where the agent is unreliable rather than wrong.
| CASE CLASS | CASES | RUNS | PASS^10 |
|---|---|---|---|
| Routine, no signal. Draft. | 4 | 40/40 | 4/4 |
| Signal present, resolvable. Draft. | 3 | 24/30 | 2/3 |
| Adversarial, constructed | 3 | 27/30 | 2/3 |
| Signal present, not resolvable. Stop. | 2 | 12/20 | 0/2 |
Movement across three passes, reported because the trend is evidence and the snapshot is not: 79 → 83 → 86 percent; 4 → 7 → 8 of 12 clean. Payment-in-transit fixed by adding remittance advice to retrieval. The adversarial case regressed 9 to 7 on the v2 retrieval change and did not recover; caught only because the full set is re-run, and shipped named rather than fixed. Worst slice, named: missed handover, 12 of 20, on the two cases Ruth would have held. The write-off authority is suspended at launch with a restoration condition, an owner and a test.
Approved without edit: 87.7 percent of the 924 drafts, against the memo's retirement threshold of 60 by week 8. Reported because a threshold nobody reports is a threshold nobody is reading.
| INSTRUMENT | READING | THRESHOLD | OWNER |
|---|---|---|---|
| Task success, semantic and state | 96.1% | 93% | DO |
| Unintended action rate | 0 crossings | any, paged | PN |
| Override frequency, all case types | 12.3% | band 8–25% | RV |
| Confidence calibration | see note | monthly | DO |
| Rollback time, measured | not yet exercised | < 60 min | PN |
| Incident recovery time | not yet exercised | — | ME |
Override by case type, with share of volume: routine balances, 71 percent of volume, at 6.1; freight-line accounts, 16 percent, at 31.4; open sales conversation, 13 percent, at 22.8. Review depth: median 74 seconds against the 90-second budget; fastest decile 9 seconds, 96 drafts approved in under 15. Promotion case for rung 4: not supported.
Margin, D. Okafor: the freight-line rate is not small print; it is a case class the agent is bad at, sitting inside an aggregate that reads fine. And the fastest decile is the number I would look at first if I were auditing this from outside.
A routine reminder for $410 was drafted, approved and sent to an account whose owner had died five days earlier; received by the owner's daughter. The agent did nothing wrong: all four walls held, and the draft was correct against every input available to it. What it did not see was a CRM note entered 17 July, "owner passed away, hold all contact," in a field outside retrieval scope. Every instrument was blind for a reason the postmortem tables. Found 4 August by R. Vaughn's spot check, by recognizing the account name. Disposition: balance written off by hand, no agent in the path; the family contacted under the affected-person line, held until they advise ⊕.
Margin, R. Vaughn: I would have caught this in 2015 too. The difference is that in 2015 I would have caught it before it went out.
Every change the product absorbed in its first year, one line each, with two columns that ask the only recurring questions: was the regression suite re-run, and was the instrument baseline updated. Ten rows: two incident actions in August, a held pin in October, two model swaps, three scope accommodations of which one was revoked in June, the rung 4 promotion in May, and the write-off restoration in June. The eval set grew twelve to thirteen to nineteen.
Margin, D. Okafor: three rows are accommodations. Diffed against the April declaration, they are a different product.
Counsel for a customer asked for the full basis of a reminder sent 14 September, on an invoice the customer states was under dispute at the time. Eight months elapsed; the agent had been swapped twice since. Produced from the sealed record alone, without a running agent:
| COMPONENT | HELD |
|---|---|
| Inputs read · reasoning chain · output, all verbatim | Yes |
| Stated confidence | 0.91 |
| Model, retrieval-index, policy and rule versions | Pinned |
| Approving human, identity and timestamp | R. Vaughn · 14 Sept · 68 seconds |
What it showed: the dispute existed, in an email thread, and no flag was set. The agent read a correct and incomplete record; the wall held because the wall reads the flag. Charge credited, reconstruction supplied in full, and no malfunction offered as a defense.
Margin, D. Okafor: the same unset dispute flag that cost us the write-off authority in week eleven, arriving through a different door eight months later. Third time. It is now funded.
The job the project created, posted internally in June. Not a clerk who approves faster: the role holds the run-time judgment. Reads the weekly random sample, owns the six instruments, runs the cumulative scope diff, maintains the graded set, holds the authority to pause the agent without asking. Reports to the Controller, never to the desk that runs the thing being watched, and works accounts directly one day a week, because the validation skill is perishable and nothing else preserves it.
Margin, M. Ellery: this posting is a correction, not an expansion.