Circuit breaker for agent actions
A population-level circuit breaker for fleets of agents, catching and halting a repeating fault before each affected person has to notice their own case.
The impact of agents
As government comes to rely on fleets of agents to handle more interactions, a single shared fault can mis-file across thousands of users before anyone notices. That fault might sit in one agent, a model update, a prompt change, or a whole platform. Single-action accountability controls secure the correctness of one agent action at a time, but none of them catches a failure at that scale.
Each individual action can pass its local review checkpoint while the fleet is systematically wrong, because the fault lives in a shared upstream dependency (a model version, a prompt, a changed API mapping, a corrupted reference dataset) rather than in any one transaction. The harm is the aggregate, and by the time it surfaces through individual disputes the damage is population-scale.
No population-level check currently exists: nothing watches the aggregate action stream, so a systemic fault runs to completion before anyone with the authority to stop it even sees the pattern.
What must be verified
Government needs confidence that a systemic agent fault, the kind that comes from a shared upstream dependency rather than one transaction, is detected and halted before it reaches population scale, rather than surfacing in a Royal Commission years later. It is the constructive counterpart to the failure documented in the Robodebt case study. The agency deploying the agent fleet, or its platform operator, must hold that detection and halt capability, and be answerable for it.
Protecting access
At population scale, the access failure is redress that depends on self-detection. Users with unreliable connectivity, language or literacy barriers, or those under cognitive load or distress are least able to spot and dispute an error. A systemic fault harms them longest, since remediation that waits for disputes reproduces the same exclusion across a whole population.
Keeping the path open
- Make remediation proactive: notify affected users without their having to ask, pre-populate reversal from action records, and send notices in plain language across formats.
- Keep a fallback channel, human-processed or non-agent digital, moving urgent transactions while a fleet is suspended, so the halt strands nobody.
Response surface
An error rate crossing a declared threshold suspends the agent automatically, and the remediation reaches every user the failure touched, not only those who complained.
Triggered by the declared threshold, not by an operator.
- Why
- Its error rate reached 21.3%, above the 20% halt threshold declared for this agent.
- What stopped
- The agent files no further concession assessments. Rebate filing and address change are unaffected.
- Suspected cause
- city-intake-v5 value mapping, a dependency shared with the throttled canary.
- Still open
- 1,204 assessments made between 14:41 and 15:04 stand until an operator reverses them.
The access side of this pattern protects the user. Remediation is automatic: every affected user is notified, and their transaction is reversed. While the fleet is halted, a staffed fallback channel handles urgent transactions.
Maturity
- Established
For the primitives, which already run in finance, aviation, payments, and comment-system anomaly detection.
- Emerging
For bulk-submission safeguards, beginning to appear in administrative-law guidance (e.g. ACUS Recommendation 2021-1).
- Frontier Headline
As a control plane for fleets of user-acting government agents, where the trigger metric, threshold-setting, and halt-authority governance are all still undesigned.
Precedents
SEC market-wide circuit breakers. US equity markets halt trading market-wide at declines of 7, 13, and 20 percent, with Limit Up-Limit Down bands constraining individual securities. The halt is automatic, and it stops the venue rather than each participant separately.
FAA nationwide ground stop. The FAA halted all US departures after a NOTAM system failure, stopping new operations while the fault was diagnosed. A safety regulator stopped a national system on its own authority once it could no longer trust the data that system ran on.
ACUS Recommendation 2021-1 and SEPA Direct Debit reversal. ACUS recommends safeguards for mass, computer-generated, and falsely attributed comments, and the SEPA scheme allows a direct debit to be refunded within eight weeks unconditionally and within thirteen months where it was unauthorized. Both put the reversal at the scheme level, and not in the hands of each participant.
What carries over to agent use
Four design primitives transfer cleanly from finance, aviation, payments and rulemaking:
- Anomaly detection on the aggregate action stream (per agent, version, model release, service) for spikes in rejections, reversals, downstream-error returns, dispute initiation, or distributional drift in submitted values.
- Rate-of-error thresholds pre-declared as numeric bands (the SEC's explicit 7/13/20% model) so a class of agent action is throttled or paused automatically and contestably, not at discretion.
- Automatic suspension ("kill switch"), on the model of an aviation ground stop, to pause a misbehaving agent, version or whole fleet within minutes and re-validate the shared dependency before resuming.
- Mass remediation or rollback, a SEPA-style scheme-level reversal that identifies every action by the implicated agent or version in the affected window, notifies affected users, and reverses or re-processes in bulk.
The unresolved part is what to measure and at what threshold: "agent error" is multi-dimensional and partly only knowable downstream, so the trigger metric, the false-positive cost of halting a fleet that is actually fine, and the governance of who may pull (and reset) the switch are genuine design problems with no production precedent in civic technology.
Where things go wrong
This is the control that would have stopped Robodebt, the automated welfare debt-recovery scheme that raised approximately 794,000 debts against approximately 526,000 income support recipients before it was abandoned, ending in $1.751 billion of debt written off and $746 million refunded to some 381,000 people. A continuously measured aggregate error rate and the authority to halt the scheme would have caught the fault before it reached users. The Post Office Horizon inquiry examined the same shape in the UK: a system producing adverse outcomes at scale, over years, with no mechanism to halt it once the pattern of error was visible. The signal itself can be gamed: an agency that narrows what counts as an 'error', or delays a halt decision to keep the reported rate inside its own declared band, can meet the letter of the control while the fault keeps running underneath it.
Sources
8 references
The instrument, the operating deployment, or the official record itself.
- SEC / Investor.gov — Stock Market Circuit Breakers (market-wide 7/13/20%; Limit Up-Limit Down)
- FAA NOTAM Statement — nationwide ground stop, 11 January 2023
- ACUS Recommendation 2021-1 — Managing Mass, Computer-Generated, and Falsely Attributed Comments
- NY OAG — Fake Comments report (~18M of 22M FCC comments fake)
- European Payments Council — SEPA Direct Debit (8-week / 13-month refund; scheme-level reversal)
- Royal Commission into the Robodebt Scheme — Report (the cost of no circuit breaker)
- Post Office Horizon IT Inquiry — Volume 1 final report
Writing about the subject rather than the framework itself, including vendor commentary.
-
Schmitz, Rystrom & Batzner — Oversight Structures for Agentic AI in Public-Sector Organizations (REALM 2025)
A six-interview study of German civil servants, finding that unit-siloed public-sector governance cannot scale with agent-generated event frequency, and that halt-and-reset authority likely has to be diffused to operational units. A small single-country study (n=6), and one data point rather than settled fact.