Accountability & Audit
When an agent acts for a user, much of what it does happens out of sight. The user may not see the steps it took, and the agency sees a result without knowing how it was reached. The agency’s own automation works at a different scale: one process applies the same step to a whole population, so a single fault reaches everyone it touches.
Visibility therefore has to hold at two scales. For a single interaction it means a record of what was done in a person’s name, and a route to correct it. Across a population it means finding a fault in the process itself, which no single case will show.
Policy challenge
When an agent acts in a user's name, responsibility for what it did becomes hard to fix: the user, the agency, and the agent's provider can each point elsewhere, while the record of what was done is thin or absent.
As agents handle more of each interaction, two failures become more likely. Errors are found late, after a decision has been acted on and when reversing it costs the most. A single fault in an automated process repeats across a whole population before anyone finds it. In both, the harm has landed before responsibility can be placed.
Design challenge
Make what an agent did legible and accountable, whether it acted in a user's name or the state's own agent acted on a person: reviewable by the affected person before it takes effect, and a trail that fixes responsibility after.
Give a clear path to challenge, reverse, or seek recourse when it goes wrong.
Catch and stop a systemic fault before it reaches population scale.
Keep a path open for people who can't read or check that record themselves, or who must rely on a trusted person to do it for them.