Signaling human-in-the-loop oversight

Agents that can make a determination, or feed one to a caseworker, are now widely available, and as government runs high-stakes decisions through them the share of decisions touched by automation grows. Whether a human still genuinely oversees each one, and who that human is, becomes hard to tell from the outside.

When that oversight is assumed rather than made legible, an approval given without real scrutiny looks the same as real review to the user it affects.

EU AI Act Article 14. High-risk AI systems must 'be designed with human-machine interface tools enabling effective oversight'. The Digital Omnibus deferred the Annex III compliance deadline, so the forcing function is adopted law without a near-term date.

The Attorney-General's Department consultation on automated decision-making. The Attorney-General's Department consulted on a framework requiring risk assessments before deployment, stronger safeguards for high-stakes decisions, a named human accountable with power to review and override, and an entitlement to timely review of high-risk automated decisions. The named accountable human with override power is the concrete element.

Parasuraman and Riley, with deployment data. The human-factors literature holds that automation serves people well only where reliance on it is calibrated, which makes human-AI collaboration rather than full automation the appropriate model. Oversight tooling lags deployment: a Cloud Security Alliance survey of 418 IT and security professionals found 82 percent of enterprises already have unmanaged or unknown AI agents in their environments.

Schmitz et al. on public-sector oversight structures, REALM. Interviews with German civil servants found existing oversight is episodic and event-triggered rather than continuous, and that governance has to diffuse toward the operational staff doing the work. The study describes an organizational condition, which no signal in the interface reaches.

Directly applicable, and treated as a requirement rather than an option where automated decisions have caused documented harm. Any government AI agent that makes or contributes to decisions affecting users has to communicate the nature and extent of human oversight. In several jurisdictions this is a legal and ethical requirement.

Proposed signaling pattern:

• Before agent action: "This recommendation will be reviewed by [role] before any decision takes effect."
• During agent action: "Processing your information now. A [role] will review the outcome."
• After agent action: "Your [application/claim/request] has been reviewed by [named officer/role]. Here is the outcome and how to request further review."

The key principle is that intervention "should not feel like an emergency feature": pause, edit, undo, and override should be integrated into primary workflows, not hidden behind escalation procedures.