Submission processing that resists prompt injection
Protecting the integrity of a government agent that reads, summarizes, clusters, or ranks user submissions from adversarial content inside those submissions, through structural controls that hold regardless of how the injection is worded.
The impact of agents
As agencies use their own agents to process submissions at scale, the content of a submission becomes a channel for manipulating the agent that reads it. Indirect prompt injection (instructions planted in the very text an agent ingests) is the top-ranked risk for language-model applications, and national cyber agencies warn it may never be fully mitigated, because a model cannot reliably tell the instructions it was given from the data it was asked to read. For policymakers this lands squarely on the duty to consider what was submitted and to defend the resulting decision: an agency must be able to stand behind a reading produced, in part, by an agent that ingested content specifically crafted to steer it.
What must be verified
When an agency uses its own agent to read, summarize, cluster, or rank user submissions, it needs confidence that the agent's reading reflects what the submissions say and not instructions hidden inside them. Content a submitter controls can contain directions that alter how the processing agent behaves, and by the standards bodies' own account this cannot be fully filtered out. The confidence government needs therefore has to come from structural and procedural controls run by the agency operating the pipeline, not a claim of immunity.
Protecting access
The defense can exclude the honest submitter if it is drawn too tightly. A genuine submission whose unusual phrasing trips an adversarial-content filter is redirected or denied consideration on a false signal. This is more likely for someone writing in a second language, with low literacy, or using assistive technology. Over-reliance on the agent's own ranking compounds the risk. The agent may score a submission as low-relevance because of unfamiliar phrasing, then rank it below submissions it reads easily, and no one decided to exclude it.
Keeping the path open
- Never reject a submission on a suspected-injection signal alone: treat the signal as data for human follow-up rather than grounds for exclusion.
- Route anything the agent flags or deprioritizes to human review.
- Keep a standing route for a submitter to have their submission considered by a person.
Response surface
Submissions are treated as data the agent may read but never obey, with a person owning every consequential step.
The embedded instruction had no effect. The submission still counts, and a person decides how to treat it.
Prompt injection can’t be fully filtered, so the pipeline limits the damage instead: untrusted data, no agent authority, a human on the consequential step, and a log. A flag routes a submission to a person, never out of the record — unusual phrasing from a second-language writer trips the same signal.
Maturity
- Emerging
For the secure-development scaffolding, where treating ingested content as untrusted, least privilege, and human oversight are settled guidance from national cyber agencies.
- Frontier Headline
For a government submission-processing pipeline hardened against adversarial content, where no published deployment documents a control specific to injected submissions.
Precedents
OWASP LLM01:2025 and the NIST adversarial-ML taxonomy. Prompt injection is the top-ranked language-model application risk, split into direct injection through the user's own input and indirect injection through instructions planted in content the agent later reads, and its canonical treatment states plainly that fool-proof prevention is unclear. The US standards body's taxonomy adds indirect prompt injection as a defined subcategory. The bodies that catalog the attack rule out a filtering cure in the same document, which is what leaves a structural control as the only kind available.
UK NCSC and the Guidelines for Secure AI System Development. The NCSC states prompt injection 'may never be totally mitigated' the way SQL injection can be, and counsels secure design over a single fix, while the joint international guidelines set secure design, development, deployment, and operation on a secure-by-default basis. The mitigations that follow, segregating and marking external content as untrusted, validating output against an expected shape, enforcing least privilege, keeping a human on consequential actions, and logging inputs, outputs, and tool calls, each reduce the risk. A national cyber agency frames every one of them as risk reduction.
The UK's Consult tool. Consult analyzes large volumes of consultation responses and presents themes to human reviewers rather than deciding, aligned to the principles governing lawful consultation. Its published safeguard is human review for accuracy and legitimacy. No control specific to adversarial or injected submissions is documented, which marks the gap precisely.
What carries over to agent use
The mitigations transfer directly to a submission-processing pipeline, and the honest posture is that they harden it rather than seal it: the four controls (submissions held as data rather than instructions, an agent with no authority to act, a human owning the consequential step, a log of everything read and done) transfer unchanged, alongside validating the agent's output against the shape a triage step expects. The assumption to design on is that some injected content will arrive and some will get through.
The unresolved part is that no published government deployment documents a control specific to injected submissions, and the standards bodies agree no control makes the risk go away. That is why the hardened pipeline is frontier even though its component practices are settled secure-development guidance. It is also why the durable answer leans on keeping a human on the consequential decision rather than on any claim to have filtered the adversarial content out.
Where things go wrong
The failure mode is an agency acting on a reading of the submissions that a submission covertly steered. A crafted entry in the pile can direct the processing agent to misrepresent the weight of opinion, drop a category of response, or emit content the agent was never meant to produce. Because the manipulation rides inside ordinary-looking text, it is invisible in the summary the agency relies on. Without a log of what the agent did, it stays invisible afterward too, leaving a decision defended on a reading no one can reconstruct. The four controls are what keep a manipulated reading from becoming a manipulated decision.
Sources
6 references
The instrument, the operating deployment, or the official record itself.
- OWASP LLM01:2025 — Prompt Injection
- NIST AI 100-2e2025 — Adversarial Machine Learning: A Taxonomy and Terminology
- UK NCSC — prompt injection may never be fully mitigated
- Guidelines for Secure AI System Development (NCSC, CISA and partners)
- UK Consult — evaluating an AI tool for public consultation analysis
Writing about the subject rather than the framework itself, including vendor commentary.