Co-design with affected communities
Giving the communities a service is built for a hand in how its agent talks, before it ships and after: co-designed conversational behavior, and standing oversight of the live system by the people it affects. Failure modes only an affected community would recognize get raised while they can still be fixed.
The impact of agents
As more services become agent-mediated, an agent's conversational design embeds assumptions about how people communicate, and those assumptions scale from a badly worded form to every conversation the service has. The communities misread by them have no hand on the design doing the misreading.
What must be verified
Government needs confidence that an agent serving a marginalized community behaves the way that community understands and expects, not the way its designers assumed people communicate. Meeting that requires the agency to be able to confirm, to the affected community's own satisfaction, that the agent behaves as they expect, both before it ships and for as long as it stays live.
Protecting access
The excluded cohort is the community the service exists for. When an agent encodes its designers' assumptions about how people talk, users with different language, literacy, or disability needs get misread requests and wrong outcomes. The affected community had no part in shaping the agent and no channel to correct it.
Keeping the path open
- Involve affected communities as active participants at every design stage.
- Give them standing oversight of the live agent through an anonymized interaction-log review panel.
- Ask in the conversation itself whether the agent said what the user meant, so the person who was misread catches it early.
Response surface
An affected-community panel keeps oversight of how the agent behaves once it is live, informed by what users say it got wrong.
The check appears whenever City Assistant restates the user’s intent across a language or a change in tone. Those are the moments when a mistranslation does the most damage.
Panel members come from the affected communities and see logs with identities removed. Every recommendation gets a published answer: adopted, or declined with reasons.
Maturity
- Established
For co-design methodology in service design, which is well settled.
- Emerging Headline
For co-design applied to conversational AI, which is beginning to appear.
- Frontier
For community oversight of deployed agent behavior, which has no established precedent.
Precedents
GOV.UK co-design. UK government service teams describe a move from user-centered design to co-design, treating users as active participants rather than as research subjects. The Design System depends on cross-government community contribution, and the accessibility guidance frames co-design as including service beneficiaries in order to understand journeys, pathways, and how people interact with services.
Australia's Digital Service Standard. The Standard includes 'Leave no one behind' alongside 'Know your user', requiring services to include users with different needs at all stages of development and prototyping. The obligation attaches at every stage, and not to a research phase that closes.
The OECD Observatory of Public Sector Innovation on GDS design principles. The GDS design principles have traveled beyond the UK: the OECD's Observatory of Public Sector Innovation catalogs them as a toolkit for other governments, and standards like Australia's share overlapping principle sets built on the same foundations. The same co-design commitment now sits in several national standards.
What carries over to agent use
The method transfers; the artifact it must govern changes. Co-designing a form or a website produces artifacts that are relatively stable and reviewable. An agent's conversational behavior is probabilistic and context-dependent, so communities need a hand not just in what the agent says but in how it responds to unexpected input, how it handles distress, and what it does when it does not understand.
Established co-design methods also assume the work ends when the design ships. Keeping the affected community involved after deployment, when the agent's behavior can drift from what was agreed, is the part no existing method covers.
Where things go wrong
The failure mode is systematic harm to a community baked into the agent's design and only confronted after it has scaled. A team can also convene a co-design or oversight panel without giving it power to change the agent's behavior, so consultation happens and nothing about the design moves as a result. Giving the affected community a hand in the design, and the power to direct changes to the live agent's behavior, lets that community correct the harm rather than absorb it.
Sources
5 references
The instrument, the operating deployment, or the official record itself.
Writing about the subject rather than the framework itself, including vendor commentary.