Co-design with affected communities
A community-oversight surface: an anonymized agent-interaction-log review panel for affected communities, plus an in-conversation 'did this say what you meant?' feedback control.
Services designed for marginalized communities without those communities' input tend to encode the assumptions of the designers. This is amplified when the service is agent-mediated, because the agent's conversational design embeds assumptions about how people communicate.
Government needs confidence that an agent serving a marginalized community behaves the way that community understands and expects, not the way its designers assumed people communicate. Meeting that requires the affected community to shape the agent's conversational behavior before it ships and to keep a hand on it once it is live, so the agency can stand behind how the agent treats them.
Members of the communities a service is built for, especially those with communication needs the designers do not share, are excluded when an agent encodes assumptions about how people talk, and the harm surfaces as misread requests and wrong outcomes they had no part in shaping. The keep-open response is to involve those communities as active participants at every stage and to give them a standing channel to say not just whether the agent helped but whether it said what they meant.
An affected-community panel reviews anonymized interaction logs to keep oversight of how the agent behaves once it is live, and an in-conversation prompt asks the citizen whether the agent said what they meant.
No surface has been built yet; the approach above is the brief for one.
- Established Headline
Co-design methodology is well-established in service design.
- Emerging
Co-design applied to conversational AI is beginning to appear.
- Frontier
Community oversight of deployed agent behavior has no established precedent.
GOV.UK co-design approach. GDS has moved from user-centred design to co-design, designing with users as active participants. The GOV.UK Design System depends on cross-government community contribution, and GDS works with Citizens Advice to analyze users' end-to-end journeys and real-world barriers.
Australia's Digital Service Standard. The DTA's Digital Service Standard (v2.0, December 2023) includes "Leave no one behind" alongside "Know your user," requiring services to include users with different needs at all stages of development and prototyping.
Cross-jurisdictional design-principle alignment. Teams across USDS, Australia's DTA, Finland's D9 unit and Canada's Ontario Digital Service have created overlapping sets of standards mostly building on GDS' original ten principles, a maturing international norm.
Essential but methodologically harder for agent-mediated services. Co-designing a form or a website produces artifacts that are relatively stable and reviewable. Co-designing an agent's conversational behavior requires iterating on something that is probabilistic and context-dependent. Communities need to be involved not just in what the agent says but in how it responds to unexpected input, how it handles distress, and what it does when it does not understand.
The pattern requires structured co-design sprints before deployment, ongoing community panels with access to anonymized interaction logs, and explicit in-interaction feedback mechanisms.
Established co-design methods assume a deterministic artifact, so extending them to govern a system that responds differently to similar inputs, and keeping the community involved after deployment rather than only before it, is what this pattern asks for.
The failure mode is systematic harm to a community baked into the agent's design and only confronted after it has scaled. Building the affected community into the agent's design, and keeping them in a position to direct changes to its live behavior, gives them the standing to correct it rather than leaving them to absorb it.