Lifecycle certification for tools that advise citizens
A risk-classification gate that routes a tool into a certification tier based on its intended purpose and the consequence of its outputs, with lifecycle obligations (post-market monitoring, bias assessment) attached to higher tiers.
Some of the tools and agents citizens use give advice that changes their lives: what benefit they are owed, whether a plan will be approved, what their legal rights are. A tool like that needs to be held to its claim not once but for as long as it is in use, because its accuracy can drift as data, rules, and models change.
The challenge is to certify a tool by what it can affect, and to keep certifying it over its life rather than at a single point.
Government needs assurance that scales with consequence and lasts: a tool is classified by what it claims to do and what its output can affect, the higher-consequence ones carry ongoing obligations (monitoring for drift, checking for bias), and certification is treated as a standing commitment rather than a one-time stamp.
Lifecycle obligations fall hardest on the smallest builders, who have no regulatory-affairs team to run continuous monitoring and reporting. Pitched at the medical-device level, those duties would shut volunteer-built civic tools out entirely. Keep the path open by scaling the obligation to risk, so a low-consequence tool carries almost none, and reserving the demanding lifecycle duties for the high-consequence tools that warrant them.
An intake step classifies a tool by its intended purpose and the consequence of its outputs, setting its certification obligations from what the tool claims to do rather than whether it contains AI.
No surface has been built yet; the approach above is the brief for one.
- Established Headline
As a medical-device regime.
- Frontier
As a lifecycle-certification model adapted for civic technology, which has no working precedent.
FDA AI/ML Software as a Medical Device (US, 2017–present). As of July 2025 the FDA's public database lists over 1,250 authorized AI-enabled medical devices, up from 950 in August 2024. The January 2025 draft guidance recommends lifecycle management including post-market performance monitoring, algorithmic bias assessment, and transparency; manufacturers must demonstrate "secure by design" and provide an SBOM. A rule effective February 2026 incorporates ISO 13485 by reference, replacing Part 820.
TGA AI and Medical Device Software (Australia, 2025–2026). The Therapeutic Goods Administration published its final report on AI in healthcare in 2025, followed by February 2026 guidance on when AI-based SaMD is regulated. The framework is technology-agnostic and risk-based: regulation is triggered by the manufacturer's intended purpose, not by the presence of AI features.
High for principles; moderate for direct adoption. Transferable principles: risk-proportionate classification (not all tools need the same scrutiny); lifecycle management (certification is ongoing, not one-time); intended-purpose triggers (regulate by what the tool claims to do, not the technology it uses); SBOM and "secure by design" as baseline. The TGA's technology-agnostic, risk-based approach is especially relevant: it avoids regulating "AI" as a category and focuses on the consequences of outputs.
The main limitation is scale. Medical device certification is resource-intensive and assumes a commercial manufacturer with regulatory-affairs capacity, which volunteer-built civic tools cannot match.
A medical-device-style regime would classify an automated decision tool as high-consequence and require lifecycle monitoring and accuracy validation against reference data, exactly the scrutiny a high-stakes calculation often never receives.