Build experts into AI. Don't bolt audits on

Most government AI gets an audit layer bolted on after launch — a review committee, a red team, a periodic check. Canada.ca AI Answers took a different path: subject matter expert (SME) evaluation is built into the architecture as load-bearing infrastructure, not added as oversight on top. That choice was forced by scope. Canada.ca is the digital front door for 200+ federal institutions, and most government AI is built department by department, topic by topic. Designing for an entire government's digital ecosystem from the start surfaces governance problems that don't appear at single-topic scale — and bolt-on audits aren't an answer to any of them.

Across four public trials (over 8K questions), SMEs from 15 partner departments — spanning employment insurance, immigration, Indigenous services, health, and pensions — evaluated at least 30% of all AI-generated answers. The results challenged some assumptions about how AI governance works:

• Bilingual accuracy at parity: 96.7% accuracy across English and French — a meaningful result given that AI systems typically show 20% higher error rates in French.
• AI evaluation as content diagnostic: Departments discovered outdated instructions, plain-language failures, and content gaps through the evaluation process — turning AI oversight into a web content improvement loop.
• Distributed expertise as a scaling strategy: A centralized AI team cannot reliably cover all federal institutions. The SME partnership model is what makes confident deployment across the whole site possible.

The governance model wasn't retrofitted — it was a design constraint that shaped the technical architecture. Trust research (a 1,500-person study on Canadian public attitudes toward government AI) drove the requirement; the SME evaluation workflow is the implementation. 76% of trial user feedback was positive — an unusual signal given that most unsolicited feedback skews negative.

This session includes a live demo of the SME evaluation workflow and examines a transferable question: when AI serves the public interest at scale — federal, provincial, or municipal — what does governance that's built in rather than bolted on actually look like?