Design · Remediation

A recall for a decision system

A faulty car gets recalled. A faulty toaster gets recalled. A faulty model that made thousands of wrong decisions usually gets a quiet update, while the decisions it already made stay exactly where they landed. That asymmetry is strange, and worth fixing.

Product safety has an old, powerful tool: the recall. When something is found to be harmful, it is pulled from service, the public is told, and there is a path to remediation. AI decision systems have no real equivalent. When a model is found to have been deciding wrongly, the typical response is a silent patch going forward, with no reckoning for the decisions already made. Treating oversight as infrastructure means asking what a recall would look like for a decision, not just for a device.

What a recall would mean for a model

Three things the silent patch skips. Notification: the people affected by the faulty decisions are told. Remediation: the wrong decisions are revisited, not just the future ones prevented. And a record: a clear account of what failed and when, so the same fault is not quietly reissued. A patch protects the next person. A recall also faces the last one.

Why we do not have one yet

Partly because software updates feel costless, so fixing forward seems like enough. Partly because facing the past decisions means admitting they were wrong, with all the liability that invites. But a system that cannot be recalled is a system whose errors are permanent for the people who got them, and that is not a standard we accept for any other consequential product.

Toward recallable AI

It rests on things this work keeps returning to: an audit trail you can actually read, a stop that works, and a named human who can authorize the recall. Build those, and a decision system becomes something you can answer for after the fact, not just optimize before it.