Manj Chenna

Trust · Requirements

What trustworthy AI actually requires

Trustworthy AI is one of the most used and least specified phrases in the field. It is worth asking what it would take to deserve the word, rather than just print it.

scroll to read
Manj Chenna · Founder, Sanctity · Building real human oversight of AI · Amsterdam

Trustworthy AI, stripped of the marketing, requires three concrete things: real human oversight, auditability, and recourse. The phrase is crowded, claimed by standards bodies and vendors alike, so I use it carefully and prefer to define it by what it demands. Each requirement is checkable, which is the point. A system that has all three has earned some trust. A system that has the label and none of them has earned only suspicion, and increasingly gets it.

TRUSTRealoversightA person whocan change theoutcomeAuditabilityA record astranger canreadRecourseA way back forthe personaffectedREMOVE ANY ONE AND THE ROOF COMES DOWN
TRUSTReal oversightA person who can change the outcomeAuditabilityA record a stranger can readRecourseA way back for the person affectedREMOVE ANY ONE AND THE ROOF COMES DOWN
Trust is not a property you add at the end. It is what stands up once these three are already in place.

The three requirements

  1. Real human oversight. Not a human who is present, but one who can and does change consequential outcomes, measurably. Meaningful human oversight, not the decorative kind.
  2. Auditability. Someone outside the vendor can reconstruct what a system decided and why, and find the human who was accountable.
  3. Recourse. A person on the receiving end of a decision can contest it and reach a human who can change it.

Why the label without these is empty

Because each of the three can be checked, and a claim that avoids all three checks is asking for faith, not offering evidence. Buyers have learned this, which is why the word alone no longer moves them. Trustworthy AI is not a property you assert. It is the name for a system that does these three things, and can show it.

Read on

See trust is engineered, not advertised and the trust layer for AI.