← Manj ChennaEssays

The stakes · Recourse

The cost of a wrong no

Companies watch the wrong yes, the fraud that got through, the loan that defaulted. The wrong no is quieter and often costlier: the good customer denied, with no person to appeal to, who becomes a complaint, a headline, or a regulator's example.

Manj Chenna · Founder, Sanctity · Building human judgment infrastructure · Amsterdam

A wrong, unappealable no is one of the most underpriced risks in automated decision-making. A wrong yes is visible, it shows up as a loss you can count. A wrong no is invisible until it is not: the qualified applicant rejected, the legitimate transaction blocked, the person who did nothing wrong and cannot reach anyone to say so. Building meaningful human oversight is partly about catching exactly these, the errors that do not announce themselves.

An example that became public

When a major consumer credit card launched, customers alleged that its algorithm offered sharply different credit limits to spouses with similar finances, and that there was no clear human path to contest it. The decisions drew a financial regulator's scrutiny. Whatever the eventual findings on intent, the lesson stands: an automated no with no visible route to a human turns an ordinary error into a public one.

Why the wrong no costs more than it looks

Because it combines harm to a blameless person with the absence of recourse, which is the exact recipe for outrage, churn, and regulatory attention. A wrong yes is a number. A wrong no with no appeal is a story, and stories travel. The cheapest insurance against it is a genuine path back to a human on consequential denials.

Read on

See the right to a human, the trust tax, and the cost of no human in the loop.

Notes and sources

  1. Allegations regarding the Apple Card's credit-limit decisions, 2019; reviewed by the New York State Department of Financial Services.