Essay · Escalation design

When should an AI agent escalate to a human?

Too much escalation and the agent is pointless and the humans are buried. Too little and it acts alone on calls it had no business making. The art is in the line between, and the line can be drawn on purpose.

An AI agent should escalate to a human when the stakes, the uncertainty, or the accountability exceed what it should carry alone. That is the principle. The craft is turning it into a rule a system can follow: escalating the decisions that genuinely need a person, while letting the ordinary run of work pass without ceremony. Get the threshold wrong in either direction and you have built either a bottleneck or a liability.

The three triggers

1. Stakes. The cost of being wrong is high or hard to reverse. A decision that can ruin someone's day, or worse, belongs with a person even when the model is fairly sure.
2. Uncertainty. The agent's own confidence is low, the situation is novel, or the inputs conflict. Confident on easy cases, humble on strange ones, is the posture you want.
3. Accountability. Someone will have to answer for this decision. If the answer cannot honestly be "the agent," it should not be the agent deciding alone.

Escalate the right things, not more things

The goal is not maximum escalation, it is well-aimed escalation. Flooding a human with every borderline call trains them to approve on reflex, which recreates the rubber stamp you were trying to avoid. A good escalation design is judged by whether the decisions that reach a person are the ones that needed one, and whether the person had room to actually decide.

What happens on the other side

Escalation is only as good as the human it reaches and the time it takes to reach them. A call routed to an unqualified or unavailable person is escalation in name only. That is why this connects directly to the accountable human on the receiving end and to the clock that measures the handoff.