Essay · The expertise layer

What is human-in-the-loop for AI agents, really?

The phrase is old and the meaning has shifted under it. For models that only predicted, a human in the loop was a labeler. For agents that act, it has to mean a person who can step into a live decision and be answerable for it.

Human-in-the-loop earned its meaning in an earlier era, when the human sat at training time, labeling examples so a model could learn. That human shaped the model and then went home. AI agents change the shape of the problem. They do not just predict, they act: they send the message, move the money, approve or deny the case. A human in that loop cannot be a labeler from last quarter. They have to be reachable now, on the decisions that carry weight, with the authority to change the outcome.

From training-time to run-time

The old loop was about getting the model right before deployment. The new loop is about being right during it. That is a different discipline. It asks not "did a person check the data" but "when this agent hits a decision it should not make alone, does it reach a qualified person, in time, who can actually decide." The work moves from data labeling to live judgment, and live judgment has a clock on it.

Not every decision, the right ones

Putting a human on every action would defeat the point of an agent and exhaust the humans. The skill is selection: directing the small share of calls that genuinely need a person, and letting the rest run. That is a design problem with real edges, knowing which decisions are too consequential to automate, and building the path that gets them to the right human fast. Done well, the human is rare and decisive. Done badly, the human is everywhere and asleep.

The accountable human

The version that matters has a name attached. For the decisions that need oversight, there should be a specific person who is answerable, not a queue and not a policy. That is what turns a loop from a formality into a safeguard: someone whose judgment was actually exercised, and who stands behind the result. The rest of this work is about making that person reachable, equipped, and real.