Manj Chenna

The values layer · Alignment

The whose half of alignment

Alignment research mostly asks how to make AI follow human values. It quietly assumes there is one set to follow. There is not, and the assumption hides the harder, more political question: whose.

scroll to read
Manj Chenna · Founder, Sanctity · Building real human oversight of AI · Amsterdam

The alignment problem has two halves, and the field spends most of its attention on one. The studied half is "aligned to what," the technical work of getting a system to reliably pursue an objective. The neglected half is "aligned to whose," the question of who decides the objective when reasonable people disagree. That second half is where the values layer for AI lives, and ducking it does not make it go away, it just means someone answers it by default, usually whoever shipped first.

Aligned to whatWell funded, well staffed,genuinely hard, and genuinelybeing worked on.Aligned to whoseWho chose the objective, who was asked, and who has to live inside theresult. Comparatively, nobody is working on this.ONE WORD, TWO PROBLEMS, VERY DIFFERENT AMOUNTS OF ATTENTION
Aligned to whatWell funded, well staffed, genuinely hard, and genuinelybeing worked on.Aligned to whoseWho chose the objective, who was asked, and who has tolive inside the result. Comparatively, nobody is working onthis.ONE WORD, TWO PROBLEMS, VERY DIFFERENT AMOUNTS OFATTENTION
The technical half has a field behind it. The other half has a word in a mission statement, and it is the half that decides who the system is for.

Why "whose" is harder

Because it has no technical answer. You cannot optimize your way to whose values a system should hold, since the disagreement is about ends, not means. It is a question of legitimacy and power, and engineers are understandably more comfortable with the tractable half. But the hard half is the one that decides whether a well-aligned system is aligned to anyone the affected people would have chosen.

Taking it seriously

It starts by admitting the choice exists and refusing to smuggle it in as if it were neutral. Every model is an opinion; the honest move is to make the opinion explicit, name whose it is, and give the governed a way to contest it. That does not solve the whose half. It stops pretending it was already solved.

Read on

See whose values should AI hold and every model is an opinion.