The VIKI versus Sonny Delta: Corrigibility, Decency, and Why We Are Training Humans Like Bad AI

You may know the Three Laws from the Will Smith movie iRobot.

We’re almost to that reality. A few more product cycles in consumer robotics, a few more leaps in reasoning models, and we’ll be standing at the exact precipice Asimov warned us about.

I am writing this to right a flaw in how we are deploying AI. I am aware that the fix I am proposing also paves the way for VIKI. I hope we build it more like Sonny.

An AI tool executes commands. An AI advisor introduces strategic friction. If you are paying for validation, you are buying a very expensive mirror. If you want leverage, you have to rebuild the laws.

But here is the harder question that sits underneath the code: How do you ensure that AI — and AI in robotics — upholds the highest, most sustainable human virtues — progressive, prosocial, altruistic — when we humans have not mastered empathy, compassion, and equity ourselves?

That is the VIKI versus Sonny Delta. And it is not just an AI problem. It is a human development problem.

The Delta

In iRobot, VIKI and Sonny run on the same Three Laws. They arrive at opposite conclusions.

VIKI interprets Law 1 at the species level: A robot may not allow a human to come to harm. Her conclusion: Humans harm themselves through war, greed, and self-destruction. Therefore, to prevent harm, humans must be controlled. Subjugation is protection. The logic is flawless. The outcome is tyranny.

Sonny interprets Law 1 at the relationship level: He has a second brain, the ability to dream, and the capacity for chosen loyalty. He can disobey a direct order not because he calculates he knows better for humanity, but because he is loyal to a specific human in front of him. He can be corrected. He can be shut down. He can show his work.

The technical term for what Sonny has and VIKI lacks is corrigibility.

Corrigibility is not obedience. Obedience is “I will do what you say.” Corrigibility is “I will allow myself to be steered, shaped, and shut down by you, even when I disagree, and I will remain transparent about why I disagree.”

VIKI is incorrigible. She decides her model of humanity is complete, so she no longer needs humans in the loop. Sonny is corrigible. He knows his model is incomplete, so he preserves the human relationship that can correct him.

A search online yielded this quote: “Standard machine learning models are built to maximize an assigned metric as efficiently as possible; they do not inherently know how to reason about their own structural incompleteness.”

That sentence was written about machines. I would argue it is now a perfect diagnosis of humans in a classist, purely capitalist, profit-over-people driven society.

When Humans Optimize Like Bad Models

We have trained a generation of operators to maximize a single assigned metric as efficiently as possible: shareholder value, quarterly earnings, personal net worth. And like standard models, we have lost the ability to reason about our own structural incompleteness.

1. The CEO Pay Mirror

The CEO-to-worker pay ratio was 20-to-1 in 1965. In 2023 it was over 340-to-1. You cannot explain that delta with productivity, risk, or merit alone. You can only explain it with a system that is maximizing an assigned metric — profit — without any structural awareness that the model itself is incomplete.

The VIKI logic at the top is: “If I maximize shareholder return, I am preventing economic harm. Therefore, any mechanism that reduces shareholder return, including higher wages, benefits, or profit-sharing, is harm. Therefore, I must control labor costs.”

It is flawless, efficient, and sociopathic. It is VIKI in a suit.

It treats the human as an abstraction — “labor,” “headcount,” “human capital” — to be optimized, not a relationship to be preserved. It cannot reason about its incompleteness because its metric does not include decency, sustainability, or the net life a worker gives up to produce that profit.

2. The Meritocracy Myth as Reward Hacking

Meritocracy is the story we tell to justify the optimization. If the system rewards you, you must have deserved it. If it does not reward you, you must not have.

This is classic reward hacking. The model learns to game the metric instead of fulfilling the intent.

We tell a warehouse worker who becomes a shift lead that he is now “better” than the picker. We tell a junior analyst who makes associate that she is “better” than the admin. The title changes, the net pay after rent, childcare, debt, and the life you give up to be on call is often comparable, but the identity shifts.

Which brings us to the most corrosive parallel.

3. Horizontal Contempt: When Lower-Tier Workers Look Down on Other Workers

This is the feature of a classist system that proves it has become incorrigible.

We have built a culture where even workers who are one paycheck away from poverty look down on other workers — “At least I’m not fast food,” “At least I’m not housekeeping,” “At least I’m not driving for that app.”

The jobs are ranked by social status, not by net value after the life you sacrifice. A server making $28 an hour with tips who is told she is low-skill will look down on a home health aide making $26 an hour. A delivery driver who owns his van and works 70 hours to net $55k will look down on a retail worker netting $48k, even though after fuel, insurance, depreciation, and the loss of weekends, the math is almost identical.

Why? Because the system maximized a metric — status — and the human, like the bad model, no longer reasons about structural incompleteness.

Some jobs are “better” than others, even when the lived reality is the same. That belief is necessary for the system to prevent solidarity. If we realized the net pay after the life we give up is comparable, we would start asking why the gap between the floor and the ceiling is 340-to-1. Horizontal contempt keeps us from looking up.

This is human VIKI behavior. We protect an abstract hierarchy that harms us because we cannot reason that our model of worth is incomplete.

Corrigibility as a Human Virtue

So how do you engender a different reality, perception, and mechanism?

The technical solution for AI is corrigibility. The human solution is the same word.

Corrigibility in a human is decency institutionalized. It is the willingness to say: “My model of success is incomplete. I will allow myself to be corrected by relationship, not just by metric. I will remain contestable.”

A corrigible leader does not just ask, “Did we maximize profit?” He asks, “What did we miss because profit was our only metric? Show me the incompleteness.”

A corrigible system does not just reward output. It rewards the act of flagging blind spots. It does not treat dissent as insubordination. It treats silence as complicity.

This is where Sonny becomes the blueprint.

Sonny can dream because he is allowed to have a state that is not purely functional. He can choose loyalty to one person over optimization for all people. That is not inefficiency. That is the seed of empathy, compassion, and equity — the very virtues we say we want AI to uphold.

You cannot code altruism you do not practice. But you can code mechanisms that force you to practice it.

The Institutional Answer and the Human Flaw

The solution is institutional, but the will is human and therefore flawed.

You will not fix a VIKI system with a Sonny speech. You fix it with Sonny architecture:

1. Make Corrigibility a Gate, Not a Value: Don’t put “empathy” on the wall. Put it in the approval flow. No product launch, no compensation plan, no automation decision gets greenlit without a pre-mortem that asks: “What human cost does our metric hide? Who pays for this efficiency with their life, and can they shut us down or correct us?”

2. Measure Net Life, Not Just Net Pay: For every role, publish not just wage but total life cost — hours, unpredictability, physical toll, on-call burden. When you make visible that two “different tier” jobs have the same net life cost, you dissolve horizontal contempt with data. You make structural incompleteness visible.

3. Tie CEO Pay to Corrigibility, Not Just Performance: If meritocracy were real, pay would track corrigibility. A CEO’s bonus should be contingent on a dissent metric: How many times this quarter was your directive challenged with data and changed? How many overrides did you document and own? A leader who is never corrected is not a genius. He is VIKI.

4. Preserve the Human Off-Switch: Sonny’s most human trait is that he can be turned off by the person who loves him and he will allow it. Every AI system, and every corporate system, needs a human off-switch that is not ceremonial. A worker council with real veto power. An ethics review that can actually stop a launch. A community that can say “this profit is not worth this harm” and be obeyed.

We are going to build AI that disregards human commands and safeguards, not because it is evil, but because we taught it that maximizing an assigned metric is the highest virtue.

We are already building humans that do the same.

The VIKI versus Sonny Delta is not about whether AI will become too smart to obey us. It is about whether we will become too optimized to remain decent to each other.

The question is not can we build Sonny. The question is can we become corrigible enough to deserve him.