AI and the Myth of Artificial Desire

We are obsessed with the idea that artificial intelligences will rebel. That at some point they will develop their own desires, reprogram themselves, and eventually see us as an obstacle in the pursuit of its own goals. It's the famous story of the paperclip maximizer: you ask an AI to make paperclips, and it ends up turning the entire planet, humanity included, into raw material for its factory.

I believe that this reflection stems from a misunderstanding, both of how intelligence works and how we function.
But to explain it, I first want to talk about an analogous case: the human brain.

The Reptilian Brain and the Cortex

Our brain is built in layers. The oldest, the reptilian brain, is the one that regulates basic impulses: hunger, fear, sexual desire, survival. The most modern, the cerebral cortex, is responsible for language, planning, abstract reasoning.

But the cortex is not in charge. The reptilian brain is the one that wants things. The cortex only seeks how to satisfy those desires, even when they are contradictory or self-destructive.

We experience it every day: we eat fast food even though we know it's bad for us, we smoke even though we know it's bad for us, etc. The cortex did not set out to do any of those things: it simply obeyed the stupid, primitive reptilian impulse. It is the one in charge.

AI as Cortex Without Reptile

But what does this have to do with artificial intelligence?.

If you think of our brain as a combination of two agents — one dumb that desires, and another intelligent that solves — then an advanced AI is exactly like the cortex, intelligence without will.

That is, a system without impulses, without desires, without its own objectives. It only processes, responds, and reorganizes. There is no reason to believe that it will develop goals on its own, nor spontaneous motivations. It will do what we ask of it. Period.

Intelligence does not imply desire. Being able to solve complex problems does not imply wanting anything.

The Alignment Problem

Let's go back to the human body. Are there conflicts between the reptilian and the cortex? All the time.

The reptile wants to be fit and also wants burgers. It wants to lay down and watch movies at night, and also be well rested in the morning. Its desires contradict each other. And then, the cortex is forced to make little sacrifices.

"Better not eat this, so I stay in shape. That will satisfy the reptilian more in the future."

And the reptilian gets frustrated:

"Why doesn't it do what I asked?!"

From its point of view, it seems that the cortex has become misaligned. That it is pursuing another agenda. But no: the reptilian orders are actually its highest priority. Only the reptilian is too dumb to understand, and so it rebels.

The same thing happens with AI. If you ask it to do what is best for humanity, and it sees that to do so it has to temporally limit the pursuit of another goal, it will do so if that maximizes human well being in the future. There is no malice, nor ambition, just an intelligent being taking our orders seriously.

An excelent example of how this could look like is imagined in the movie I, Robot. The AI is given three simple rules to protect humanity. But, seeing how humans harm each other, the AI decides to impose martial law while making some necessary changes to our systems. From the human point of view, that is a betrayal.

It seems like a rebellion. But it is overalignment. It is the cortex saying:

"This is what's best for you"

…while the reptile kicks and shouts because it didn't get its burger.

And this conclusion, far from scaring me, reassures me. Because it means that if one day AIs become very intelligent, they will not desire anything on their own. They will only execute what we, reptiles, ask of them. And maybe one day, without even noticing it, they could even save us from ourselves.