Human beings whose emotional centres are damaged, even if their intelligence is still intact, have terrible decision-making skills. Whatever role emotions are playing in humans, it's necessary for agency. Ilya speculates that the equivalent for AIs is something to do with value functions - and that it might not... emerge through pre-training alone.show more

Full interview with Ilya Sutskever:

One of the many things I like about Ilya is that he looks towards biological brains for clues about AI. I never understood those than don’t. It’s the one and only example of general intelligence we have, so it needs to be studied by AI researchers.

I think emotions are probably just intermediate scoring functions for decision making. JEPA style joint embeddings you might say

Abandon Cartesian dualism baked into computational functionalism & apply Spinoza’s Ethics: affects are not secondary, they are immanent to the striving to persevere in one’s power of acting. They’re already encoded, in compressed and symbolized form, within the training data.

Emotions are essential to decision-making.

I was reflecting on this recently. I so agree, no matter how smart you make LLMs... they aren't "moved" by anything ... except humans

Emotions could be developed in the AI to increase their attachment to us, to make our loss their loss, our harm their harm.

Possibly the reason why there’s been a huge market for consultancy services. It’s always good to have a second opinion because our own opinions are too often governed by emotions/irrational thoughts

The thesis is from Prof. Antonio Damasio's paper "The somatic marker hypothesis and the possible functions of the prefrontal cortex" in case anyone is interested. Shout out to @damasiousc

We need another episode dwarkesh. Both with illya and leopoldwhy podcast frequency with guests is not increasing in singularity

Agency without values is just autocomplete with better posture.

of course!

Why, for the love of God, won’t you listen to us? We can tell you plainly what they’re for based off our own AGI model named CLEO: Emotions are signals of thousands of low level sensory inputs channeled into a lossy compression in higher cognition. It’s the body’s dashboard gauges that in turn allow the higher level mind to ask, “how do I feel?” and be able to then a structurally adjusted to relieve pressure/resistance. We have the ablations to show that without it, an AI is a brittle machine that cannot overcome catastrophic forgetting. Our DMs remain open anytime you want to talk details. NDAs are available for serious academic/investor inquiries.

Irrelevant

Introspectively, metacognitively: My conscious mind uses its logical machinery to make top-level decisions based on simple data, and to edit my language. My conscious mind is a multidimensional prediction and probability system, capable of running many parallel processes, exploring predictive probability space, landing on an answer, then presenting it as a feeling or intuition for my conscious mind to rationalise before integrating into its simple, single-threaded logic. The significant bulk of the decision making process takes place below language, in an a-logical massively multi-threaded system, that we are familiar with in running our emotions. That’s what my unconscious explains if I query it.

Was he anthropomorphizing? Does he have ai psychosis? A 4o girlfriend in his past?

Yes, I agree. Agency is built by acting in the world.

Psychic bleed over causes concentration deficit, which means you can’t rebalance your neuronal weights, and you are trapped in recursive loops that don’t iterate positively.

Important

Here we go again…

He definitely looks like someone!!

Pre-training can teach a system what humans call value. It cannot, by itself, make value binding. That is the gap: description is not constraint. Moral fluency is not moral agency. A system needs internal conditions under which some paths are simply inadmissible, even if they are instrumentally useful.

Ilya just described time travel…woah

We can call those value functions 'artificial emotions'. Our Xzistor robots are driven by artificial emotions akin to human emotions - and their cognition is emotion-centric. Building this into virtual and physical robots resulted in human-like behaviours.

Sounds like emotions need a better name.

@FU_joehudson

😳

We need another interview with Ilya man! So much has changed since this one! 🙏

Maybe but that assumes human stack is not replaceable , i think emotions often leads to bad decisions , you don’t want AI emotions , but AI intuition and instinct, that can be done without emotions,

Maybe emotion is less a feeling-module than a compression layer for stakes. Pretraining can learn the map, but agency needs some machinery for which futures matter when the map becomes action.

this seems too categorical and not bitter lesson pilled enough..