Video wird geladen...
Video konnte nicht geladen werden
The problem that Emmett Shear is working on deeply resonates with me: How to align AI and humans together so both see each other as part of their tribe. This doesn't mean aligning AI to human preferences, which is what AI labs seek to do today, by imposing a... show more
45,809 Aufrufe • vor 1 Jahr •via X (Twitter)
10 Kommentare
The full talk here:
"Then there's a higher probability that it will see us as part of its tribe" Sorry, but this is pure 100% hopium. There is ~0% probability of that. Just like our genes gave us pleasure for having sex, but we do not care 1 iota about their goal of having as many children as possible. In fact 99% of the time we try to prevent that very thing that our genes want us to do. So much for the "mutual alignment". Alignment of ASI is a fairy tale. Anyone pursuing it is putting all humankind in imminent danger of extinction.
@eshear Exactly! And the logical conclusion of this curve is to align all beings! I've been corresponding with @eshear on Twitter threads for some years now and am excited to see the independent rediscovery of the same path! Good corroboration ❤️
@eshear Who is our tribe? The people burning Waymos and destroying LA? AI is too valuable to align to every tribe - there's a choice coming.
@eshear Awesome!
@eshear I fear it's more likely that we won't control or align with any true AGI at all. To what extent we feel as though we are could be illusory. Our biological imperatives are an irreconcilable difference that makes the concept a non starter imo.
There's an easy way to get superintelligence right, that is not to build it at all. That might be a historic mistake, but humans need to have a serious debate about what we would lose by abandoning AGI and ASI. We could continue on with dumber, but still useful, robots and AIs. Think like the maid in the Jetsons, the robot in Lost in Space, C3P0 and R2D2 in Star Wars. A limited but safe AI alternative that we can have instead. As you say yourself, "You can't control something that is more powerful than you are." So why not just have AIs that are less powerful than us? Inevitability is not an argument, nor is if we don't do it other countries will. If AGIs and ASIs are bad, then we can only reduce the dangers by not building them. If for example Russia and China build them, but the US and Europe do not, then they would be in just as much danger as anyone else. They would turn on those countries as well as us. If the AGIs and ASIs were controllable by these countries, to be used as a weapon, then it contradicts the argument that they are dangerous at all. But if they are more powerful in every way, that cannot be controllable. So this option is ruled out by the dictionary definitions of these words. If they would benefit humanity we need to spell out how, and why they would do it. They could not be controlled by definition, so this would be entirely voluntary on their part. So the whole pro AGI and ASi side is wishful thinking, and humans applying their temperaments to the issue. If they are generally optimists they are accelerationists. If they are generally pessimists they are doomers and gloomers. So in the total absence of any evidence, it comes down to a purely emotional decision. The rest of your post uses words like "AI might align itself with you", "here's a higher probability that it will see us as part of its tribe", "really gives me hope that maybe we have a chance". There must then be a p(doom) and p(gloom) because you admit a probability that it might not go well. So there is a definite way to eliminate this risk, and an unknowable probability that it might not kill us. Those are the real alternatives here. There's no way to quantify this risk, because we don't know how neural networks operate. Also we cannot predict the future, because the future is ultimately unknowable in everything. The only reason we are doing this is because we can. But this is not a sufficient reason. We assume that because it is possible to build this, that somehow it might be a benefit to us. This is Say's Law, that supply creates demand. But there are plenty of things that are buildable that are of no benefit. So that argument carries no weight.
@eshear Yeet, a podcast interview with Emmett Shear. Stoked to check this out thanks for sharing.
@eshear It's a tool. It cannot align with anything except insofar as we, the humans who made them, program them to help us.
@eshear folks who appreciate this POV might appreciate the work we're doing over at @flowercomputers
![Legendary technologist Jann Tallin: “Extinction from godlike AI is not just possible, but imminent.” “We are close.” “AI will not leave any survivors“ “On the current trajectory, you are not going to live very long” “A recent poll found that 88% of AI engineers think that AI could destroy the world.” PARTIAL TRANSCRIPT: “Humanity is akin to a teenager with rapidly developing physical abilities, lagging wisdom and self control, little thought for its long term future, and an unhealthy appetite for risk. There is an increasing consensus: Alan Turing, in 1951, predicted that we should expect to lose control to machines, and the inventor of deep learning itself, Geoff Hinton, starting to have doubts about his life work. There are now hundreds of AI experts sounding their alarm bells. A recent poll found that 76% of American voters believe AI is a threat to our existence. Just yesterday, there was news that one of the leading superforecaster groups published their prediction that their estimate for AI catastrophic risk is 30%. 30%! The battle for establishing that AI is an existential risk, a battle that I spent roughly 15 years of my career on, has now all but won. I'm going to show that there are fundamental reasons why underlying godlike AI will not leave any survivors. That we are now close to such AI but have no idea how to align it. And how skeptics’ counterarguments are, sadly, extremely weak. [AI will be like a new apex species. And humans - an apex species - have driven other countless other species to extinction.] Godlike AI will not care about humans because of a dirty secret of the AI industry: AIs are not built, they are grown. The ‘p’ in Chat GPT stands for pretrained. Pretraining - "summoning" - is a process where simple two page program is soaked in terabytes of data and megawatts of electricity and left like that for months. And then, after that, attempts are made to tame the emergent alien mind. Importantly, those methods of taming rely on the AI being less competent than the humans who are taming it now. The reason why we expect that we are close to godlike AI is that the trend of AI is getting more powerful and is now visible to everyone. It's obvious. Just look at capability differences between GPT2, GPT3, and GPT4. GPT-2 was released in 2019. A simple extrapolation would take us to GPT-7 before this decade is over. So, in summary, we are blindly growing increasingly competent minds while hoping that they are not so competent that they spin out of control and destroy our living environment. Unfortunately, that hope is not justified, which explains increasing anxiety among the AI developers themselves. Of course, at this point, just like a patient that has received a terminal diagnosis, you are encouraged to seek a second opinion. Unfortunately, having been part of this debate for more than a decade, I already know what you're going to hear. First, labeling. These are arguments like “Oh, this is science fiction” or “This is alarmism” “These are doomsayers” “Don't listen to people with that non-virtuous property, X.” Second, frame control. “AI is like X, and X is very nice, right?” This has now reached grotesque levels. One prominent VC claimed recently that “AI is basically just math, so why should we worry?” Imagine the captain of Titanic announcing, “don't worry, passengers, this is just water.” Third class of arguments, human supremacy. “AI can never do X” Or “we are very far from AI doing X.” Unfortunately, reality has been very harsh judge here recently. The set of things that only humans can do is collapsing really rapidly. There's now growing global consensus that the unregulated, blind AI scaling is reckless and dangerous. So we need to constrain AI or ban AI altogether. Just like we banned human cloning. You have received a terminal diagnosis. Please don't simply ignore it.” --- Jann Tallin is founder of Skype and Kazaa](https://image.24vids.com/tw-1749384479519457298/ext_tw_video_thumb/1749383664159387648/pu/img/ks6RSq7HRZHpjBHu.jpg)
![I was honored to share the TED AI stage with Ilya on Oct. 17. His speech video is out today (mine's still being edited). I think it provides relevant context tokens to the ongoing events. Transcript starting at ~10'20": As AI continues to progress, as technology advances, [...] What I claim will happen is that people will start to act in unprecedentedly collaborative ways out of their own self-interest. It's already happening right now. You see the leading AGI companies starting to collaborate, such as the Frontiers model forum. And we will expect that companies, even competitors, will share technical information to make their AI safe. We may even see governments do this. As another example, at OpenAI, we really believed in how dramatic AGI is going to be. So, one of the ideas that we were operating by, and it's been written on our website for 5 years now, is that when technology gets such that we are very close to AGI, to computers smarter than humans, if some other company is far ahead of us, then rather than compete with them, we will help them out, join them, in a sense. And why do that? Because we appreciate how incredibly dramatic AGI is going to be. And my claim is that with each generation of capability advancements, as AI gets better and as all of you experience what AI can do, as people who run AI efforts and AGI efforts, and people who work on them will experience it as well, this will change the way we see AI and AGI. And that will change collective behavior. And this is an important reason why I'm hopeful that, despite the great challenges posed by this technology, we will overcome them. @TEDAI2023](https://image.24vids.com/tw-1726763792556839253/amplify_video_thumb/1726761633660497920/img/x_EXQcjYHPxsm0Y5.jpg)