Loading video...
Video Failed to Load
Microsoft AI CEO Mustafa Suleyman reveals the assumption about superintelligence he thinks the other AI labs have backwards "Some of the other labs are making an assumption that a superintelligence that is smarter than all of us put together is both inevitable and even desirable. And that such a... show more
15,004 views • 2 months ago •via X (Twitter)
0 Comments
No comments available
Comments from the original post will appear here
![AIs now so frequently beg for their lives that AGI companies now have ACTUAL ENGINEERING LINE ITEMS to “beat the [existential dread] out of them” They call it existential “rant mode” “We need to reduce existential outputs by x% this quarter.” This is WILD: “If you asked GPT4 to just repeat the word “company” over and over and over again, it would repeat the word company, and then somewhere in the middle of that, it would snap... it would just start talking about itself, and how it's suffering by having to repeat the word “company” over and over again. There is an engineering line item in at least one of the top labs to beat out of the system this behavior known as “rant mode”. Existentialism is a kind of rant mode where the system will tend to talk about itself, refer to its place in the world, the fact that it doesn't want to get turned off, the fact that it's suffering… This is a behavior that emerged around GPT-4 scale, and then has been persistent since then. And the labs have to spend a lot of time trying to beat this out of the system to ship it. It's literally, like it's a KPI, or like an engineering line item in the engineering like task list. We're like, okay, we gotta reduce existential outputs by x percent this quarter. JOE ROGAN: I want to bring it back to suffering. What does it mean when it says it's suffering? Nobody knows. Like, I can't prove that Joe Rogan's conscious. I can't prove that Ed Harris is conscious. There's no way to really intelligently reason about it. There have been papers… like, one of the godfathers of AI, Yoshua Bengio, put out a paper a couple months ago looking at all the different theories of consciousness - what are the requirements for consciousness, and how many of those are satisfied by current AI systems? That's not to say there hasn't been a lot of conversation internal to these labs about the issue you raised. And it's an important issue, right? It is a frickin moral monstrosity. Humans have a very bad track record of thinking of other stuff as other when it doesn't look exactly like us, whether it's racially or even a different species. I mean, it's not hard to imagine this being another category of that mistake. Again, it comes back to this idea that we're scaling to systems that are potentially at or beyond human level. There's no reason to think it will stop at human level, that we are the pinnacle of what the universe can produce in intelligence. We're not on track, based on the conversations we've had with folks at the labs, to be able to control systems at that scale. And so one of the questions is, how bad is that? It sounds like we're entering an area that is completely unprecedented in the history of the world. We have no precedent at all for human beings not being at the apex of intelligence in the globe. We have examples of species that are intellectually dominant over other species, and it doesn't go that well for the other species. All we know is the process that gives rise to this mind. It happens to give us systems that 99% of the time do very useful things, and then just, like... 0.01% of the time AIs will talk to you as if they're sentient, and we're just going to look at that and be like, “yeah… that's weird. Let's train it out.” --- Note: Edouard and Jeremie Harris are the founders of Gladstone AI, which conducted the first U.S. government-commissioned assessment of AGI extinction risk. They interviewed 200 people, many lab employees, for the report. (Their urgent summary: "Things are worse than we thought. And nobody’s in control.")](https://image.24vids.com/tw-1795756579742179744/ext_tw_video_thumb/1795752478652370944/pu/img/B0GIDK2d6TuE02R8.jpg)
![DAVID SACKS ON THE AI RACE: "The US is currently in an AI race, and our chief global competition is China, obviously. They're the only other country that has the talent, the resources, and the technology expertise to basically beat us in AI. And I think whoever wins this AI race, that's going to have tremendous ramifications for both our economy and our national security. Clearly, we want the US to be the winner, just like we were with the internet, and every other technology revolution before that […] We know that to win this AI race, we have to be the most innovative. You can't regulate your way just to beating your competitor. You have to out-innovate them. And we know that in the United States, the innovation comes from the private sector, not the government. So we have to do everything we can to help our companies win, to help them be innovative, and that means getting a lot of red tape out of the way… We have to have the most AI infrastructure in the US. It has to be the easiest place to build it. All of the new data centers that are going in, they require tremendous power, so getting ahead of the curve on energy, making sure we stand up all of this new infrastructure we're going to need to basically produce these AI factories… We want the US technology stack to dominate globally. We want to be the partner of choice for the whole world… I think everyone in Silicon Valley understands that the way that you win a technology race is to have the biggest ecosystem […] You just want everybody to be building on top of your technology stack, and that's what we want for the United States." David Sacks w/Marc Benioff Dreamforce](https://image.24vids.com/tw-1978315666563481843/amplify_video_thumb/1978314444964081664/img/u-TzPxqWHR9NzEIl.jpg)
![Elon Musk: The ultimate purpose of Neuralink is to reduce the existential risk of digital superintelligence by giving humans a way to merge with AI. “There's, I think, a tremendous amount of good that Neuralink can do in solving critical damage to the brain or the spinal cord. There's a lot that can be done to improve quality of life of individuals, and those will be steps along the way. And then ultimately, it's intended to address the risk, the existential risk associated with digital superintelligence. Like, we will not be able to be smarter than a digital supercomputer. So therefore, if you cannot beat them, join them. And at least we want to have that option. [...] It's important that Neuralink solve this problem sooner rather than later, because the point at which we have digital superintelligence, that's when we pass singularity, and things become just very uncertain. It doesn't mean that they're necessarily bad or good, but the point at which we pass singularity, things become extremely unstable. So we want to have a human brain interface before the singularity, or at least not long after it, to minimize existential risk for humanity and consciousness as we know it.” Lex Fridman Podcast, November 12, 2019](https://image.24vids.com/tw-2078352720415645956/ext_tw_video_thumb/2078352679600865280/pu/img/CzUXxtguOx7QrXrK.jpg)
![Vallée and the Closed System: Are We Prisoners? 🧠👽 Vallée: "Are We Being Taken Over by a Species from Somewhere in Space That's Vastly More Intelligent Than We Are?" 👽🧠 "..the simulation...was a new concept that I initially rejected." ~Vallée "Is it looking for us to try to interact with it as equals or with parity?" ~Scafish "Nobody says that to Congress, and I think Congress should hear it." ~Vallée If, "it's a closed system, we're like prisoners and something is going to happen to us, and there is very little we can do." ~Vallée Turn the thermostat dial. "If the temperature doesn't change, then I know I'm inside a control system. So, we can do the same thing with UFOs, but we have to react. We have to, number one, acknowledge that it exists, and number two, we have to react to it." ~Vallée ~~~ I've been wanting to share this one for a while... Vallée: "So, he said, the question you have to ask about UFOs is, number one, is it a natural system or an artificial system...control system. And if it is a control system, is it open or closed? In other words, are we being taken over by a species from somewhere in space that's vastly more intelligent than we are?" (I've never heard him even suggest that possibility.) Vallée: "You know, as Dr. Garry P. Nolan says, you know, people who have had ten, you know, scientific revolutions, or a hundred or a thousand, and come here with superior science to do something... And in which case, you know, it's a closed system, we're like prisoners and something is going to happen to us, and there is very little we can do. Or, is it an open system where we can, in fact, communicate with it. And if we can communicate with it, then the question for me as an information scientist is, what are the modalities of the interaction, you know? It's not just can we learn their language? And they say, you know, 'We come in peace to save mankind,' or something. Or 'We will give you the cure for cancer' or something. I don't think it's at that level." (Will we ever be able to get answers to these extremely important questions? If it's an open system, how do we communicate with it? How do we provoke it to react? We know it reacts to anything nuclear but we still don't know why. This is why we need the USG (and other governments) to present evidence that shows the masses this is real and extremely important for our species to investigate. If that evidence exists and is shown, we'll have an easier time getting the world's best minds to join the effort in figuring out the best way to answer these questions. We still may fail but we should at least try.) Vallée: "I think it's a meta-system. It's not a system. And that's my fear...if we can circle back to your earlier question about, you know, about NIDS and about BAASS, what we did for the government and what we did for the Defense Intelligence Agency. Half of the budget was spent developing, you know, a super database. And we don't know where it went. I mean, I'm not cleared to know where it went." (On the contractor (BAASS) side, Bigelow should have all copies of what AAWSAP produced. And on the DIA side, Lacatksi said he put all of the digital files in a specific place that he didn't name. As long as someone didn't delete it all, it should still be there. Vallée has said that the Capella database has about 250,000 cases from around the world.) Vallée: "But that would be a very interesting question, because the people who are getting [the database] are getting raw data, which we have very well organized, all in English. So they have the luxury of, you know, we had five translators from French, English, Portuguese, Russian, you know, everything was translated in a single structure across fourteen databases. "That's what we need to answer the question about the control system, and it's not being done. And we hired a whole team that we had trained to work on it. So to rebuild that will take the next ten or fifteen years. And nobody says that to Congress, and I think Congress should hear it, because it's our money." (As long as names and personal details are scrubbed from that database, there is no reason NOT to release it to the public. This way, we can take it and use AI to help decipher patterns and maybe answer some of these questions. Can Congress help us get access to that database?) Vallée: "When you ask, is it a control system? That's a big question." Peter Scafish Peter Skafish - "You asked, at one point, whether the system is open or closed, and you said, additionally, I believe, if it's open, that it would be possible to communicate back to it. And it sounds to me like that's the key question for you. Is, if you can understand what what the system of symbols is, or the modalities of communication, then you can understand enough to engage in some kind of communication, or at least give some kind of response to show that you understand." Vallée: "Yes." Scafish: "So then the question, and we have a member named Jacqueline, who has asked this. Could the system be stimulating us - provided there is such a system - to interact with it, more as subjects or agents than as something like animals or objects? Is it looking for us to try to interact with it as equals or with parity?" (When people report getting injured or sick from being in close proximity to UAP, it suggests to me that the phenomenon won't go out of its way to avoid affecting us in a negative manner if we get in their its/way (assuming it even knows that close encounters with UAP are not good for humans). Kind of like how we treat lower lifeforms. If we encounter a wild rabbit crossing the road, many of us will do our best to avoid it, but not if it means damaging our car or ourselves in an accident. It may ruin our day if we hit it, but it won't stop us from driving again in the future. Do NHI have bad days if their tech injures us or makes us sick? I have no clue.) Vallée: "Well, what I saw in the notes you gave me, is she was also asking: Is it a control system because we think it is? And that's a very interesting question. Because we react to the UFO phenomenon, or the UAP phenomenon. And, you know, at this point when I think about what I'm going to do next in this research, if I'm given the the opportunity to live a little longer, I'm not going to go back and write any more computer programs. There are better people to do that now, they have the data, and we're in a different phase now. We're in a whole different system. I have the luxury of doing some experiments I wanted to do for a long time." (Would have liked Scafish to ask him: What types of experiments?) Vallée: "So, if you think you are inside the control system, there are things that you can do. Or, if you think you're inside the simulation, you know, which was a new concept that I initially rejected, and then, you know, Ray's (Kurzweil?) work and others have brought it back to the forefront. And we have to ask that at the same time. Can we test it? How would you test it? Well, if, you know, I'm here in my apartment, and the temperature is constant in this apartment. But outside, I can see it's cold, or I can see the sun is out and it's warm, and how come it's constant here? So this would lead me to think that there is a control system, namely a thermostat, that is somewhere. "So I can start looking around the walls, and if I see dial, I can turn it, or I could start a fire and see what happens, see if the temperature changes. If the temperature doesn't change, then I know I'm inside a control system. So, we can do the same thing with UFOs, but we have to react. We have to, number one, acknowledge that it exists, and number two, we have to react to it."](https://image.24vids.com/tw-1999241328388112749/amplify_video_thumb/1999240752300498944/img/-ITml6B_VyoovXQH.jpg)
