Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

Haider.

67,336 subscribers

552,355 views • 2 years ago •via X (Twitter)

Science & Technology News & Politics

Anya Rossi• Live Now

Private livecam show

10 Comments

Haider.2 years ago

Everyone needs to watch the 'HER' movie now.

Luis Saffie2 years ago

Looking back at the history of computing, it's clear that we've always built on top of simpler foundations: from Assembly to C, C++ (OO), Java, and so on. Computing naturally evolves by aggregating lower-level concepts into higher-level ones. That's why thinking of LLMs as an OS makes perfect sense. Karpathy nails this idea in his talk. It’s a logical and exciting progression.

Haider.2 years ago

Do you think this means we'll eventually have LLMs that can understand and interact with hardware directly, kind of like a super-powered OS?

Alex Northstar2 years ago

Very fascinating concept. I wonder if the LMOS will spread.

Haider.2 years ago

That's what I'm curious about too! It feels like we're on the brink of a major shift in computing.

Educating with AI2 years ago

I tend to think LLMs will be more like a human-machine interface. Receives human information (speech likely), "understand it" and translate to multiple devices, apps, CPUs (perhaps AI with a different architecture). If needed, translate back.

Haider.2 years ago

LLMs could become the ultimate middleman between us and all this crazy tech.

JoJo Ventures2 years ago

Generative AI tools now function like customised CPUs renting out RAM for tasks like text-to-image or text-to-video. Eventually, advanced LMOS will differentiate based on the capabilities they offer via GenAI apps built on top of them.

Haider.2 years ago

You're right, the analogy of specialized CPUs renting out RAM for tasks is quite apt. It highlights a crucial shift in the AI landscape. And now we're moving away from the monolithic, "one-size-fits-all" approach to AI development and towards a more modular and specialized ecosystem.

Mike Harris2 years ago

What talk is this from?

Related Videos

Introducing FlexOlmo, a new paradigm for language model training that enables the co-development of AI through data collaboration. 🧵

Introducing FlexOlmo, a new paradigm for language model training that enables the co-development of AI through data collaboration. 🧵

Ai2

370,042 views • 1 year ago

From the Quantum Labs: Dynex's Quantum-Diffusion LLM (qdLLM) is set to launch soon. It marks a paradigm shift in generative AI, combining quantum computing principles with cutting-edge language modelling to unlock new levels of reasoning, efficiency, and scalability.

From the Quantum Labs: Dynex's Quantum-Diffusion LLM (qdLLM) is set to launch soon. It marks a paradigm shift in generative AI, combining quantum computing principles with cutting-edge language modelling to unlock new levels of reasoning, efficiency, and scalability.

Dynex

19,064 views • 11 months ago

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Pushmeet Kohli

203,455 views • 1 year ago

OpenAI Chief Research Officer, Mark Chen "We've found a new paradigm through Reasoning, which we're also scaling" - GPT-5 could be a significant advancement because we're scaling models not just through unsupervised learning (like with GPT-4.5), but also through a new "reasoning paradigm" - GPT-5 maybe a combination of these approaches

OpenAI Chief Research Officer, Mark Chen "We've found a new paradigm through Reasoning, which we're also scaling" - GPT-5 could be a significant advancement because we're scaling models not just through unsupervised learning (like with GPT-4.5), but also through a new "reasoning paradigm" - GPT-5 maybe a combination of these approaches

Haider.

153,382 views • 1 year ago

CPUs and GPUs power the world today—but a new kind of computing is emerging. Enter quantum computing. BlackRock’s Tony Kim explains what QPUs are, how they complement today’s systems, and why quantum could transform what’s possible across science, technology, and investing.

CPUs and GPUs power the world today—but a new kind of computing is emerging. Enter quantum computing. BlackRock’s Tony Kim explains what QPUs are, how they complement today’s systems, and why quantum could transform what’s possible across science, technology, and investing.

BlackRock

56,720 views • 3 months ago

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.

LMQL (Language Model Query Language)

198,966 views • 3 years ago

we're turning University of Waterloo into a supercomputer! arceus is a cross-device distributed compute network for training large models, using model/tensor/pipeline parallelism. you can train anything, from deep neural networks to language models on the network. oss & deployment soon!

we're turning University of Waterloo into a supercomputer! arceus is a cross-device distributed compute network for training large models, using model/tensor/pipeline parallelism. you can train anything, from deep neural networks to language models on the network. oss & deployment soon!

rajan agarwal

35,720 views • 1 year ago

New Anthropic research: Tracing the thoughts of a large language model. We built a "microscope" to inspect what happens inside AI models and use it to understand Claude’s (often complex and surprising) internal mechanisms.

New Anthropic research: Tracing the thoughts of a large language model. We built a "microscope" to inspect what happens inside AI models and use it to understand Claude’s (often complex and surprising) internal mechanisms.

Anthropic

1,572,171 views • 1 year ago

Sam Altman makes a strong point here. Human computer interaction hasn’t fundamentally changed in decades. Windows, pointers, apps, even mobile are variations of the same model. AI breaks that pattern. AI can understand language, hold long context, and operate continuously instead of being a simple on/off tool. That enables a new class of products. The next shift isn’t a better interface. It’s context-aware, unobtrusive technology that integrates into daily life and acts with you, not just for you. This will be hard to build and many attempts will fail. But this is clearly the direction computing is moving.

Sam Altman makes a strong point here. Human computer interaction hasn’t fundamentally changed in decades. Windows, pointers, apps, even mobile are variations of the same model. AI breaks that pattern. AI can understand language, hold long context, and operate continuously instead of being a simple on/off tool. That enables a new class of products. The next shift isn’t a better interface. It’s context-aware, unobtrusive technology that integrates into daily life and acts with you, not just for you. This will be hard to build and many attempts will fail. But this is clearly the direction computing is moving.

VraserX e/acc

11,601 views • 4 months ago

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,644 views • 2 years ago

I do feel this is the "iPhone of the Web3" moment. A disruptive paradigm shift on a paradigm shift. "Build with AI using natural language, own e2e on sovereign $CLOUD infrastructure." Trying to wrap my head around the magnitude 🤯 Only #ICP ♾🌐☁️👌 Slides in the comments 🧵

I do feel this is the "iPhone of the Web3" moment. A disruptive paradigm shift on a paradigm shift. "Build with AI using natural language, own e2e on sovereign $CLOUD infrastructure." Trying to wrap my head around the magnitude 🤯 Only #ICP ♾🌐☁️👌 Slides in the comments 🧵

Eerik Potter

18,190 views • 1 year ago

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

AK

125,319 views • 3 years ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

New 🤗 transformers release includes a very powerful Multimodel Large Language Model (MLLM) by Microsoft called KOSMOS-2! 🤩 The highlight of KOSMOS-2 is grounding, the model is *incredibly* accurate! 🌎 Play with the demo here 👉 But how does this model work? Let's take a look! 👀🧶

New 🤗 transformers release includes a very powerful Multimodel Large Language Model (MLLM) by Microsoft called KOSMOS-2! 🤩 The highlight of KOSMOS-2 is grounding, the model is incredibly accurate! 🌎 Play with the demo here 👉 But how does this model work? Let's take a look! 👀🧶

merve

143,830 views • 2 years ago

We've been tinkering with protein structure design. Our (new?) method works like a language model, but instead of a discrete token, it samples the placement of the next amino acid backbone from a continuous distribution. Meet the Generative Invariant Angle Transformer (GIAT):

We've been tinkering with protein structure design. Our (new?) method works like a language model, but instead of a discrete token, it samples the placement of the next amino acid backbone from a continuous distribution. Meet the Generative Invariant Angle Transformer (GIAT):

Ben Murrell

68,093 views • 2 years ago

"Google came along as the first intent-driven experience" - "In this new paradigm, the path from intent to action gets shortened massively with automation and agent automation." "There will be a future where the majority of new users onboard to blockchain rails through a natural language model on the front end, like talking to an assistant" "In order to achieve that, we need to trust and be able to work with and understand what's on the other side of what we're prompting"

"Google came along as the first intent-driven experience" - "In this new paradigm, the path from intent to action gets shortened massively with automation and agent automation." "There will be a future where the majority of new users onboard to blockchain rails through a natural language model on the front end, like talking to an assistant" "In order to achieve that, we need to trust and be able to work with and understand what's on the other side of what we're prompting"

The Rollup

106,265 views • 1 year ago

$Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.$

Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.

Sapient Intelligence

512,741 views • 2 months ago

OpenAI just dropped a completely new kind of model gpt-realtime-translate takes in speech audio from any language and outputs speech in your target language LLMs are great, but you need specialized models for specialized use cases We're running this on our smart glasses

OpenAI just dropped a completely new kind of model gpt-realtime-translate takes in speech audio from any language and outputs speech in your target language LLMs are great, but you need specialized models for specialized use cases We're running this on our smart glasses

cayden 凯登

100,395 views • 1 month ago

UPDATE: #Cardano Founder Charles Hoskinson says "Midnight is being developed by more than a dozen companies. We're already working with members-based organizations, like the Linux Foundation, and we're a member of the Confidential Computing Consortium with NVIDIA." $ADA $NIGHT

UPDATE: #Cardano Founder Charles Hoskinson says "Midnight is being developed by more than a dozen companies. We're already working with members-based organizations, like the Linux Foundation, and we're a member of the Confidential Computing Consortium with NVIDIA." $ADA $NIGHT

Angry Crypto Show

15,155 views • 2 months ago