Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it. And AI Models actually do learn during the context window, without changing the weights." ~ Anthropic CEO Dario Amodei (On the 2nd... show more

Rohan Paul

149,427 subscribers

631,390 views • 9 months ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Google keeps sharing a ton of free content! Here is one of their most complete examples: a long-running AI agent that pauses, resumes, and never loses context. It simulates the onboarding of a new employee. There are the 3 architectural patterns you'll learn from this:

Google keeps sharing a ton of free content! Here is one of their most complete examples: a long-running AI agent that pauses, resumes, and never loses context. It simulates the onboarding of a new employee. There are the 3 architectural patterns you'll learn from this:

Santiago

20,000 views • 23 days ago

in 9 minutes on the Tokyo stage, Angela Jang, head of product for the Claude platform, said the part most builders still haven't figured out: "a model is only as good as the context that you actually give it." that's the whole game now. not the prompt. the context. what your agent remembers. what skills it can pull. what it learns from its own past runs. anthropic now ships all three: memory, skills, and dreaming. agents that inspect their own trajectories and self-improve. demoed live this week. the prompt was never the bottleneck. the context was. most people are still tweaking words in a chat box. the ones who get this engineer the context their agents run on. watch the 9 minutes. then read the full breakdown below ⬇️

in 9 minutes on the Tokyo stage, Angela Jang, head of product for the Claude platform, said the part most builders still haven't figured out: "a model is only as good as the context that you actually give it." that's the whole game now. not the prompt. the context. what your agent remembers. what skills it can pull. what it learns from its own past runs. anthropic now ships all three: memory, skills, and dreaming. agents that inspect their own trajectories and self-improve. demoed live this week. the prompt was never the bottleneck. the context was. most people are still tweaking words in a chat box. the ones who get this engineer the context their agents run on. watch the 9 minutes. then read the full breakdown below ⬇️

Avid

33,919 views • 21 days ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,752 views • 1 year ago

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

vitrupo

41,352 views • 2 months ago

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Daniel San

51,642 views • 2 months ago

The Bible in its context is not a Christian book. The Bible in its nature is a catholic book which was edited and given to us in the context of what it is. - Joshua Maponga

The Bible in its context is not a Christian book. The Bible in its nature is a catholic book which was edited and given to us in the context of what it is. - Joshua Maponga

Typical African

12,868 views • 3 months ago

Dario Amodei directly argues that the US Department of War is misrepresenting the law to claim power that the government simply does not have. CBS News asked "Do you think Anthropic can survive this as a business?" Dario effectively says that the Department of War is using social media to trick the public into believing they have powers that Congress never actually gave them. Dario Amadoui confidently states that the impact of the "supply chain risk" designation is actually very limited and will not stop their success, and that Anthropic will be just fine. ---- Video from 'CBS News + CBS Evening News + CBS Mornings' YT channel (link in comment)

Dario Amodei directly argues that the US Department of War is misrepresenting the law to claim power that the government simply does not have. CBS News asked "Do you think Anthropic can survive this as a business?" Dario effectively says that the Department of War is using social media to trick the public into believing they have powers that Congress never actually gave them. Dario Amadoui confidently states that the impact of the "supply chain risk" designation is actually very limited and will not stop their success, and that Anthropic will be just fine. ---- Video from 'CBS News + CBS Evening News + CBS Mornings' YT channel (link in comment)

Rohan Paul

30,664 views • 4 months ago

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

Lior Alexander

83,409 views • 2 years ago

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,179,858 views • 2 months ago

New ways for developers to build with Gemini → 😎 Gemini 1.5 Flash joins 1.5 Pro in public preview via the Gemini API in Google AI Studio 🥳 A new Context Caching feature in the Gemini API 🤗 A preview of our 2 million context window #GoogleIO

New ways for developers to build with Gemini → 😎 Gemini 1.5 Flash joins 1.5 Pro in public preview via the Gemini API in Google AI Studio 🥳 A new Context Caching feature in the Gemini API 🤗 A preview of our 2 million context window #GoogleIO

Google for Developers

17,467 views • 2 years ago

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 views • 1 month ago

Anthropic CEO Dario Amodei says there will be "a great acceleration" due to AI, with 100 years of progress in biology over the next 5-10 years, resulting in a doubling of the human lifespan in that timeframe

Anthropic CEO Dario Amodei says there will be "a great acceleration" due to AI, with 100 years of progress in biology over the next 5-10 years, resulting in a doubling of the human lifespan in that timeframe

Tsarathustra

281,804 views • 1 year ago

Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

Dustin

685,335 views • 4 months ago

There is no wall. AI can combine and synthesize specialized expert knowledge from multiple fields in ways that humans cannot. - Anthropic CEO Dario Amodei From Lex Fridman's podcast.

There is no wall. AI can combine and synthesize specialized expert knowledge from multiple fields in ways that humans cannot. - Anthropic CEO Dario Amodei From Lex Fridman's podcast.

Rohan Paul

18,493 views • 10 months ago

Launched by popular demand. Qwen3-32B is now live on GroqCloud. Supports over 100 languages and dialects with a 131k context window. It is not incremental. It is a leap. Only Groq delivers this level of context with real-time speed. Demo video below to spark your next build. GitHub link in the comments. ⬇️

Launched by popular demand. Qwen3-32B is now live on GroqCloud. Supports over 100 languages and dialects with a 131k context window. It is not incremental. It is a leap. Only Groq delivers this level of context with real-time speed. Demo video below to spark your next build. GitHub link in the comments. ⬇️

Groq Inc

26,498 views • 1 year ago

Anthropic CEO on the transition from human-AI collaboration (Centaur) to full AI dominance. Amodei warns that while demand for coders might spike briefly, the "Centaur" window could be very short before full automation.

Anthropic CEO on the transition from human-AI collaboration (Centaur) to full AI dominance. Amodei warns that while demand for coders might spike briefly, the "Centaur" window could be very short before full automation.

Wes Roth

261,870 views • 4 months ago

you're running /compact every 10 minutes to fix a problem that disappears the second you give Claude one job instead of five. Anthropic shipped a clean 4-minute video on context management. /compact when the window fills up. /clear to reset. small files over big files. the practical tips work. the video covers the cleanup. it doesn't cover why your window keeps filling up in the first place, and the answer is usually that you're holding too many jobs in one prompt. look at what most people put in a single Claude call: > guess the right answer > choose between options > set the position size > watch the exit window > read the news for sentiment five tools' worth of state in one conversation. of course the window fills. i ran into this on a Polymarket bot: the prediction-layer version ate context constantly. /compact every 8-10 turns. i thought i needed better memory hygiene. i needed fewer jobs per call. i stripped the prompt down to one job: veto an entry i had already decided to take. the context window stopped filling. /compact went from every 8 turns to once a session, if ever. over 90 days that prompt blocked 128 trades. watch the 4 min, then ask why your window keeps needing it. clean context. lean prompt. nothing to compact.

you're running /compact every 10 minutes to fix a problem that disappears the second you give Claude one job instead of five. Anthropic shipped a clean 4-minute video on context management. /compact when the window fills up. /clear to reset. small files over big files. the practical tips work. the video covers the cleanup. it doesn't cover why your window keeps filling up in the first place, and the answer is usually that you're holding too many jobs in one prompt. look at what most people put in a single Claude call: > guess the right answer > choose between options > set the position size > watch the exit window > read the news for sentiment five tools' worth of state in one conversation. of course the window fills. i ran into this on a Polymarket bot: the prediction-layer version ate context constantly. /compact every 8-10 turns. i thought i needed better memory hygiene. i needed fewer jobs per call. i stripped the prompt down to one job: veto an entry i had already decided to take. the context window stopped filling. /compact went from every 8 turns to once a session, if ever. over 90 days that prompt blocked 128 trades. watch the 4 min, then ask why your window keeps needing it. clean context. lean prompt. nothing to compact.

Mnimiy

20,635 views • 1 month ago

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 views • 11 months ago