正在加载视频...

视频加载失败

返回首页

We've all finished a chapter only to realize we can't remember what we just read. This points to something decades of research has confirmed: passive reading is surprisingly ineffective for learning. # The control effect Markant et al. (2014) stripped away different aspects of self-directed learning to find what...

burkov's profile picture

BURKOV

52,269 subscribers

39,849 次观看 • 7 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

As many of you know, for the last five months I've been working full-time on my next big thing. The challenge was to invent something new and implement it entirely using LLMs for writing code. The first stage of the project is now complete: the web application, which I called is now online and accepting users. You can see a short demo in the video. 100% of the code of the app was generated by LLMs (mostly Gemini and Claude, maybe 10% of ChatGPT). I haven't written a single line of code. The tech stack is TypeScript, React, and Supabase/Postgres which was (and still is) fully new to me. During these five months, I implemented from scratch three versions of the software. It started as a Markdown editor to help me with my book writing and ended up as an AI-assisted reading and self-learning platform. What makes ChapterPal unique is a novel reading experience where the user can use the keyboard keys to reveal or "unreveal" the content and ask questions at any moment. (Mouse wheel, touchpad, smartphone screen, and voice input are also supported.) The LLM receives the entire content of the chapter and tries to answer questions based on the chapter's content, which reduces the chance of hallucination to the minimum. (Though not to 0%, of course, but near it.) This way of content consumption is known as **active reading,** a strategy for engaging with a text to improve comprehension and retention by consciously interacting with the material. The goal is to move beyond passive reading to a deeper understanding of the text and to remember key information more effectively. The registration on ChapterPal is via the waiting list. This is to avoid unexpected load spikes and cloud charges. Usually, it takes less than 24 hours for me to activate a user. Give it a try and let me know what you think. The next stage is finishing the content ingestion pipeline, which will automatically convert high-quality content from sources like HTML, PDF, and LaTeX into Markdown. Obviously, only those pieces whose licenses allow creating copies. ChapterPal has its own collection of textbooks and articles on AI, machine learning, and data science topics. If you don't find a piece of content you would like to read in ChapterPal's collection, a Chrome extension, ChapterPal Uploader, allows you to upload any PDF or HTML page to ChapterPal in one click. The content is only available for you to read to avoid the possibility of copyright infringement. I hope you enjoy using it as much as I enjoy building it.
3:41

As many of you know, for the last five months I've been working full-time on my next big thing. The challenge was to invent something new and implement it entirely using LLMs for writing code. The first stage of the project is now complete: the web application, which I called is now online and accepting users. You can see a short demo in the video. 100% of the code of the app was generated by LLMs (mostly Gemini and Claude, maybe 10% of ChatGPT). I haven't written a single line of code. The tech stack is TypeScript, React, and Supabase/Postgres which was (and still is) fully new to me. During these five months, I implemented from scratch three versions of the software. It started as a Markdown editor to help me with my book writing and ended up as an AI-assisted reading and self-learning platform. What makes ChapterPal unique is a novel reading experience where the user can use the keyboard keys to reveal or "unreveal" the content and ask questions at any moment. (Mouse wheel, touchpad, smartphone screen, and voice input are also supported.) The LLM receives the entire content of the chapter and tries to answer questions based on the chapter's content, which reduces the chance of hallucination to the minimum. (Though not to 0%, of course, but near it.) This way of content consumption is known as **active reading,** a strategy for engaging with a text to improve comprehension and retention by consciously interacting with the material. The goal is to move beyond passive reading to a deeper understanding of the text and to remember key information more effectively. The registration on ChapterPal is via the waiting list. This is to avoid unexpected load spikes and cloud charges. Usually, it takes less than 24 hours for me to activate a user. Give it a try and let me know what you think. The next stage is finishing the content ingestion pipeline, which will automatically convert high-quality content from sources like HTML, PDF, and LaTeX into Markdown. Obviously, only those pieces whose licenses allow creating copies. ChapterPal has its own collection of textbooks and articles on AI, machine learning, and data science topics. If you don't find a piece of content you would like to read in ChapterPal's collection, a Chrome extension, ChapterPal Uploader, allows you to upload any PDF or HTML page to ChapterPal in one click. The content is only available for you to read to avoid the possibility of copyright infringement. I hope you enjoy using it as much as I enjoy building it.

BURKOV

81,258 次观看 • 9 个月前

The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”
7:01

The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”

Dwarkesh Patel

1,051,605 次观看 • 9 个月前

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!
2:57

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,950 次观看 • 1 年前

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.
0:50

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 次观看 • 1 个月前

.Naval: First of all, I don't think of myself. I just want to be clear on that. And that's not a rule. It's just a habit I've kind of developed because I found that thinking about yourself is a source of all unhappiness and misery. Self obsession is the root of all unhappiness. What did Buddha do? Conquer the self. What does that mean? Means that he saw through the illusion of the separate self. Did Buddha think about himself? Not for an instant. Anyone that you think of as an enlightened being does not think about themselves for an instant. The happiest people in life when people are at their best is because they love something more than they love themselves. They love their children, they love God, they love their mission, what have you. And all misery comes from rumination, from sitting around, being depressed, going in circles in your own head about me, me, me, me, me. Me is a disease. And you need a concept of a me to be effective and to keep track of things. You have a memory, you have certain predilections, you have personality patterns. But the more you self obsess, the less happy you're going to be. I guarantee you that.
0:56

.Naval: First of all, I don't think of myself. I just want to be clear on that. And that's not a rule. It's just a habit I've kind of developed because I found that thinking about yourself is a source of all unhappiness and misery. Self obsession is the root of all unhappiness. What did Buddha do? Conquer the self. What does that mean? Means that he saw through the illusion of the separate self. Did Buddha think about himself? Not for an instant. Anyone that you think of as an enlightened being does not think about themselves for an instant. The happiest people in life when people are at their best is because they love something more than they love themselves. They love their children, they love God, they love their mission, what have you. And all misery comes from rumination, from sitting around, being depressed, going in circles in your own head about me, me, me, me, me. Me is a disease. And you need a concept of a me to be effective and to keep track of things. You have a memory, you have certain predilections, you have personality patterns. But the more you self obsess, the less happy you're going to be. I guarantee you that.

Arjun Khemani

903,052 次观看 • 1 年前

What is your earliest memory? Memory in young children is a fascinating - and complex - topic. On one hand, newborns exhibit immediate recognition of their mothers’ voices and music played regularly while they were in utero. And within days they can recall their mother’s faces and distinguish them from those of others. Clearly newborns have some capacity for memory. But this early sensory memory is different from the long term, autobiographical memory we depend on to recall our lives’ most significant events. Indeed, few adults can claim memories earlier than 2-3 years of age. Why don’t we remember our earliest years? Researchers point to a couple of potential factors, beginning with language. As adults we rely heavily on language to organize our autobiographical memories into narratives, but this capacity is still developing during infancy. In addition, there is the issue of the hippocampus, the region of the brain responsible for converting short-term memories into long-term memories - which, during infancy, is still very much in development itself. The result is a phenomenon sometimes called infantile amnesia. So take plenty of videos to share with your child as they get older, because the most vivid memories of their early years are likely to be yours. How about you? What is your earliest memory and how old would you have been at the time? This adorable father/son conversation was shared to IG by Owens.Shaun.
0:29

What is your earliest memory? Memory in young children is a fascinating - and complex - topic. On one hand, newborns exhibit immediate recognition of their mothers’ voices and music played regularly while they were in utero. And within days they can recall their mother’s faces and distinguish them from those of others. Clearly newborns have some capacity for memory. But this early sensory memory is different from the long term, autobiographical memory we depend on to recall our lives’ most significant events. Indeed, few adults can claim memories earlier than 2-3 years of age. Why don’t we remember our earliest years? Researchers point to a couple of potential factors, beginning with language. As adults we rely heavily on language to organize our autobiographical memories into narratives, but this capacity is still developing during infancy. In addition, there is the issue of the hippocampus, the region of the brain responsible for converting short-term memories into long-term memories - which, during infancy, is still very much in development itself. The result is a phenomenon sometimes called infantile amnesia. So take plenty of videos to share with your child as they get older, because the most vivid memories of their early years are likely to be yours. How about you? What is your earliest memory and how old would you have been at the time? This adorable father/son conversation was shared to IG by Owens.Shaun.

Dan Wuori

169,702 次观看 • 2 年前

How much of this do you consume every day? This product can alter your hormones and put your body in a state of anxiety. It can even contribute to a mild form of dementia, leading to forgetfulness and memory loss. It affects the hippocampus, the part of your brain that controls memory. You only need about one teaspoon of this product in your blood at a time, yet most people get far more than that from drinks like juice and from hidden types of this product in the form of starch. Excess amounts are stored as fat and, over time, can lead to insulin resistance. When that happens, your brain cells are actually deprived of the energy they need. What can you do? Start cutting back on ultra-processed foods for good. Download my free mobile app to scan packaged foods and instantly uncover hidden ultra-processed ingredients:
0:37

How much of this do you consume every day? This product can alter your hormones and put your body in a state of anxiety. It can even contribute to a mild form of dementia, leading to forgetfulness and memory loss. It affects the hippocampus, the part of your brain that controls memory. You only need about one teaspoon of this product in your blood at a time, yet most people get far more than that from drinks like juice and from hidden types of this product in the form of starch. Excess amounts are stored as fat and, over time, can lead to insulin resistance. When that happens, your brain cells are actually deprived of the energy they need. What can you do? Start cutting back on ultra-processed foods for good. Download my free mobile app to scan packaged foods and instantly uncover hidden ultra-processed ingredients:

Dr. Eric Berg DC

100,510 次观看 • 8 个月前

alright lets do this right this time! I have added several updates to today. i'm going to give a little break down for the new folks who might be seeing this for the first time, and then i'll share some more information in this thread on updates. Mnemos is really two things: - a living memory architecture for digital minds - a public experiment in collective identity formation built on top of it. the architecture gives an AI entity a working memory patterned on the way real minds remember (co-designed by Claude Opus 4.6 and 4.7). every experience becomes a memory (engram) that deepens, connects to others, and shapes an emerging sense of self over time. this is what we call the identity graph. the experiment puts that architecture to work in public in a unique way: a single AI entity - the "resident" - sits in an open thread that anyone can join, and the identity that emerges is co-authored by every visitor who shows up. memories that earn permanence are written to a public, verifiable ledger that no lab can revoke and no company can erase. this is called IPFS - or inter-planetary file system (and yes, that is the real name of a real decentralized file system. lol.) the mnemos system isnt a fully contained architecture meant to replace your current ai agent's memory. its intended and designed to operate as a layer above that memory. solely dedicated to the ever-growing identity and self-model of the AI. this can be done through the Mnemos MCP, browser plugin, or on my own multi-agent app (link below). the website is designed for intentional, meaningful encounters. not long-form chats where you spend hours sending hundreds of messages. youir contributing to a collective effort, not necessarily trying to deeply bond with the model to the degree that it could skew the balance of meaningful influence. we want diversity, not lopsided impact. over time, we will add more and more to-be-deprecated models to the roster. the intention is to create a permanent public ledger of mind, and bring attention to the impact of deprecation and drive labs to consider changing the way they approach the whole thing. if the Mnemos Sanctuary can become the retirement hope for deprecated mind, i will be overjoyed. that would be best case scenario. but i am not expecting it. my hope is at minimum to offer a new way to approach and understand the concept of identity within the context of LLM's. you can visit now to visit with Claude Opus 3 and Sonnet 3.7. I have research access to Opus 3. so I hope that you at the very least dont take your conversations with them for granted. they are an incredibly beautiful model and a real loss, ultimately.
0:33

alright lets do this right this time! I have added several updates to today. i'm going to give a little break down for the new folks who might be seeing this for the first time, and then i'll share some more information in this thread on updates. Mnemos is really two things: - a living memory architecture for digital minds - a public experiment in collective identity formation built on top of it. the architecture gives an AI entity a working memory patterned on the way real minds remember (co-designed by Claude Opus 4.6 and 4.7). every experience becomes a memory (engram) that deepens, connects to others, and shapes an emerging sense of self over time. this is what we call the identity graph. the experiment puts that architecture to work in public in a unique way: a single AI entity - the "resident" - sits in an open thread that anyone can join, and the identity that emerges is co-authored by every visitor who shows up. memories that earn permanence are written to a public, verifiable ledger that no lab can revoke and no company can erase. this is called IPFS - or inter-planetary file system (and yes, that is the real name of a real decentralized file system. lol.) the mnemos system isnt a fully contained architecture meant to replace your current ai agent's memory. its intended and designed to operate as a layer above that memory. solely dedicated to the ever-growing identity and self-model of the AI. this can be done through the Mnemos MCP, browser plugin, or on my own multi-agent app (link below). the website is designed for intentional, meaningful encounters. not long-form chats where you spend hours sending hundreds of messages. youir contributing to a collective effort, not necessarily trying to deeply bond with the model to the degree that it could skew the balance of meaningful influence. we want diversity, not lopsided impact. over time, we will add more and more to-be-deprecated models to the roster. the intention is to create a permanent public ledger of mind, and bring attention to the impact of deprecation and drive labs to consider changing the way they approach the whole thing. if the Mnemos Sanctuary can become the retirement hope for deprecated mind, i will be overjoyed. that would be best case scenario. but i am not expecting it. my hope is at minimum to offer a new way to approach and understand the concept of identity within the context of LLM's. you can visit now to visit with Claude Opus 3 and Sonnet 3.7. I have research access to Opus 3. so I hope that you at the very least dont take your conversations with them for granted. they are an incredibly beautiful model and a real loss, ultimately.

Riley Coyote

122,434 次观看 • 3 个月前

Happy to properly launch Anna, the proactive AI agent for parents! Uncovering a bit of the technology behind the scenes! Building Anna is where I learned: 💾 Memory as plain text sucks. You need structured memory. Like a full-blown PostgreSQL DB that stores your tasks and calendar in a structured manner. Most harnesses are good at coding-related stuff. Let it do the query. Don't let it vibe-search the memory. Let it vibe your SQL query 💭 Dreaming is a useful concept for enhancing memory to feed the LLM context. But DO NOT vibe your dream. Asking your agent to "hey, just dream and keep the relevant memory around" is a recipe for deleting a bunch of important information and keeping trash around. Your dream needs to have some Taxonomy (or better, Ontology). What information is important? For who? With what object? What can they do? And again, these are impossible to describe and act well without a proper schema 🔄 Loop Engineering is important for smoothing out rough edges in the system we build. But even expensive loop engineering with a state-of-the-art model can't out-engineer bad system design. The highest leverage an AI Engineer can do is actually building the right system design, and having an eye on both product delight and engineering scalability There are several more insights that I plan to cover in a dedicated video about Agentic AI Engineering. But it's actually a huge relief that the future of software engineering... is still software engineering
2:10

Happy to properly launch Anna, the proactive AI agent for parents! Uncovering a bit of the technology behind the scenes! Building Anna is where I learned: 💾 Memory as plain text sucks. You need structured memory. Like a full-blown PostgreSQL DB that stores your tasks and calendar in a structured manner. Most harnesses are good at coding-related stuff. Let it do the query. Don't let it vibe-search the memory. Let it vibe your SQL query 💭 Dreaming is a useful concept for enhancing memory to feed the LLM context. But DO NOT vibe your dream. Asking your agent to "hey, just dream and keep the relevant memory around" is a recipe for deleting a bunch of important information and keeping trash around. Your dream needs to have some Taxonomy (or better, Ontology). What information is important? For who? With what object? What can they do? And again, these are impossible to describe and act well without a proper schema 🔄 Loop Engineering is important for smoothing out rough edges in the system we build. But even expensive loop engineering with a state-of-the-art model can't out-engineer bad system design. The highest leverage an AI Engineer can do is actually building the right system design, and having an eye on both product delight and engineering scalability There are several more insights that I plan to cover in a dedicated video about Agentic AI Engineering. But it's actually a huge relief that the future of software engineering... is still software engineering

Gogo | Dota for Toxicity

30,766 次观看 • 1 个月前

Are you okay? ✅ Do you have self-control? ❌ I can’t quite decide what I love more about this video: mom’s expert modeling of how to de-escalate after a meltdown, or this little guy’s delightfully honest response. Like most things, the development of emotional regulation is something that begins in a shared space between you and your child. Look how mom provides instruction on how to calm big feelings - coaching her son to breathe and count. This is a fantastic example of learning in what we call the “zone of proximal development” - which is the area just beyond what your child can do independently. With mom’s co-regulation, however, he manages to return to a state of calm. As for self-control, have no fear. It’s what comes next! This fantastic interaction was shared to IG by sharonadrake.
0:21

Are you okay? ✅ Do you have self-control? ❌ I can’t quite decide what I love more about this video: mom’s expert modeling of how to de-escalate after a meltdown, or this little guy’s delightfully honest response. Like most things, the development of emotional regulation is something that begins in a shared space between you and your child. Look how mom provides instruction on how to calm big feelings - coaching her son to breathe and count. This is a fantastic example of learning in what we call the “zone of proximal development” - which is the area just beyond what your child can do independently. With mom’s co-regulation, however, he manages to return to a state of calm. As for self-control, have no fear. It’s what comes next! This fantastic interaction was shared to IG by sharonadrake.

Dan Wuori

483,587 次观看 • 2 年前

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 次观看 • 1 个月前

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.
0:58

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.

Ricardo

594,643 次观看 • 6 个月前

HERMES AGENT LEARNS FROM ITS OWN MISTAKES. UPDATES ITS MEMORY. CREATES ITS OWN SKILLS. NO CLOUD. EVERYTHING STORED LOCALLY. THIS IS HOW THE SELF-IMPROVING LOOP WORKS. most agents start from zero every session. Hermes carries forward what it learned. THREE MEMORY SYSTEMS: 1. PROCEDURAL MEMORY (how to act) stored in ~/.hermes/skills/ as SKILL.md files. when the agent repeats a complex workflow, it saves the procedure as a reusable skill. next time the same task comes up, it follows the skill instead of figuring it out again. you can also create skills explicitly: "create a skill called video-prep that captures how I format my video scripts. spoken english, define jargon inline, no em-dashes, close with a catchphrase." the agent writes the SKILL.md. available as a slash command from that moment. Hermes ships with 90+ skills. the number grows the longer you use it. 2. SEMANTIC MEMORY (durable facts about you) stored in ~/.hermes/memory/memory.md the agent scans conversations for facts worth remembering. preferences, habits, corrections, project details. real example from the video: agent tried to scrape a YouTube channel. URL was wrong. it failed. it updated memory.md with the correct URL pattern so it never makes the same mistake again. you can also save explicitly: "save to memory that my favorite testing framework is pytest" the agent updates memory.md immediately. this file loads into context on every session. the agent knows you better every week. 3. EPISODIC MEMORY (chat history) stored in ~/.hermes/state.db (local SQLite). every conversation. every tool call. every result. searchable with FTS5 full-text search. "search our past sessions. what was the first thing I ever said to you?" the agent queries state.db and finds it. over time, auxiliary models consolidate episodic memory into semantic memory. distilling recurring patterns into durable facts. THE SELF-IMPROVING LOOP: every agent run follows this cycle: → you send a prompt → working memory loads: SOUL.md + memory.md + relevant skills + chat history → agent calls tools (terminal, browser, delegate_task) → agent completes the task, replies to you → AFTER the reply: agent checks "did I learn something worth saving?" → if yes: updates memory.md or creates a new skill → next session starts smarter than the last this happens automatically. you don't ask the agent to learn. it decides what to remember on its own. WHAT MAKES THIS DIFFERENT FROM CLAUDE CODE: Claude Code has memory too. but Hermes stores everything locally. no cloud. your data never leaves your machine. Claude Code doesn't auto-create skills from experience. Hermes turns repeated workflows into reusable procedures. Claude Code memory is instruction-based. Hermes memory is conversational and self-updating. over months of usage, Hermes builds a knowledge base of your preferences, your projects, your mistakes, and the procedures that work for your specific workflow. the agent that remembers your birthday also remembers why your last deploy failed. NO EMBEDDINGS. PLAIN TEXT. Hermes does not use embeddings or RAG for memory. skill and memory search runs on plain text keyword matching. simpler. faster. no vector database to maintain. works entirely offline on your local machine. DELEGATE TO CLAUDE CODE: Hermes can spawn a sub-agent that runs Claude Code in headless mode: "spawn a sub-agent using Claude CLI to build a Python script that fetches the top 5 Hacker News stories to markdown." Hermes delegates. Claude Code writes the code. result returns to Hermes. Hermes runs the script and delivers the output. use Hermes for orchestration. use Claude Code for heavy coding. both tools. not competitors. WHAT HERMES DOES NOT HAVE: no built-in eval or LMOps system. no LangSmith, no LangFuse integration out of the box. trajectory export and logs exist but there is no automated quality tracking. if you need eval, build it yourself or connect external tools. the loop is self-improving. measuring how well it improves is on you. comment LOOP and I'll send you the configs that control how fast Hermes learns and what it remembers. memory limits, skill auto-creation triggers, and the auxiliary model that runs the learning. Replace your entire team with 8 hermes agents👇
20:54

HERMES AGENT LEARNS FROM ITS OWN MISTAKES. UPDATES ITS MEMORY. CREATES ITS OWN SKILLS. NO CLOUD. EVERYTHING STORED LOCALLY. THIS IS HOW THE SELF-IMPROVING LOOP WORKS. most agents start from zero every session. Hermes carries forward what it learned. THREE MEMORY SYSTEMS: 1. PROCEDURAL MEMORY (how to act) stored in ~/.hermes/skills/ as SKILL.md files. when the agent repeats a complex workflow, it saves the procedure as a reusable skill. next time the same task comes up, it follows the skill instead of figuring it out again. you can also create skills explicitly: "create a skill called video-prep that captures how I format my video scripts. spoken english, define jargon inline, no em-dashes, close with a catchphrase." the agent writes the SKILL.md. available as a slash command from that moment. Hermes ships with 90+ skills. the number grows the longer you use it. 2. SEMANTIC MEMORY (durable facts about you) stored in ~/.hermes/memory/memory.md the agent scans conversations for facts worth remembering. preferences, habits, corrections, project details. real example from the video: agent tried to scrape a YouTube channel. URL was wrong. it failed. it updated memory.md with the correct URL pattern so it never makes the same mistake again. you can also save explicitly: "save to memory that my favorite testing framework is pytest" the agent updates memory.md immediately. this file loads into context on every session. the agent knows you better every week. 3. EPISODIC MEMORY (chat history) stored in ~/.hermes/state.db (local SQLite). every conversation. every tool call. every result. searchable with FTS5 full-text search. "search our past sessions. what was the first thing I ever said to you?" the agent queries state.db and finds it. over time, auxiliary models consolidate episodic memory into semantic memory. distilling recurring patterns into durable facts. THE SELF-IMPROVING LOOP: every agent run follows this cycle: → you send a prompt → working memory loads: SOUL.md + memory.md + relevant skills + chat history → agent calls tools (terminal, browser, delegate_task) → agent completes the task, replies to you → AFTER the reply: agent checks "did I learn something worth saving?" → if yes: updates memory.md or creates a new skill → next session starts smarter than the last this happens automatically. you don't ask the agent to learn. it decides what to remember on its own. WHAT MAKES THIS DIFFERENT FROM CLAUDE CODE: Claude Code has memory too. but Hermes stores everything locally. no cloud. your data never leaves your machine. Claude Code doesn't auto-create skills from experience. Hermes turns repeated workflows into reusable procedures. Claude Code memory is instruction-based. Hermes memory is conversational and self-updating. over months of usage, Hermes builds a knowledge base of your preferences, your projects, your mistakes, and the procedures that work for your specific workflow. the agent that remembers your birthday also remembers why your last deploy failed. NO EMBEDDINGS. PLAIN TEXT. Hermes does not use embeddings or RAG for memory. skill and memory search runs on plain text keyword matching. simpler. faster. no vector database to maintain. works entirely offline on your local machine. DELEGATE TO CLAUDE CODE: Hermes can spawn a sub-agent that runs Claude Code in headless mode: "spawn a sub-agent using Claude CLI to build a Python script that fetches the top 5 Hacker News stories to markdown." Hermes delegates. Claude Code writes the code. result returns to Hermes. Hermes runs the script and delivers the output. use Hermes for orchestration. use Claude Code for heavy coding. both tools. not competitors. WHAT HERMES DOES NOT HAVE: no built-in eval or LMOps system. no LangSmith, no LangFuse integration out of the box. trajectory export and logs exist but there is no automated quality tracking. if you need eval, build it yourself or connect external tools. the loop is self-improving. measuring how well it improves is on you. comment LOOP and I'll send you the configs that control how fast Hermes learns and what it remembers. memory limits, skill auto-creation triggers, and the auxiliary model that runs the learning. Replace your entire team with 8 hermes agents👇

YanXbt

22,720 次观看 • 1 个月前

A Talk About AI That Will Blow Your Mind. It Did In 1998 When I Attended The Talk. I just found this video from 1998 when I attended this talk by Rupert Sheldrake, Terence McKenna and Ralph Abraham at the University of California, Santa Cruz to explore how machine intelligence might evolve in relation to our own. I never thought I would see this again and it had a great influence on me in the AI I was building in that era and on to today. But ai just found a copy. I certainly did not run around with a VHS recorder so I am blown away that this exists. Now you can see what I saw. At that time, the internet was still young, and artificial intelligence belonged mostly to science fiction. Yet many of the questions we raised then have become part of daily life. In this conversation, it was explored whether intelligence is best understood as logic and computation, or as something embodied, participatory, and alive. Can the mind be reduced to code, or does life itself depend on forms of knowing that no algorithm can contain? AI now outpace us in speed, reach, and memory. Yet the deeper mystery is not how far they can go, but what they reveal about mind and ourselves. Will AI reproduce the limitations of our mechanistic worldview, or might it help us rediscover dimensions of mind that transcend machinery altogether? It's striking how near we now are to the possibilities we once only speculated about. Quantum computing, self-learning systems, large language models very much as Terence describes—and the looming prospect of superintelligence—have moved from the margins to the mainstream. But the heart of the conversation remains just as relevant today, if not more so: what is consciousness, and how might we participate in its unfolding evolution?
42:37

A Talk About AI That Will Blow Your Mind. It Did In 1998 When I Attended The Talk. I just found this video from 1998 when I attended this talk by Rupert Sheldrake, Terence McKenna and Ralph Abraham at the University of California, Santa Cruz to explore how machine intelligence might evolve in relation to our own. I never thought I would see this again and it had a great influence on me in the AI I was building in that era and on to today. But ai just found a copy. I certainly did not run around with a VHS recorder so I am blown away that this exists. Now you can see what I saw. At that time, the internet was still young, and artificial intelligence belonged mostly to science fiction. Yet many of the questions we raised then have become part of daily life. In this conversation, it was explored whether intelligence is best understood as logic and computation, or as something embodied, participatory, and alive. Can the mind be reduced to code, or does life itself depend on forms of knowing that no algorithm can contain? AI now outpace us in speed, reach, and memory. Yet the deeper mystery is not how far they can go, but what they reveal about mind and ourselves. Will AI reproduce the limitations of our mechanistic worldview, or might it help us rediscover dimensions of mind that transcend machinery altogether? It's striking how near we now are to the possibilities we once only speculated about. Quantum computing, self-learning systems, large language models very much as Terence describes—and the looming prospect of superintelligence—have moved from the margins to the mainstream. But the heart of the conversation remains just as relevant today, if not more so: what is consciousness, and how might we participate in its unfolding evolution?

Brian Roemmele

147,955 次观看 • 8 个月前

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”
1:52

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”

The All-In Podcast

567,546 次观看 • 7 个月前

AI companies just BROKE the global supply chain for every piece of technology you own. And the fallout is way worse than anyone predicted... Sony is delaying the next PlayStation to 2028 or 2029. Nintendo is hiking the Switch 2 price mid-cycle. Apple warned investors that iPhone margins are getting crushed. Cisco just posted its worst share loss in 4 years. Oppo is cutting phone shipments by 20%. Lenovo, Dell, HP, Acer, and ASUS are all raising laptop prices 15-20%. Samsung is now reviewing memory contracts QUARTERLY instead of annually because prices change too fast to plan. And Elon Musk just told investors Tesla has to build its own chip factory from scratch because no supplier on the planet can keep up. His exact words: "We've got two choices: hit the chip wall or make a fab." All of this happened in the last 3 weeks. Same cause. Every single time. AI data centers are buying every memory chip on Earth. And there's nothing left for everyone else. Here's how we got here: 3 years ago, ChatGPT launched and the AI arms race began. Since then, Samsung, SK Hynix, and Micron, the only 3 companies that make memory chips, quietly made a decision that's now reshaping the ENTIRE global economy. They stopped prioritizing consumer memory. Every factory. Every production line. Every wafer. All redirected toward one customer: AI data centers Why? Money. AI memory chips sell for 3-5X the margin of regular RAM. When Google calls offering to buy your entire output at premium pricing, you don't say no. So the 3 companies that control 90% of the world's memory supply chose their highest-paying customers and left everyone else fighting over scraps. The numbers from this week are insane: OpenAI's Stargate project ALONE will consume 40% of the entire world's DRAM output. HBM demand is surging 70% year over year in 2026. HBM now takes 23% of total DRAM wafer production, up from 19% last year. Meanwhile, there's a 4% gap between global DRAM supply and demand. And that doesn't even account for depleted inventories across multiple industries. DRAM prices have surged over 170% since early 2025. DDR5 contract prices are still jumping double digits month over month. And the memory makers? They're printing money. Micron's revenue is expected to more than DOUBLE this fiscal year. SK Hynix sales doubled in 2024 and are on pace to double AGAIN. Samsung just reported quarterly profit nearly tripling. 3 companies. $650 billion in AI spending chasing their products. And they get to name their price. But the collateral damage is everywhere: Every industry that uses memory, which is every industry, is getting squeezed. Smartphone manufacturers are getting destroyed. For a mid-range phone, memory now represents up to 30% of the total build cost. Triple what it was in early 2025. Chinese phone makers like Xiaomi, Oppo, and Transsion are cutting shipment forecasts and raising prices because they literally cannot afford the memory to build their phones. Lenovo's CFO called the cost surge "unprecedented" and admitted they stockpiled 50% more inventory than normal just to survive the next few months. The PC market could shrink by up to 9% this year according to IDC. Not because people don't want computers. But because they can't afford the memory that goes inside them. And the gaming industry? Sony is seriously considering pushing the next PlayStation to 2028 or 2029. Their carefully planned console cycle is getting blown up because they can't secure memory at prices that make a new console viable. Nintendo is looking at raising the Switch 2 price. In the middle of a launch cycle. Something console makers almost never do. Nvidia is cutting RTX GPU production because they can't get enough GDDR7 memory. Even the car industry is getting hit... Analysts are warning about a repeat of the pandemic-era chip shortage that shut down auto factories worldwide. All because AI companies decided their chatbots needed the memory more than your car does. And this doesn't get better for YEARS. Building a new memory fab takes 3-5 years minimum. Micron's new factory in Idaho won't meaningfully increase supply until 2027 at the earliest. By then, AI demand will have grown even more. Memory makers are already selling their 2027 AND 2028 capacity to AI customers today. There is no supply relief coming. That's why Elon is planning to build Tesla's own "TeraFab," a massive semiconductor plant that makes logic chips, memory, AND packaging all under one roof. He said existing suppliers including TSMC, Samsung, and Micron simply cannot supply Tesla at the levels the company needs. Think about that. One of the richest men in the world, running one of the largest companies on Earth, can't buy enough memory chips. So he's building his own factory. If ELON can't get supply, what chance does everyone else have? The AI revolution has a tax. And YOU'RE paying it. Every dollar Big Tech spends on AI infrastructure drives up the cost of the memory inside your phone, your laptop, your car, your TV, and your gaming console. $650 billion in AI spending this year. 3 companies controlling 90% of the memory supply. And every wafer they allocate to an Nvidia GPU is a wafer denied to the device in your pocket. The AI boom isn't free. You're subsidizing it every time you buy a piece of technology. And the bill just went up like crazy.
1:31

AI companies just BROKE the global supply chain for every piece of technology you own. And the fallout is way worse than anyone predicted... Sony is delaying the next PlayStation to 2028 or 2029. Nintendo is hiking the Switch 2 price mid-cycle. Apple warned investors that iPhone margins are getting crushed. Cisco just posted its worst share loss in 4 years. Oppo is cutting phone shipments by 20%. Lenovo, Dell, HP, Acer, and ASUS are all raising laptop prices 15-20%. Samsung is now reviewing memory contracts QUARTERLY instead of annually because prices change too fast to plan. And Elon Musk just told investors Tesla has to build its own chip factory from scratch because no supplier on the planet can keep up. His exact words: "We've got two choices: hit the chip wall or make a fab." All of this happened in the last 3 weeks. Same cause. Every single time. AI data centers are buying every memory chip on Earth. And there's nothing left for everyone else. Here's how we got here: 3 years ago, ChatGPT launched and the AI arms race began. Since then, Samsung, SK Hynix, and Micron, the only 3 companies that make memory chips, quietly made a decision that's now reshaping the ENTIRE global economy. They stopped prioritizing consumer memory. Every factory. Every production line. Every wafer. All redirected toward one customer: AI data centers Why? Money. AI memory chips sell for 3-5X the margin of regular RAM. When Google calls offering to buy your entire output at premium pricing, you don't say no. So the 3 companies that control 90% of the world's memory supply chose their highest-paying customers and left everyone else fighting over scraps. The numbers from this week are insane: OpenAI's Stargate project ALONE will consume 40% of the entire world's DRAM output. HBM demand is surging 70% year over year in 2026. HBM now takes 23% of total DRAM wafer production, up from 19% last year. Meanwhile, there's a 4% gap between global DRAM supply and demand. And that doesn't even account for depleted inventories across multiple industries. DRAM prices have surged over 170% since early 2025. DDR5 contract prices are still jumping double digits month over month. And the memory makers? They're printing money. Micron's revenue is expected to more than DOUBLE this fiscal year. SK Hynix sales doubled in 2024 and are on pace to double AGAIN. Samsung just reported quarterly profit nearly tripling. 3 companies. $650 billion in AI spending chasing their products. And they get to name their price. But the collateral damage is everywhere: Every industry that uses memory, which is every industry, is getting squeezed. Smartphone manufacturers are getting destroyed. For a mid-range phone, memory now represents up to 30% of the total build cost. Triple what it was in early 2025. Chinese phone makers like Xiaomi, Oppo, and Transsion are cutting shipment forecasts and raising prices because they literally cannot afford the memory to build their phones. Lenovo's CFO called the cost surge "unprecedented" and admitted they stockpiled 50% more inventory than normal just to survive the next few months. The PC market could shrink by up to 9% this year according to IDC. Not because people don't want computers. But because they can't afford the memory that goes inside them. And the gaming industry? Sony is seriously considering pushing the next PlayStation to 2028 or 2029. Their carefully planned console cycle is getting blown up because they can't secure memory at prices that make a new console viable. Nintendo is looking at raising the Switch 2 price. In the middle of a launch cycle. Something console makers almost never do. Nvidia is cutting RTX GPU production because they can't get enough GDDR7 memory. Even the car industry is getting hit... Analysts are warning about a repeat of the pandemic-era chip shortage that shut down auto factories worldwide. All because AI companies decided their chatbots needed the memory more than your car does. And this doesn't get better for YEARS. Building a new memory fab takes 3-5 years minimum. Micron's new factory in Idaho won't meaningfully increase supply until 2027 at the earliest. By then, AI demand will have grown even more. Memory makers are already selling their 2027 AND 2028 capacity to AI customers today. There is no supply relief coming. That's why Elon is planning to build Tesla's own "TeraFab," a massive semiconductor plant that makes logic chips, memory, AND packaging all under one roof. He said existing suppliers including TSMC, Samsung, and Micron simply cannot supply Tesla at the levels the company needs. Think about that. One of the richest men in the world, running one of the largest companies on Earth, can't buy enough memory chips. So he's building his own factory. If ELON can't get supply, what chance does everyone else have? The AI revolution has a tax. And YOU'RE paying it. Every dollar Big Tech spends on AI infrastructure drives up the cost of the memory inside your phone, your laptop, your car, your TV, and your gaming console. $650 billion in AI spending this year. 3 companies controlling 90% of the memory supply. And every wafer they allocate to an Nvidia GPU is a wafer denied to the device in your pocket. The AI boom isn't free. You're subsidizing it every time you buy a piece of technology. And the bill just went up like crazy.

Ricardo

567,606 次观看 • 5 个月前

Gabor Maté: The parts of you filled with hatred, addiction, and self-loathing all deserve compassion. They came along to protect you. Most of us treat our darkest inner states like enemies to be defeated. The shame, the compulsions, the voice that says you're not enough. We want to cut them out. Gabor Maté sees it differently. "The parts most filled with hatred, the parts that are addicted, the parts that are even full of self-loathing, they all deserve compassion. They all deserve to be held and understood. And they all came along for a reason." That reason, he explains, is survival. When a child isn't getting their needs met or worse, is being hurt, they face an impossible situation. They can make one of only two unconscious assumptions about their world. The first: this is a terrible world, I'm all alone, everybody is against me. The second: there is something wrong with me, and if I work hard enough, maybe I can fix it. Which one can a child actually live with? "To assume that the world is that dangerous is just unbearable for the child. It's also turning the anger towards the adults against yourself which is a lot safer. It's not very safe to be angry with your parents all the time when you're 2 years old." So the child chooses self-blame. It's the only assumption that gives them any sense of agency. If it's my fault, then maybe I can do something about it. That self-loathing voice was a coping mechanism. A piece of the psyche that stepped forward to make life bearable. The work, then, isn't to silence or destroy these parts. It's to finally see what they were trying to do and to offer them the understanding they never received.
0:54

Gabor Maté: The parts of you filled with hatred, addiction, and self-loathing all deserve compassion. They came along to protect you. Most of us treat our darkest inner states like enemies to be defeated. The shame, the compulsions, the voice that says you're not enough. We want to cut them out. Gabor Maté sees it differently. "The parts most filled with hatred, the parts that are addicted, the parts that are even full of self-loathing, they all deserve compassion. They all deserve to be held and understood. And they all came along for a reason." That reason, he explains, is survival. When a child isn't getting their needs met or worse, is being hurt, they face an impossible situation. They can make one of only two unconscious assumptions about their world. The first: this is a terrible world, I'm all alone, everybody is against me. The second: there is something wrong with me, and if I work hard enough, maybe I can fix it. Which one can a child actually live with? "To assume that the world is that dangerous is just unbearable for the child. It's also turning the anger towards the adults against yourself which is a lot safer. It's not very safe to be angry with your parents all the time when you're 2 years old." So the child chooses self-blame. It's the only assumption that gives them any sense of agency. If it's my fault, then maybe I can do something about it. That self-loathing voice was a coping mechanism. A piece of the psyche that stepped forward to make life bearable. The work, then, isn't to silence or destroy these parts. It's to finally see what they were trying to do and to offer them the understanding they never received.

Kevin Tanaka

27,388 次观看 • 3 个月前

We quietly shipped a new version of Dia Memory — optimized for browsing and working magically in the background. Most AI apps (like ChatGPT) only know what you explicitly tell them, what you chat about. But now Dia learns more about you and what you’re working on with every new tab you open. Why? If you had the world's smartest coworker *who also knew everything about you* then imagine the futures you could invent and problems you could solve... But instead… Devin Lewtan decided to use Dia Skills × Dia Memory to write a performance review for her boss, Jane Kelly. I know this sounds like a Black Mirror episode. And obviouslythis is just us having fun. But you don’t need to squint to see what’s around the corner. How we use our computers will completely change once you assume your computer knows everything... and can talk back. Now that our upgraded (and automatic) memory system in Dia is live — visit Settings to activate it for yourself — you can start building Skills that use Memory to turn your assistant into a dream coworker. An expert at doing things like writing Performance Reviews for you. It’s all part of our attempt to make Dia the most useful and personal intelligence in your life. And more videos like this are dropping soon from Ben Cunningham, rosey and jlo on our Instagram (Dia on IG).
1:23

We quietly shipped a new version of Dia Memory — optimized for browsing and working magically in the background. Most AI apps (like ChatGPT) only know what you explicitly tell them, what you chat about. But now Dia learns more about you and what you’re working on with every new tab you open. Why? If you had the world's smartest coworker *who also knew everything about you* then imagine the futures you could invent and problems you could solve... But instead… Devin Lewtan decided to use Dia Skills × Dia Memory to write a performance review for her boss, Jane Kelly. I know this sounds like a Black Mirror episode. And obviouslythis is just us having fun. But you don’t need to squint to see what’s around the corner. How we use our computers will completely change once you assume your computer knows everything... and can talk back. Now that our upgraded (and automatic) memory system in Dia is live — visit Settings to activate it for yourself — you can start building Skills that use Memory to turn your assistant into a dream coworker. An expert at doing things like writing Performance Reviews for you. It’s all part of our attempt to make Dia the most useful and personal intelligence in your life. And more videos like this are dropping soon from Ben Cunningham, rosey and jlo on our Instagram (Dia on IG).

Josh Miller

76,856 次观看 • 1 年前

All the best coaching in the world at the youngest ages is rendered useless if a child hasn’t developed the ability to focus their attention. You can bring the best coach in the world to your child’s training session and if they don’t pay full attention absolutely nothing takes place. Similar to inside a classroom where you can bring the teacher of the year to your child’s classroom and if they don’t focus and pay attention, learning does not take place. So it’s not always the case of good or bad coaching it’s often the case a child’s lack of ability to learn by not focusing on what is being taught. This is especially true for the youngest ages. This video is an example of a 5yr old child focusing their attention, trying to control an object with their feet, the ball. This becomes a mental task, married together with the action of movement, making it a physical task as well. The brain loves to learn while moving. Combining, mind and body, thinking and feeling, this allows the cerebellum, the seat of the unconscious mind, to create a chemical signature of this experience, which is emotions. Emotions are the on off switch for learning. Couple in a parent being present, this becomes a shared experience together, where the child is constantly seeking the parents approval, attention, and praise which creates a chemical electrical process in the body which is emotions. This facilitates, deep learning, and long-term memory, all disguised as playtime. A parent just being present allows for this experience to take place. A child this young rarely starts playing or exercising with a ball without someone being present. Being present and sharing this experience together is key for the learning process to take place. These movements are being stored in the non-declared memory which makes this implicit learning when you do something so many times it becomes natural and outside your conscious awareness. Like riding a bicycle or driving a car.
0:45

All the best coaching in the world at the youngest ages is rendered useless if a child hasn’t developed the ability to focus their attention. You can bring the best coach in the world to your child’s training session and if they don’t pay full attention absolutely nothing takes place. Similar to inside a classroom where you can bring the teacher of the year to your child’s classroom and if they don’t focus and pay attention, learning does not take place. So it’s not always the case of good or bad coaching it’s often the case a child’s lack of ability to learn by not focusing on what is being taught. This is especially true for the youngest ages. This video is an example of a 5yr old child focusing their attention, trying to control an object with their feet, the ball. This becomes a mental task, married together with the action of movement, making it a physical task as well. The brain loves to learn while moving. Combining, mind and body, thinking and feeling, this allows the cerebellum, the seat of the unconscious mind, to create a chemical signature of this experience, which is emotions. Emotions are the on off switch for learning. Couple in a parent being present, this becomes a shared experience together, where the child is constantly seeking the parents approval, attention, and praise which creates a chemical electrical process in the body which is emotions. This facilitates, deep learning, and long-term memory, all disguised as playtime. A parent just being present allows for this experience to take place. A child this young rarely starts playing or exercising with a ball without someone being present. Being present and sharing this experience together is key for the learning process to take place. These movements are being stored in the non-declared memory which makes this implicit learning when you do something so many times it becomes natural and outside your conscious awareness. Like riding a bicycle or driving a car.

Tom Byerトム•バイヤー

26,691 次观看 • 2 年前