Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🪞Symmetry helps shape RL policies. 🧠Can it also improve memory and task understanding? ✅We introduce Symmetry-Guided Memory Augmentation, combining structured experience augmentation with memory-based context inference to boost learning efficiency. 🔗

breadli428's profile picture

Chenhao Li

6,942 subscribers

14,863 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Automatically create Agent Skills from past sessions. Gemini CLI can now comb through past session data and suggest new skills based on past patterns of things you do frequently! Helps the agent self-improve. 🧠 Enable Auto Memory in /settings to try it out.
0:20

Automatically create Agent Skills from past sessions. Gemini CLI can now comb through past session data and suggest new skills based on past patterns of things you do frequently! Helps the agent self-improve. 🧠 Enable Auto Memory in /settings to try it out.

Jack Wotherspoon

24,793 Aufrufe • vor 3 Monaten

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!
1:37

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

Andrew Ng

160,148 Aufrufe • vor 4 Monaten

Context memory essentially unlocks Agentic AI Much needed for Opus 4.6's "multi-agent swarms" In this SemiDoped pod, Vikram Sekar talks to Val Bercovici from Weka about context storage. - How token warehouses save inference costs - A new networking tier? Context Storage Network! - High Bandwidth Flash for context? - Weka's Augmented Memory Grid for context storage - Where this is all headed The convo is info packed. Don't miss out on it! b/acc, context platform engineer Chapters (00:00) Introduction to Weka and AI Storage Solutions (05:18) The Evolution of Context Memory in AI (09:30) Understanding Memory Hierarchies and Their Impact (16:24) Latency Challenges in Modern Storage Solutions (21:32) The Role of Networking in AI Storage Efficiency (29:42) Dynamic Resource Utilization in AI Networks (30:04) Introducing the Context Memory Network (31:13) High Bandwidth Flash: A Game Changer (32:54) Weka’s Neural Mesh and Storage Solutions (35:01) Axon: Transforming GPU Storage into Memory (39:00) Augmented Memory Grid Explained (42:00) Pooling DRAM and CXL Innovations (46:02) Token Warehouses and Inference Economics (52:10) The Future of Storage Innovations
54:27

Context memory essentially unlocks Agentic AI Much needed for Opus 4.6's "multi-agent swarms" In this SemiDoped pod, Vikram Sekar talks to Val Bercovici from Weka about context storage. - How token warehouses save inference costs - A new networking tier? Context Storage Network! - High Bandwidth Flash for context? - Weka's Augmented Memory Grid for context storage - Where this is all headed The convo is info packed. Don't miss out on it! b/acc, context platform engineer Chapters (00:00) Introduction to Weka and AI Storage Solutions (05:18) The Evolution of Context Memory in AI (09:30) Understanding Memory Hierarchies and Their Impact (16:24) Latency Challenges in Modern Storage Solutions (21:32) The Role of Networking in AI Storage Efficiency (29:42) Dynamic Resource Utilization in AI Networks (30:04) Introducing the Context Memory Network (31:13) High Bandwidth Flash: A Game Changer (32:54) Weka’s Neural Mesh and Storage Solutions (35:01) Axon: Transforming GPU Storage into Memory (39:00) Augmented Memory Grid Explained (42:00) Pooling DRAM and CXL Innovations (46:02) Token Warehouses and Inference Economics (52:10) The Future of Storage Innovations

Semi Doped

12,796 Aufrufe • vor 6 Monaten

Dr. Mark D'Esposito (Mark D'Esposito), neurologist and professor at UC Berkeley, discusses medications that may improve memory and brain function. "There are a number of people out there with challenges in working memory, attention, task switching, etc." "When we get to a point where we can map out everyone's dopamine, norepinephrine, serotonin levels, then we'll make real progress in helping them."
8:28

Dr. Mark D'Esposito (Mark D'Esposito), neurologist and professor at UC Berkeley, discusses medications that may improve memory and brain function. "There are a number of people out there with challenges in working memory, attention, task switching, etc." "When we get to a point where we can map out everyone's dopamine, norepinephrine, serotonin levels, then we'll make real progress in helping them."

Huberman Lab Clips

44,029 Aufrufe • vor 1 Jahr

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”
1:31

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 Aufrufe • vor 2 Monaten

Long-running agents blow up context fast. So we built our own memory agent. Using it feels very cool and useful Twin starts to know you better over time, resurfaces old context when it matters, and makes agents more efficient with every task. It’s live today. Try it out!
0:21

Long-running agents blow up context fast. So we built our own memory agent. Using it feels very cool and useful Twin starts to know you better over time, resurfaces old context when it matters, and makes agents more efficient with every task. It’s live today. Try it out!

Hugo Mercier

18,611 Aufrufe • vor 6 Monaten

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:
2:06

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:

Andrew Ng

131,850 Aufrufe • vor 1 Jahr

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!
2:57

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,950 Aufrufe • vor 1 Jahr

Most robot policies today still largely lack memory: they make all their decisions based on what they can see right now. MemER aims to change that by learning which frames are important; this lets it deal with tasks like object search. Ajay Sridhar, Jenny Pan, and @satviks107Sharma tell us about how to achieve this fundamental capability for long-horizon task execution. Watch Episode #54 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!
50:41

Most robot policies today still largely lack memory: they make all their decisions based on what they can see right now. MemER aims to change that by learning which frames are important; this lets it deal with tasks like object search. Ajay Sridhar, Jenny Pan, and @satviks107Sharma tell us about how to achieve this fundamental capability for long-horizon task execution. Watch Episode #54 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

RoboPapers

17,850 Aufrufe • vor 7 Monaten

Memory for OpenClaw is now Native! Our first OpenClaw Memory Skill was a massive success: 30k+ downloads in a week and 500k+ organic impressions overnight for launch post. But we knew memory needed to be native. On March 21, OpenClaw merged PR #50848, allowing us to go beyond the skill layer and integrate directly into the agent’s context assembly flow. We try to make OpenClaw a truly 24/7 employee capable of complex workflows. The technical setup isn’t the hardest part but the real challenge is giving it a "brain" that remembers exact project details, past decisions, and team changes over time. The Native Memory Plugin is now live on NPM & ClawHub. Here is what it brings to your OpenClaw agents: 👉 Native Integration: Automatically manages a Three-Layer Memory architecture (Context Tree, Workspace Memory, Daily Memory). 👉 Git-like Stateful Memory: Organizes memory into a semantic hierarchy of human-readable, diffable Markdown files. You always get updated knowledge and can actually see and fix what your agent learns. 👉 Top Market Accuracy: Achieves an industry-leading 92.2% retrieval accuracy (LoCoMo & LongMemEval benchmarks), maintaining 90% accuracy even with cheap, lightweight models. 👉 Local-first & Portable: Local-by-default, fully portable for multi-agent teams. 👉 Super Easy Setup
1:46

Memory for OpenClaw is now Native! Our first OpenClaw Memory Skill was a massive success: 30k+ downloads in a week and 500k+ organic impressions overnight for launch post. But we knew memory needed to be native. On March 21, OpenClaw merged PR #50848, allowing us to go beyond the skill layer and integrate directly into the agent’s context assembly flow. We try to make OpenClaw a truly 24/7 employee capable of complex workflows. The technical setup isn’t the hardest part but the real challenge is giving it a "brain" that remembers exact project details, past decisions, and team changes over time. The Native Memory Plugin is now live on NPM & ClawHub. Here is what it brings to your OpenClaw agents: 👉 Native Integration: Automatically manages a Three-Layer Memory architecture (Context Tree, Workspace Memory, Daily Memory). 👉 Git-like Stateful Memory: Organizes memory into a semantic hierarchy of human-readable, diffable Markdown files. You always get updated knowledge and can actually see and fix what your agent learns. 👉 Top Market Accuracy: Achieves an industry-leading 92.2% retrieval accuracy (LoCoMo & LongMemEval benchmarks), maintaining 90% accuracy even with cheap, lightweight models. 👉 Local-first & Portable: Local-by-default, fully portable for multi-agent teams. 👉 Super Easy Setup

andy nguyen

169,136 Aufrufe • vor 4 Monaten

🧠 Can agent memory scale without losing reasoning? 🔥 We’re excited to share our latest work, SimpleMem, a principled memory framework for LLM agents built around semantic lossless compression. 📉 30× fewer inference tokens 📈 +26.4% avg F1 (vs Mem0) ⚡ 50.2% faster retrieval (vs Mem0) Instead of storing raw interaction history 🗂️ or relying on costly iterative reasoning loops 🔁, SimpleMem treats memory as a structured, evolving representation whose primary objective is 🎯 maximizing information density per token. 📄 Paper: 🔗 Code: 📦 Website： Nice work @JiaqiLiu835914, Yaofeng Su, Peng (Richard) Xia, Siwei Han, and great collab. w/ Cihang Xie, Zeyu Zheng, Mingyu Ding
0:22

🧠 Can agent memory scale without losing reasoning? 🔥 We’re excited to share our latest work, SimpleMem, a principled memory framework for LLM agents built around semantic lossless compression. 📉 30× fewer inference tokens 📈 +26.4% avg F1 (vs Mem0) ⚡ 50.2% faster retrieval (vs Mem0) Instead of storing raw interaction history 🗂️ or relying on costly iterative reasoning loops 🔁, SimpleMem treats memory as a structured, evolving representation whose primary objective is 🎯 maximizing information density per token. 📄 Paper: 🔗 Code: 📦 Website： Nice work @JiaqiLiu835914, Yaofeng Su, Peng (Richard) Xia, Siwei Han, and great collab. w/ Cihang Xie, Zeyu Zheng, Mingyu Ding

Huaxiu Yao

119,767 Aufrufe • vor 7 Monaten

We analyzed Anthropic’s memory architecture and built something better: a persistent, human-inspectable, and token-efficient memory layer that scales with your projects. Today, it’s OPEN-SOURCE. ByteRover CLI gives agents (like OpenClaw, Claude Code, and Hermes) persistent, structured memory. Built on the exact architecture that became the #1 memory system for OpenClaw (30,000+ downloads in a week), it lets developers curate project knowledge into a file-based hierarchy. This guarantees highly accurate, lightning-fast retrieval, even with lightweight models. 👉Highly Accurate: >92% retrieval accuracy across long-running sessions - the highest proven production accuracy on the market. ⚡Fast: ~1.6s average retrieval time. 💰Economic: Maintains >90% accuracy even with lightweight models, saving 50-70% on token costs. ☁ Portable: Runs locally by default, with cloud-sync to share memory across agents and teammates.
1:19

We analyzed Anthropic’s memory architecture and built something better: a persistent, human-inspectable, and token-efficient memory layer that scales with your projects. Today, it’s OPEN-SOURCE. ByteRover CLI gives agents (like OpenClaw, Claude Code, and Hermes) persistent, structured memory. Built on the exact architecture that became the #1 memory system for OpenClaw (30,000+ downloads in a week), it lets developers curate project knowledge into a file-based hierarchy. This guarantees highly accurate, lightning-fast retrieval, even with lightweight models. 👉Highly Accurate: >92% retrieval accuracy across long-running sessions - the highest proven production accuracy on the market. ⚡Fast: ~1.6s average retrieval time. 💰Economic: Maintains >90% accuracy even with lightweight models, saving 50-70% on token costs. ☁ Portable: Runs locally by default, with cloud-sync to share memory across agents and teammates.

andy nguyen

28,980 Aufrufe • vor 4 Monaten

Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add ` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve ` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project ( but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): - Blog: - Beta implementation in chat client soon: Work done with amazing project co-leads Amelia Kuang Coco Xu Erik Chi !!
0:29

Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add ` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve ` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project ( but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): - Blog: - Beta implementation in chat client soon: Work done with amazing project co-leads Amelia Kuang Coco Xu Erik Chi !!

Ken Liu

73,947 Aufrufe • vor 3 Monaten

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 Aufrufe • vor 1 Monat

ChatGPT is getting better at remembering what matters: your preferences, constraints, and the context that helps you pick things up where you left off. And with memory summaries, you can review and steer what it remembers. Rolling to all users over the next few weeks, starting today with Plus and Pro users in the US.
0:33

ChatGPT is getting better at remembering what matters: your preferences, constraints, and the context that helps you pick things up where you left off. And with memory summaries, you can review and steer what it remembers. Rolling to all users over the next few weeks, starting today with Plus and Pro users in the US.

ChatGPT

141,323 Aufrufe • vor 2 Monaten

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.
0:29

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

Lior Alexander

808,102 Aufrufe • vor 3 Jahren

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”
2:27

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

38,220 Aufrufe • vor 1 Monat

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.
0:44

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.

Jiaxuan You

31,451 Aufrufe • vor 6 Monaten

Dave Asprey claims you can boost your IQ by around 12 points in a month with one specific brain training method. On The Iced Coffee Hour, he described dual n-back training: a frustrating grid game where you track both the position and the symbol from several moves back. Just 20 minutes a day. Most people quit, but sticking with it for ~20 sessions can dramatically improve working memory (e.g., going from remembering 7–8 items to 16). Dual n-back training reliably improves performance on the trained task and can enhance working memory capacity. Some studies show modest transfer to fluid intelligence (problem-solving aspect of IQ), but results are mixed, meta-analyses find small average gains (often 2–5 points) that are stronger in lower-quality studies and don’t always hold up in well-controlled trials. The idea that you can deliberately train a core cognitive skill like working memory is intriguing, even if the IQ boost isn’t guaranteed to be that large. In a world full of distractions that shrink our attention, a simple daily practice that strengthens your brain’s ability to hold and manipulate information could be genuinely useful. Would you commit to 20 minutes of frustrating brain training every day for a month if it could meaningfully improve your focus and memory?
1:52

Dave Asprey claims you can boost your IQ by around 12 points in a month with one specific brain training method. On The Iced Coffee Hour, he described dual n-back training: a frustrating grid game where you track both the position and the symbol from several moves back. Just 20 minutes a day. Most people quit, but sticking with it for ~20 sessions can dramatically improve working memory (e.g., going from remembering 7–8 items to 16). Dual n-back training reliably improves performance on the trained task and can enhance working memory capacity. Some studies show modest transfer to fluid intelligence (problem-solving aspect of IQ), but results are mixed, meta-analyses find small average gains (often 2–5 points) that are stronger in lower-quality studies and don’t always hold up in well-controlled trials. The idea that you can deliberately train a core cognitive skill like working memory is intriguing, even if the IQ boost isn’t guaranteed to be that large. In a world full of distractions that shrink our attention, a simple daily practice that strengthens your brain’s ability to hold and manipulate information could be genuinely useful. Would you commit to 20 minutes of frustrating brain training every day for a month if it could meaningfully improve your focus and memory?

Camus

131,291 Aufrufe • vor 2 Monaten

researchers gave a tiny local model human-style memory and its context limit basically stopped existing a team from MBZUAI, Princeton and Weizmann took a 1B model and rebuilt how it reads. instead of attending to everything at once, the model reads in 1,024 token chunks and passes the important stuff forward through an associative memory, the same way you carry the plot of a book between chapters without rereading them. the design mirrors human memory on purpose. full attention inside a chunk works as short-term memory. the module that carries information between chunks works as long-term memory. they even trained it like a person, starting with short easy texts and raising the difficulty gradually, because memory thrown into the deep end learns nothing. the numbers back it up. the normal model burns 40GB of GPU memory on a long document and collapses hard past its limit, dropping from 0.86 to 0.32 accuracy. the memory version holds 0.71 at double that length while using a flat 12GB no matter how long the input gets. it also needs about 30% fewer FLOPs. the part i keep thinking about is that nobody scaled anything here. they didn't build a bigger model, didn't stretch the window, didn't add compute. they looked at how a brain handles a long day and copied the architecture. a model small enough to run on a consumer gpu now survives documents its own architecture used to choke on. we keep treating intelligence as a compute problem. sometimes it's a memory problem.
0:43

researchers gave a tiny local model human-style memory and its context limit basically stopped existing a team from MBZUAI, Princeton and Weizmann took a 1B model and rebuilt how it reads. instead of attending to everything at once, the model reads in 1,024 token chunks and passes the important stuff forward through an associative memory, the same way you carry the plot of a book between chapters without rereading them. the design mirrors human memory on purpose. full attention inside a chunk works as short-term memory. the module that carries information between chunks works as long-term memory. they even trained it like a person, starting with short easy texts and raising the difficulty gradually, because memory thrown into the deep end learns nothing. the numbers back it up. the normal model burns 40GB of GPU memory on a long document and collapses hard past its limit, dropping from 0.86 to 0.32 accuracy. the memory version holds 0.71 at double that length while using a flat 12GB no matter how long the input gets. it also needs about 30% fewer FLOPs. the part i keep thinking about is that nobody scaled anything here. they didn't build a bigger model, didn't stretch the window, didn't add compute. they looked at how a brain handles a long day and copied the architecture. a model small enough to run on a consumer gpu now survives documents its own architecture used to choke on. we keep treating intelligence as a compute problem. sometimes it's a memory problem.

Alex Veremeyenko

16,147 Aufrufe • vor 16 Tagen