Loading video...

Video Failed to Load

Go Home

Context memory essentially unlocks Agentic AI Much needed for Opus 4.6's "multi-agent swarms" In this SemiDoped pod, Vikram Sekar talks to Val Bercovici from Weka about context storage. - How token warehouses save inference costs - A new networking tier? Context Storage Network! - High Bandwidth Flash for context?...

semidoped's profile picture

Semi Doped

3,349 subscribers

12,796 views • 4 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Breaking the Memory Wall: Micron’s Strategy for the AI Era Ben Bajarin and Jay Goldberg 顾忠南 talk micron's strategy for the AI era with Jeremy Werner SVP and GM of Micron’s Core Data Center Business Unit. Chapters: (00:00) Introduction and Guest Welcome (01:12) How the AI Cycle Differs from Past Volatility (04:23) Anticipating the Explosive Growth of AI (06:40) The Long-Term Sustainability of the AI Market (10:07) The "Memory Wall" in AI Inference (12:05) Explaining KV Cache and Context Windows (15:07) The Memory Hierarchy: From HBM to SSDs (20:21) Scaling for Billions of Simultaneous Users (23:05) Solving the Power and Efficiency Bottleneck (26:27) Why AI is Driving a Storage Shortage (29:57) Personal AI Agents and Persistent Memory Needs (34:13) Innovations in Ultra-High-Capacity SSDs (39:00) Accelerating the Pace of Engineering Innovation (43:50) Market Misunderstandings and the Future Potential of AI (47:38) The Global Race to Build New Fabs (51:34) Closing Remarks
52:31

Breaking the Memory Wall: Micron’s Strategy for the AI Era Ben Bajarin and Jay Goldberg 顾忠南 talk micron's strategy for the AI era with Jeremy Werner SVP and GM of Micron’s Core Data Center Business Unit. Chapters: (00:00) Introduction and Guest Welcome (01:12) How the AI Cycle Differs from Past Volatility (04:23) Anticipating the Explosive Growth of AI (06:40) The Long-Term Sustainability of the AI Market (10:07) The "Memory Wall" in AI Inference (12:05) Explaining KV Cache and Context Windows (15:07) The Memory Hierarchy: From HBM to SSDs (20:21) Scaling for Billions of Simultaneous Users (23:05) Solving the Power and Efficiency Bottleneck (26:27) Why AI is Driving a Storage Shortage (29:57) Personal AI Agents and Persistent Memory Needs (34:13) Innovations in Ultra-High-Capacity SSDs (39:00) Accelerating the Pace of Engineering Innovation (43:50) Market Misunderstandings and the Future Potential of AI (47:38) The Global Race to Build New Fabs (51:34) Closing Remarks

The Circuit

93,925 views • 1 month ago

Introducing HydraDB. The graph native context infrastructure for agents. Purpose built to deliver precise context & observability into why agents act the way they do. We've always believed graphs are the best way to manage AI context, but they've been too expensive to scale or impractical for storing full context. Until now. HydraDB combines in memory, NVMe, and object storage into a single graph layer, making context delivery faster, cheaper, and more precise. We want context delivery to be extremely fast, 1000x cheap, and highly precise. Give your agents a brain.
1:18

Introducing HydraDB. The graph native context infrastructure for agents. Purpose built to deliver precise context & observability into why agents act the way they do. We've always believed graphs are the best way to manage AI context, but they've been too expensive to scale or impractical for storing full context. Until now. HydraDB combines in memory, NVMe, and object storage into a single graph layer, making context delivery faster, cheaper, and more precise. We want context delivery to be extremely fast, 1000x cheap, and highly precise. Give your agents a brain.

Nishkarsh

2,282,400 views • 15 days ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!
2:57

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,729 views • 1 year ago

What’s next for AI storage? At NVIDIA GTC 2025, Seagate is revealing a proof of concept that could redefine mass-capacity AI storage—integrating NVMe hard drives into AI data pipelines with composable NVMe-oF architecture, DPUs, and SSD caching. 💡 The result? Lower latency, scalable AI infrastructure, and cost-effective storage for petabyte-to-exabyte workloads. Check out the infographic for a visual breakdown of the AI storage pipeline of the future. 🔗 Read more: #GTC24 #AI #NVMe
0:25

What’s next for AI storage? At NVIDIA GTC 2025, Seagate is revealing a proof of concept that could redefine mass-capacity AI storage—integrating NVMe hard drives into AI data pipelines with composable NVMe-oF architecture, DPUs, and SSD caching. 💡 The result? Lower latency, scalable AI infrastructure, and cost-effective storage for petabyte-to-exabyte workloads. Check out the infographic for a visual breakdown of the AI storage pipeline of the future. 🔗 Read more: #GTC24 #AI #NVMe

Seagate

157,168 views • 1 year ago

⚒️ How to build an AI agent with persistent memory? Watch Shaw (spirit/acc) building an agent from scratch with elizaOS & showing how external, queryable memory based on OriginTrail gives agents context and extends their memory beyond a single session. 02:15 – Set up your first agent 08:55 – Initializing runtime & memory foundations 12:50 – Wiring message handling & agent memory 24:45 – Persistent conversational memory in action 38:00 – Extending agents with plugins
1:05:27

⚒️ How to build an AI agent with persistent memory? Watch Shaw (spirit/acc) building an agent from scratch with elizaOS & showing how external, queryable memory based on OriginTrail gives agents context and extends their memory beyond a single session. 02:15 – Set up your first agent 08:55 – Initializing runtime & memory foundations 12:50 – Wiring message handling & agent memory 24:45 – Persistent conversational memory in action 38:00 – Extending agents with plugins

OriginTrail

37,178 views • 4 months ago

Stop prompt-stuffing yesterday’s context into your agent. Give it verifiable memory. 🕸️ When memory has provenance, agents can reuse it, audit it, and carry it across tools. Brana Rakic, Shaw (spirit/acc) (elizaOS), Ortecha | data made human & JoeVezz (LunarCrush 🌕™) break it down. 00:57 – What agents need to scale 05:45 – Social data, signals & real-time context 13:42 – Knowledge graphs as shared memory for AI agents 29:13 – Interoperability, trust & coordination
37:35

Stop prompt-stuffing yesterday’s context into your agent. Give it verifiable memory. 🕸️ When memory has provenance, agents can reuse it, audit it, and carry it across tools. Brana Rakic, Shaw (spirit/acc) (elizaOS), Ortecha | data made human & JoeVezz (LunarCrush 🌕™) break it down. 00:57 – What agents need to scale 05:45 – Social data, signals & real-time context 13:42 – Knowledge graphs as shared memory for AI agents 29:13 – Interoperability, trust & coordination

OriginTrail

13,293 views • 3 months ago

Perplexity Introduces Personalized Memory 🔥 TLDR : - Gives AI assistants memory to recall preferences, interests, and past chats. - It removes manual context engineering by auto loading relevant context. - Answers become more accurate and personalized using direct memory retrieval. - Users stay fully in control with on or off switches, incognito protection, and encryption. - Memory works across all models so your context follows you anywhere in Perplexity. - Comet Assistant also gets true memory, making it more consistent, personalized, and powerful.
0:30

Perplexity Introduces Personalized Memory 🔥 TLDR : - Gives AI assistants memory to recall preferences, interests, and past chats. - It removes manual context engineering by auto loading relevant context. - Answers become more accurate and personalized using direct memory retrieval. - Users stay fully in control with on or off switches, incognito protection, and encryption. - Memory works across all models so your context follows you anywhere in Perplexity. - Comet Assistant also gets true memory, making it more consistent, personalized, and powerful.

AshutoshShrivastava

22,006 views • 6 months ago

holy, competition is heating up a lot Anthropic introduces a memory feature that lets users transfer their context and preferences from other AI tools into Claude by copying a generated prompt and pasting the result into Claude’s memory settings. This allows Claude to immediately continue conversations with retained context, available for all paid plans.
0:59

holy, competition is heating up a lot Anthropic introduces a memory feature that lets users transfer their context and preferences from other AI tools into Claude by copying a generated prompt and pasting the result into Claude’s memory settings. This allows Claude to immediately continue conversations with retained context, available for all paid plans.

Chubby♨️

364,150 views • 3 months ago

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.
0:38

📈 NVIDIA tops AI leaderboards and benchmarks with open models driven by extreme co-design across compute, networking, memory, storage, and software. This includes models for biology, AI physics, agentic AI, physical AI, robotics, and autonomous vehicles. By being vertically integrated and horizontally open, NVIDIA enables its ecosystem to scale agentic workflows at every layer of the stack with the lowest cost per token.

NVIDIA

85,035 views • 1 month ago

Decentralised Storage, Served Fresh ft pranav | Shelby from Shelby In this episode of the Touch Grass Podcast, Karthikeya and I sat down with Pranav to chat about - > hot and cold storage > top usecases for decentralised storage > the 6 pillars that make shelby work and much more! 0:44 - Pranav's experience at IBW over the years 2:14 - the story behind Shelby 5:00 - why 2026 is the year for decentralised storage 7:50 - the competitor landscape 9:54 - the 6 pillars that make shelby work (and where Aptos fits in) 12:11- top usecases for shelby 15:42 - the rise of new AI marketplaces 16:24 - is India a prime market for shelby? 17:51 - shelby's long term vision 18:48 - hot vs cold data 20:53 - RFP: shared memory layer for all AI agents 25:08 - lessons from facebook → aptos → shelby (career advice) 27:16 - Pranav's #1 message to the shelby community
28:13

Decentralised Storage, Served Fresh ft pranav | Shelby from Shelby In this episode of the Touch Grass Podcast, Karthikeya and I sat down with Pranav to chat about - > hot and cold storage > top usecases for decentralised storage > the 6 pillars that make shelby work and much more! 0:44 - Pranav's experience at IBW over the years 2:14 - the story behind Shelby 5:00 - why 2026 is the year for decentralised storage 7:50 - the competitor landscape 9:54 - the 6 pillars that make shelby work (and where Aptos fits in) 12:11- top usecases for shelby 15:42 - the rise of new AI marketplaces 16:24 - is India a prime market for shelby? 17:51 - shelby's long term vision 18:48 - hot vs cold data 20:53 - RFP: shared memory layer for all AI agents 25:08 - lessons from facebook → aptos → shelby (career advice) 27:16 - Pranav's #1 message to the shelby community

Rachit Agarwal

11,979 views • 6 months ago

Where are all the consumer AI apps??? This is the question I posed to s-tier consumer investors scott belsky (Uber, Pinterest) and steve jang (Coinbase, Postmates). We're almost 3 years into the AI hype the we're still all mostly just talking about ChatGPT, Perplexity, and Claude. We explore the answer through the ways in which the entire consumer stack (input, memory, personalization, browsers, hardware) are being re-written by AI and how that may give birth to the consumer opportunity we've all been waiting for. Chapters: 00:00 Welcome and Introductions 00:43 The Current State of AI Products 02:48 Surprises and Challenges with LLMs 05:24 Future of Consumer AI and Personal AI 15:49 Memory and Personalization in AI 21:52 Potential Risks and Business Models 24:22 The Battle for User Data in AI 25:11 Optimism in AI Model Inference 27:00 The Role of Open Source Models 28:13 The Future of AI Hardware 28:54 Challenges and Opportunities in AI Hardware Startups 30:29 The Evolution of Consumer Hardware 38:36 The Reinvention of Browsers 46:23 The Potential of AI-Enhanced Browsers 47:54 Conclusion and Final Thoughts Many thanks to my good friends for joining me in this discussion 🙏
45:47

Where are all the consumer AI apps??? This is the question I posed to s-tier consumer investors scott belsky (Uber, Pinterest) and steve jang (Coinbase, Postmates). We're almost 3 years into the AI hype the we're still all mostly just talking about ChatGPT, Perplexity, and Claude. We explore the answer through the ways in which the entire consumer stack (input, memory, personalization, browsers, hardware) are being re-written by AI and how that may give birth to the consumer opportunity we've all been waiting for. Chapters: 00:00 Welcome and Introductions 00:43 The Current State of AI Products 02:48 Surprises and Challenges with LLMs 05:24 Future of Consumer AI and Personal AI 15:49 Memory and Personalization in AI 21:52 Potential Risks and Business Models 24:22 The Battle for User Data in AI 25:11 Optimism in AI Model Inference 27:00 The Role of Open Source Models 28:13 The Future of AI Hardware 28:54 Challenges and Opportunities in AI Hardware Startups 30:29 The Evolution of Consumer Hardware 38:36 The Reinvention of Browsers 46:23 The Potential of AI-Enhanced Browsers 47:54 Conclusion and Final Thoughts Many thanks to my good friends for joining me in this discussion 🙏

Michael Mignano

25,155 views • 11 months ago

Satya Nadella says the 3 capabilities needed for AI agents are now in place and improving exponentially: a universal interface of multimodality; the neural algebra of reasoning and planning; and long-term memory, rich context and tool use
1:09

Satya Nadella says the 3 capabilities needed for AI agents are now in place and improving exponentially: a universal interface of multimodality; the neural algebra of reasoning and planning; and long-term memory, rich context and tool use

Tsarathustra

69,091 views • 1 year ago

MemOS now has a plugin for OpenClaw that enables your AI agents to work on the common memory layer and cut down token usage. - Multiple agents read/write the same memory — no manual context handoff - 72% lower token costs (15.6M → 4.4M on LOCOMO dataset) - Cross-session and long-term memory that actually works Sponsored
1:31

MemOS now has a plugin for OpenClaw that enables your AI agents to work on the common memory layer and cut down token usage. - Multiple agents read/write the same memory — no manual context handoff - 72% lower token costs (15.6M → 4.4M on LOCOMO dataset) - Cross-session and long-term memory that actually works Sponsored

🚨 AI News | TestingCatalog

41,003 views • 4 months ago

Andrew meets with John, where they discuss Supermicro and VΛST Data's collaboration on exciting storage projects such as the CNode-X AI Data Platform. Learn more about Supermicro and VAST Data's AI and Storage solutions: #Supermicro #VASTData #NVIDIA
2:01

Andrew meets with John, where they discuss Supermicro and VΛST Data's collaboration on exciting storage projects such as the CNode-X AI Data Platform. Learn more about Supermicro and VAST Data's AI and Storage solutions: #Supermicro #VASTData #NVIDIA

Supermicro

65,960 views • 2 months ago

.Juan Benet from Protocol Labs shares that Filecoin is now the world’s largest decentralized storage network. It reduces storage costs while enabling new capabilities like smart contracts and AI agent integration.
1:02

.Juan Benet from Protocol Labs shares that Filecoin is now the world’s largest decentralized storage network. It reduces storage costs while enabling new capabilities like smart contracts and AI agent integration.

Filecoin

13,377 views • 1 year ago

Latest episode of the AGI Podcast is out, featuring Wei Zhou, Head of AIUR at SemiAnalysis. Token Wei and I go deep on compute, capital markets, tokenomics, and and the future of AGI. Inside the conversation: 01:00 — Wei’s background and early journey 3:38 — The most cracked researchers in China 5:30 — My path after finishing a Math PhD at Stanford 7:52 — A based, unhinged take on Andrew Ng 10:29 — Choosing money vs “contributing to society” 19:32 — What Wei learned working with George Soros 24:00 — Intel vs Nvidia and the road to becoming the next compute central bank 27:15 — CUDA’s early days and how it became Tencent 2.0 for AI 35:22 — The true base unit of token cost: gigawatts vs dollar capex/opex 39:18 — “How do you price everything?” — a compute-first worldview 42:22 — How investment flows actually move in public markets 43:54 — Understanding depreciation distribution 52:00 — Is memory/storage scarcity real, and what drives it? 54:41 — Should storage cost be part of tokenomics models? 57:44 — Why inter-chip interconnect defines the economics of AI tokens 1:05:56 — Wei’s surprising love for Pop Mart 1:10:52 — How insiders really define AGI and ASI Watch the full podcast here 👇
1:11:41

Latest episode of the AGI Podcast is out, featuring Wei Zhou, Head of AIUR at SemiAnalysis. Token Wei and I go deep on compute, capital markets, tokenomics, and and the future of AGI. Inside the conversation: 01:00 — Wei’s background and early journey 3:38 — The most cracked researchers in China 5:30 — My path after finishing a Math PhD at Stanford 7:52 — A based, unhinged take on Andrew Ng 10:29 — Choosing money vs “contributing to society” 19:32 — What Wei learned working with George Soros 24:00 — Intel vs Nvidia and the road to becoming the next compute central bank 27:15 — CUDA’s early days and how it became Tencent 2.0 for AI 35:22 — The true base unit of token cost: gigawatts vs dollar capex/opex 39:18 — “How do you price everything?” — a compute-first worldview 42:22 — How investment flows actually move in public markets 43:54 — Understanding depreciation distribution 52:00 — Is memory/storage scarcity real, and what drives it? 54:41 — Should storage cost be part of tokenomics models? 57:44 — Why inter-chip interconnect defines the economics of AI tokens 1:05:56 — Wei’s surprising love for Pop Mart 1:10:52 — How insiders really define AGI and ASI Watch the full podcast here 👇

Bill Sun

33,056 views • 6 months ago

🧬A trust layer for genomic AI powered by OriginTrail, has been implemented under the ELSA - European Lighthouse on Secure and Safe AI funded by the EU. Trace Labs - Trusted Context for AI Agents's Jurij Skornik demonstrated it in action ↯ 00:00 - Introduction 00:30 - Key challenges in decentralized science 00:46 - Decentralized Knowledge Graph power 01:35 - How an AI agent can assess Alzheimer’s risk 02:32 - Working with private patient data 03:50 - Solution in action: Personal healthcare AI agent
9:59

🧬A trust layer for genomic AI powered by OriginTrail, has been implemented under the ELSA - European Lighthouse on Secure and Safe AI funded by the EU. Trace Labs - Trusted Context for AI Agents's Jurij Skornik demonstrated it in action ↯ 00:00 - Introduction 00:30 - Key challenges in decentralized science 00:46 - Decentralized Knowledge Graph power 01:35 - How an AI agent can assess Alzheimer’s risk 02:32 - Working with private patient data 03:50 - Solution in action: Personal healthcare AI agent

OriginTrail

37,752 views • 6 months ago

HERE’S FRIDAY’S FOX BUSINESS SEGMENT • $MU becoming the picks-and-shovels for “AI that remembers • $SNDK catching the second-order storage wave • $RKLB building an end-to-end space logistics • $PL monetizing continuous Earth data from orbit • $ASTS turning space into a global cellular network The 2026 memory story is that AI demand is shifting from stop-start training runs to always-on inference making memory less cyclical and more foundational infrastructure.
5:25

HERE’S FRIDAY’S FOX BUSINESS SEGMENT • $MU becoming the picks-and-shovels for “AI that remembers • $SNDK catching the second-order storage wave • $RKLB building an end-to-end space logistics • $PL monetizing continuous Earth data from orbit • $ASTS turning space into a global cellular network The 2026 memory story is that AI demand is shifting from stop-start training runs to always-on inference making memory less cyclical and more foundational infrastructure.

Shay Boloor

400,708 views • 4 months ago

"What happens when we can access each other's AI memory in enterprises?" scott belsky, investor/founder of Behance: "If I was working as an intern for years, and the context window is accessible to the enterprise, can the enterprise (keep prompting) and asking me what I'd think or do, after I leave?" "Every company right now is trying to build this AI memory for each of its products and users. A lot of them are building connectors to form a collective memory." "In the social/consumer world, imagine your girlfriend saying she wants access to your context window or memory." "There are all kinds of weird questions and implications that arise from this."
1:10

"What happens when we can access each other's AI memory in enterprises?" scott belsky, investor/founder of Behance: "If I was working as an intern for years, and the context window is accessible to the enterprise, can the enterprise (keep prompting) and asking me what I'd think or do, after I leave?" "Every company right now is trying to build this AI memory for each of its products and users. A lot of them are building connectors to form a collective memory." "In the social/consumer world, imagine your girlfriend saying she wants access to your context window or memory." "There are all kinds of weird questions and implications that arise from this."

TBPN

44,908 views • 1 year ago