Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Meet the minds building supermemory! 🧠 Our research engineer Prasanna breaks down how memory actually works, in your head, and in AI. inside the episode: → why forgetting is a feature and not a failure → what your hippocampus inside brain is really doing all day → how "dreaming"...

13,926 görüntüleme • 26 gün önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Happy to properly launch Anna, the proactive AI agent for parents! Uncovering a bit of the technology behind the scenes! Building Anna is where I learned: 💾 Memory as plain text sucks. You need structured memory. Like a full-blown PostgreSQL DB that stores your tasks and calendar in a structured manner. Most harnesses are good at coding-related stuff. Let it do the query. Don't let it vibe-search the memory. Let it vibe your SQL query 💭 Dreaming is a useful concept for enhancing memory to feed the LLM context. But DO NOT vibe your dream. Asking your agent to "hey, just dream and keep the relevant memory around" is a recipe for deleting a bunch of important information and keeping trash around. Your dream needs to have some Taxonomy (or better, Ontology). What information is important? For who? With what object? What can they do? And again, these are impossible to describe and act well without a proper schema 🔄 Loop Engineering is important for smoothing out rough edges in the system we build. But even expensive loop engineering with a state-of-the-art model can't out-engineer bad system design. The highest leverage an AI Engineer can do is actually building the right system design, and having an eye on both product delight and engineering scalability There are several more insights that I plan to cover in a dedicated video about Agentic AI Engineering. But it's actually a huge relief that the future of software engineering... is still software engineering

Gogo | Dota for Toxicity

30,766 görüntüleme • 1 ay önce

researchers gave a tiny local model human-style memory and its context limit basically stopped existing a team from MBZUAI, Princeton and Weizmann took a 1B model and rebuilt how it reads. instead of attending to everything at once, the model reads in 1,024 token chunks and passes the important stuff forward through an associative memory, the same way you carry the plot of a book between chapters without rereading them. the design mirrors human memory on purpose. full attention inside a chunk works as short-term memory. the module that carries information between chunks works as long-term memory. they even trained it like a person, starting with short easy texts and raising the difficulty gradually, because memory thrown into the deep end learns nothing. the numbers back it up. the normal model burns 40GB of GPU memory on a long document and collapses hard past its limit, dropping from 0.86 to 0.32 accuracy. the memory version holds 0.71 at double that length while using a flat 12GB no matter how long the input gets. it also needs about 30% fewer FLOPs. the part i keep thinking about is that nobody scaled anything here. they didn't build a bigger model, didn't stretch the window, didn't add compute. they looked at how a brain handles a long day and copied the architecture. a model small enough to run on a consumer gpu now survives documents its own architecture used to choke on. we keep treating intelligence as a compute problem. sometimes it's a memory problem.

Alex Veremeyenko

16,147 görüntüleme • 26 gün önce

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,950 görüntüleme • 1 yıl önce