Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Announcing... Voice x Memory! We’re unpacking what makes agents listen, respond, and remember, or sometimes forget, and what that means for building better voice systems. We will move towards a world of large LLMs remembering a lot of information to smaller LMs with finite real-time intelligence and infinite memory.... show more

Sudarshan Kamath

22,231 subscribers

23,599 views • 6 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🚨 KEEP CALM AND AI - AUTONOMOUS AGENTS WITH INFINITE MEMORY We are launching the ability to create arbitrary agents that run on schedule and have access to a persistent and infinite memory The agents will be able to store, retrieve, and update information across sessions and perform repetitive tasks that require persistent memory Another step towards AGI and automating white-collar work 🚀🚀🚀

🚨 KEEP CALM AND AI - AUTONOMOUS AGENTS WITH INFINITE MEMORY We are launching the ability to create arbitrary agents that run on schedule and have access to a persistent and infinite memory The agents will be able to store, retrieve, and update information across sessions and perform repetitive tasks that require persistent memory Another step towards AGI and automating white-collar work 🚀🚀🚀

Bindu Reddy

12,626 views • 6 months ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,950 views • 1 year ago

We walk into Church, we walk up to Holy Communion, and we walk out of Church. We sometimes call it “processing,” which means a holy walk. Walking is the sign of a journey, and we’re on a journey. We crystallize that at every Mass. The Good Newsroom

We walk into Church, we walk up to Holy Communion, and we walk out of Church. We sometimes call it “processing,” which means a holy walk. Walking is the sign of a journey, and we’re on a journey. We crystallize that at every Mass. The Good Newsroom

Cardinal Dolan

12,364 views • 1 year ago

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Stripe

18,535 views • 3 months ago

Introducing our unified Voice Agent API! ⚡️ Quickly and easily build voice agents that listen, think, and respond in real-time, as naturally as a human can. Powered by the industry's fastest, most performant speech recognition and voice synthesis models to deliver: -Natural-sounding real-time conversations -Revolutionary interruption handling -Choice of open source, closed source, or BYO LLM -Low cost See our drive-thru agent in action, smoothly handling interruptions and complex orders in the noisy streets of SF. Learn how to get access (🧵)

Introducing our unified Voice Agent API! ⚡️ Quickly and easily build voice agents that listen, think, and respond in real-time, as naturally as a human can. Powered by the industry's fastest, most performant speech recognition and voice synthesis models to deliver: -Natural-sounding real-time conversations -Revolutionary interruption handling -Choice of open source, closed source, or BYO LLM -Low cost See our drive-thru agent in action, smoothly handling interruptions and complex orders in the noisy streets of SF. Learn how to get access (🧵)

Deepgram

726,214 views • 1 year ago

80 years since the liberation of Auschwitz is a long stretch of time. A lot has changed. But what won’t change is our memory – how that war started, who was guilty, and who did the killing. And we will never forgive. We also remember that thanks to unity and determination, life won in the end. This is how it should always be. Thank you to everyone who keeps and shares the truth about that horrific time. Eternal memory to all the victims of the Holocaust! And thank you everyone who defends life today. It’s time for Europe to become a place without war. We dream about it. And we’re working for it.

80 years since the liberation of Auschwitz is a long stretch of time. A lot has changed. But what won’t change is our memory – how that war started, who was guilty, and who did the killing. And we will never forgive. We also remember that thanks to unity and determination, life won in the end. This is how it should always be. Thank you to everyone who keeps and shares the truth about that horrific time. Eternal memory to all the victims of the Holocaust! And thank you everyone who defends life today. It’s time for Europe to become a place without war. We dream about it. And we’re working for it.

Volodymyr Zelenskyy / Володимир Зеленський

302,228 views • 1 year ago

Memory isn’t just a feature in AI, it’s the difference between a system that responds and one that truly understands. In the first episode of The Long Walk, Sudarshan Kamath and I dive into how memory is shaping the next generation of voice agents, and why it’s becoming critical for real enterprise workflows.

Memory isn’t just a feature in AI, it’s the difference between a system that responds and one that truly understands. In the first episode of The Long Walk, Sudarshan Kamath and I dive into how memory is shaping the next generation of voice agents, and why it’s becoming critical for real enterprise workflows.

Nishkarsh

30,285 views • 5 months ago

That's a wrap on Alif Summit. We set out ~2 months ago with one goal: to bring the brightest Muslim builders from across the world under one roof. And we did it to prove a point — that as a community, it's time to raise the bar of what we think we're capable of. To move beyond trying to build the best "Muslim" version of anything, and shoot directly for building the best. And this weekend, it felt like we made real progress towards that. Alhamdulilah. Ty to everyone who came by.

That's a wrap on Alif Summit. We set out ~2 months ago with one goal: to bring the brightest Muslim builders from across the world under one roof. And we did it to prove a point — that as a community, it's time to raise the bar of what we think we're capable of. To move beyond trying to build the best "Muslim" version of anything, and shoot directly for building the best. And this weekend, it felt like we made real progress towards that. Alhamdulilah. Ty to everyone who came by.

Omar Waseem

14,437 views • 1 year ago

Will the countries of the Security Council choose to stand with a regime that slaughters its own people, that brutalizes its neighbors, that strangles the world’s economy? Or will they standing with the shining cities of Manama, Kuwait City, Dubai, Riyadh, and Doha—with the people that seek a better future, for not only their people, but for the region, and for the world? I know what decision the United States is making. And we’re choosing to stand for freedom of navigation. We’re choosing to stand for international law. We’re choosing to stand with our partners as we move forward.

Will the countries of the Security Council choose to stand with a regime that slaughters its own people, that brutalizes its neighbors, that strangles the world’s economy? Or will they standing with the shining cities of Manama, Kuwait City, Dubai, Riyadh, and Doha—with the people that seek a better future, for not only their people, but for the region, and for the world? I know what decision the United States is making. And we’re choosing to stand for freedom of navigation. We’re choosing to stand for international law. We’re choosing to stand with our partners as we move forward.

Ambassador Mike Waltz

61,518 views • 3 months ago

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

Andrew Ng

160,148 views • 4 months ago

Here’s a demo of our Agentic Memory system, inspired by how our own brain holds information in a 3D spatial space. This feels, natural. Extending this further, we announced the Agentic Memory Protocol on July 15th in SF - which enables memory to be local, encrypted and available to other agents and apps based on your permission-only. We believe this is the future of memory - not owned by any one app, spatial, and always improving. Aaron Levie Andrej Karpathy

Here’s a demo of our Agentic Memory system, inspired by how our own brain holds information in a 3D spatial space. This feels, natural. Extending this further, we announced the Agentic Memory Protocol on July 15th in SF - which enables memory to be local, encrypted and available to other agents and apps based on your permission-only. We believe this is the future of memory - not owned by any one app, spatial, and always improving. Aaron Levie Andrej Karpathy

Varun

187,489 views • 1 year ago

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:

Andrew Ng

131,850 views • 1 year ago

As sons, the loss of a father leaves a space that no triumph can ever truly fill. What we can do, however, is carry ourselves in a way that honours their memory, in how we stand, how we lead, and how we endure. Proud of my nephew. Spoke with conviction

As sons, the loss of a father leaves a space that no triumph can ever truly fill. What we can do, however, is carry ourselves in a way that honours their memory, in how we stand, how we lead, and how we endure. Proud of my nephew. Spoke with conviction

DON STRAPZY *SE DØNS*

23,981 views • 5 months ago

Pan: I think about this effort in sort of three pillars of innovation, collaboration, and communication. On communication, we have really valued the work of Katelyn Jetelina…people are getting information in different ways now. The old way of doing things was getting it out on the evening news… and it’s a very different and smaller demographic that does that now. So, really thinking about how do you get information out in a different way and then get that feedback on what people are hearing, what information do they need to know to make healthy choices. On collaboration, We’ve already started these alliances with the West Coast health alliance and building on other kind of partnerships we had but there’s a lot of different emerging efforts as we worry about withdrawal from the global health community and some of the communication and coordination that’s needed—Dr. Houry is going to help us on that—how do we make sure if federal resources and coordination to decrease—how we can continue to uphold that science and communication. Then the innovation is also really exciting around thinking about how do we better partner with the private sector with our academic partners and one key area we’re talking a lot about is data exchange and how do we better work with healthcare systems and public health to improve and accelerate making sure we also have access to tools to do more efficient exchange of information, there’s required reporting of certain diseases or for all of us to know what are the health trends in my community or in my state—where are the highest portion people with the most uncontrolled diabetes or where is this emerging outbreak happening—how do we do a better job using modern tools to have awareness and have communication with our healthcare system so that’s another area we’re really excited to work on with Dr. Monarez.

Pan: I think about this effort in sort of three pillars of innovation, collaboration, and communication. On communication, we have really valued the work of Katelyn Jetelina…people are getting information in different ways now. The old way of doing things was getting it out on the evening news… and it’s a very different and smaller demographic that does that now. So, really thinking about how do you get information out in a different way and then get that feedback on what people are hearing, what information do they need to know to make healthy choices. On collaboration, We’ve already started these alliances with the West Coast health alliance and building on other kind of partnerships we had but there’s a lot of different emerging efforts as we worry about withdrawal from the global health community and some of the communication and coordination that’s needed—Dr. Houry is going to help us on that—how do we make sure if federal resources and coordination to decrease—how we can continue to uphold that science and communication. Then the innovation is also really exciting around thinking about how do we better partner with the private sector with our academic partners and one key area we’re talking a lot about is data exchange and how do we better work with healthcare systems and public health to improve and accelerate making sure we also have access to tools to do more efficient exchange of information, there’s required reporting of certain diseases or for all of us to know what are the health trends in my community or in my state—where are the highest portion people with the most uncontrolled diabetes or where is this emerging outbreak happening—how do we do a better job using modern tools to have awareness and have communication with our healthcare system so that’s another area we’re really excited to work on with Dr. Monarez.

Acyn

107,372 views • 7 months ago

In the early days of Notion, we published a changelog weekly, sometimes multiple times a week. As we grew into a larger organization, with many people shipping many things, we moved to a monthly cadence. It helped consolidate everything, but it somewhat blurred what was changing week to week. For a long time, I’ve wanted to get back to a weekly changelog. I think it’s a more transparent reflection of what we’re building, fixing, and improving each week. Excited to share that we’re back to our startup roots, and now publish the changelog ~weekly. We also do a short video highlighting the most important ships. See the latest one below. Would love your feedback! How do you like the format? How can we communicate this better?

In the early days of Notion, we published a changelog weekly, sometimes multiple times a week. As we grew into a larger organization, with many people shipping many things, we moved to a monthly cadence. It helped consolidate everything, but it somewhat blurred what was changing week to week. For a long time, I’ve wanted to get back to a weekly changelog. I think it’s a more transparent reflection of what we’re building, fixing, and improving each week. Excited to share that we’re back to our startup roots, and now publish the changelog ~weekly. We also do a short video highlighting the most important ships. See the latest one below. Would love your feedback! How do you like the format? How can we communicate this better?

Akshay Kothari

41,450 views • 7 months ago

We went from being the worst, to the best AI memory system in the world. While being students with ~no prior research experience. .... but, how do we measure "good" vs bad memory? Evals (and vibes!) here's a small video on how we think about memory.

We went from being the worst, to the best AI memory system in the world. While being students with ~no prior research experience. .... but, how do we measure "good" vs bad memory? Evals (and vibes!) here's a small video on how we think about memory.

Dhravya Shah

120,929 views • 8 months ago

“I have a future!" “We will abandon the Russian way.” "It means hope". A generation of Hungarians, who’ve only ever known Orbán’s pro-Russian government, take to the streets to celebrate what they see as a reclaiming of democracy, and a future closer to Europe.

“I have a future!" “We will abandon the Russian way.” "It means hope". A generation of Hungarians, who’ve only ever known Orbán’s pro-Russian government, take to the streets to celebrate what they see as a reclaiming of democracy, and a future closer to Europe.

The News Agents

10,265 views • 4 months ago

#YOUNGJAE: starting around the middle of our career, we came to a shared idea, “let’s go with the image of the group that just knows how to have a great time on stage.” no matter what the song is, and since we had a lot of songs, instead of going with super sharp synchronized choreography, we decided, “let’s be the group that just feels free and comfortable on stage.” that's the direction we agreed on as a team. that's what we talked about among ourselves

#YOUNGJAE: starting around the middle of our career, we came to a shared idea, “let’s go with the image of the group that just knows how to have a great time on stage.” no matter what the song is, and since we had a lot of songs, instead of going with super sharp synchronized choreography, we decided, “let’s be the group that just feels free and comfortable on stage.” that's the direction we agreed on as a team. that's what we talked about among ourselves

𝐣𝐨𝐲𝐜𝐞

25,442 views • 1 year ago

AI will determine the future of WORK, BUSINESS, and LIFE. And there are huge questions: Will AI unleash our creativity and help us to flourish? Or will we lose even more power, agency, and ownership? It all depends on the design of the tech infrastructure. As Chris Dixon often says, “Architecture is destiny.” And how AI systems are designed will be key to the future we all live in. The team at gensyn is building a protocol that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. They are shaping a future of AI that is good for all of us. At a recent Innoveum event, I chatted to cofounder and CEO of gensyn about what they’re building, how it transforms AI, and why we should all be relieved, excited, and get involved. The full conversation is below in Episode One of Innoveum | The Podcast. 👇

AI will determine the future of WORK, BUSINESS, and LIFE. And there are huge questions: Will AI unleash our creativity and help us to flourish? Or will we lose even more power, agency, and ownership? It all depends on the design of the tech infrastructure. As Chris Dixon often says, “Architecture is destiny.” And how AI systems are designed will be key to the future we all live in. The team at gensyn is building a protocol that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. They are shaping a future of AI that is good for all of us. At a recent Innoveum event, I chatted to cofounder and CEO of gensyn about what they’re building, how it transforms AI, and why we should all be relieved, excited, and get involved. The full conversation is below in Episode One of Innoveum | The Podcast. 👇

Misha

27,481 views • 1 year ago