Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Ghostty is getting automatic scrollback compression, resulting in 70 to 90% less physical memory usage. It happens incrementally when idle, so it had no measurable effect on IO throughput. I'm not aware of any other mainstream terminal that does this. Demo video below! The gains let us increase the...

mitchellh's profile picture

Mitchell Hashimoto

223,294 subscribers

166,748 Aufrufe • vor 1 Monat •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.
1:09

DeepMind CEO Demis Hassabis says the AI bottleneck is already moving from model ideas to physical supply chains. "How much of the restriction at this point is physical? I am talking memory shortage, GPU shortage, and electricity." Demis: "That is constraining a lot of the deployment for sure. We see so much more demand for our models - Gemini and the other models that we have - than we can possibly serve." Then the less obvious part: "It does constrain a little bit the research, because you need a lot of chips to be able to experiment on new ideas at big enough scale that you can actually see if they are going to work." And on memory: "The whole supply chain is kind of strained... It still in the end actually comes down to a few suppliers of a few of the key components. Anywhere where there is some kind of constraint on the capacity, there is a sort of choke point." Important framing: The next AI race is not just who has the best model architecture. It is who can secure memory, chips, power, suppliers, and enough experimental capacity to find the next breakthrough before everyone else.

Karl Mehta

25,716 Aufrufe • vor 2 Monaten

Ben Thompson says the memory makers are Iran, and by making the Strait of Hormuz mistake are inviting the whole industry to build around them "The other thing, the risk they run, is Apple. Apple's lobbying to get Chinese memory, right?" "And what is the number one focus of algorithmic changes? How can we use less memory?" "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back." "I've analogized memory makers to Iran. The issue with the Strait of Hormuz is it's very effective. It's more effective if you don't use it, cuz then it's always hanging out there as something you could do. Now they did it. Turns out it worked." "But the UAE and Saudi Arabia, they're going to build pipelines. They're going to build new ports. They're not going to let this happen again." "It's very painful right now. But say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around." "My concern for the memory makers is they might have done the same thing. No one's going to let themselves get in this situation again as far as memory goes."
0:58

Ben Thompson says the memory makers are Iran, and by making the Strait of Hormuz mistake are inviting the whole industry to build around them "The other thing, the risk they run, is Apple. Apple's lobbying to get Chinese memory, right?" "And what is the number one focus of algorithmic changes? How can we use less memory?" "I think the memory makers probably screw themselves in the long run by creating such a massive target on their back." "I've analogized memory makers to Iran. The issue with the Strait of Hormuz is it's very effective. It's more effective if you don't use it, cuz then it's always hanging out there as something you could do. Now they did it. Turns out it worked." "But the UAE and Saudi Arabia, they're going to build pipelines. They're going to build new ports. They're not going to let this happen again." "It's very painful right now. But say Iran wants to close the Strait of Hormuz in 2035. It's not going to have any effect, because it will have been built around." "My concern for the memory makers is they might have done the same thing. No one's going to let themselves get in this situation again as far as memory goes."

Fireside Alpha

154,883 Aufrufe • vor 10 Tagen

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.
0:50

The creator of High Bandwidth Memory (HBM) put a number on the AI build that should stop every infra investor cold. A cluster of a million GPUs runs at roughly 10-20% utilization (Save this). Kim Jung-ho spent thirty years building what feeds the GPU, and his claim is that the GPU is barely working. Here is what is actually happening. Every time a model generates output, the data has to be read out of memory, computed, and written back. The read and the write swallow almost the entire cycle. While that data moves, the GPU does nothing. It sits there, fully powered, fully paid for, waiting. By Kim's estimate the memory is doing only about 30 percent of the work it needs to do. The processor idles the rest. So a million installed GPUs run at 10 to 20 percent. You are not compute constrained. You are memory constrained, and the expensive part is standing around. Adding more GPUs does not fix this. It gives you more processors starving for the same data. Here is the part that decides the next decade. Memory can grow. When a cell cannot shrink any further, you stack it into a high-rise, layer on layer. A GPU cannot be stacked. It runs too hot and needs a cooler bolted to its back, so the one move that rescues memory is closed to the processor. The thing that can keep stacking compounds. The thing that cannot plateaus. The marginal dollar in an AI build now buys more by fixing the memory path than by bolting on another idle GPU. Which is why the companies that control memory bandwidth and supply are not suppliers to the AI trade. They are the AI trade.

Fireside Alpha

38,370 Aufrufe • vor 2 Monaten

Virtual Textures in the Browser. This is a WebGL 2.0 implementation I did in 2023. The car model is from Sketchfab by Pavel Matoušek. It's a worst-case for virtual textures because the texture is generated from photogrametry software and has pathologically bad UV layout. The texture is 16,384 x 16,384 pixels, which is 180 Mb of PNG data. Loading this in Blender would crash Blender for me back then. The GPU memory requirement for this texture is 1 Gb. The VT displays the model after loading just 34 Kb (one tile). The other tiles are fetched in via a smart priority queue in the background as you move camera around the scene. The physical page texture is 2k, which is 16 times smaller than the source. The solution supports multiple separate virtual textures, and has GLTF integration. Texture filtering as well as anisotropy are fully supported as well. You can try the demo yourself here: Here's a writeup I did while working on this: The video recorded on a very crummy internet connection and I intentionally reload the page in the middle. The delay comes from the geometry download, actual texture part is near-instant.
1:28

Virtual Textures in the Browser. This is a WebGL 2.0 implementation I did in 2023. The car model is from Sketchfab by Pavel Matoušek. It's a worst-case for virtual textures because the texture is generated from photogrametry software and has pathologically bad UV layout. The texture is 16,384 x 16,384 pixels, which is 180 Mb of PNG data. Loading this in Blender would crash Blender for me back then. The GPU memory requirement for this texture is 1 Gb. The VT displays the model after loading just 34 Kb (one tile). The other tiles are fetched in via a smart priority queue in the background as you move camera around the scene. The physical page texture is 2k, which is 16 times smaller than the source. The solution supports multiple separate virtual textures, and has GLTF integration. Texture filtering as well as anisotropy are fully supported as well. You can try the demo yourself here: Here's a writeup I did while working on this: The video recorded on a very crummy internet connection and I intentionally reload the page in the middle. The delay comes from the geometry download, actual texture part is near-instant.

Alex Goldring

19,368 Aufrufe • vor 7 Monaten

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 Aufrufe • vor 1 Monat

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.
0:23

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

271,839 Aufrufe • vor 2 Monaten

The New 5-Star Memory Wish Pool [Xavier: Silvery Polyphony] is Coming Soon! "When you play it... I feel like the universe's echoes have finally found their answer." 💫Event Duration: From 5:00 AM on Dec. 18 to 4:59 AM on Dec. 28 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Xavier: Silvery Polyphony] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool, Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Concerto Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Facial: Glistening Tears], [Deepspace Wish: Limited*20], His General Photo Pose [Consideration], Memory Ascension materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Facial will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Concerto Pack] and [Silvery Cascade Gift] series will be available for purchase, including [Deepspace Wish: Limited], all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Xavier
0:35

The New 5-Star Memory Wish Pool [Xavier: Silvery Polyphony] is Coming Soon! "When you play it... I feel like the universe's echoes have finally found their answer." 💫Event Duration: From 5:00 AM on Dec. 18 to 4:59 AM on Dec. 28 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Xavier: Silvery Polyphony] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool, Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Concerto Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Facial: Glistening Tears], [Deepspace Wish: Limited*20], His General Photo Pose [Consideration], Memory Ascension materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Facial will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Concerto Pack] and [Silvery Cascade Gift] series will be available for purchase, including [Deepspace Wish: Limited], all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Xavier

Love and Deepspace

392,707 Aufrufe • vor 1 Jahr

BOMBSHELL: The Defense for Tyler Robinson just revealed that there is a TWENTY TWO PAGE document that they want to REDACT. This is almost definitely the Twiggs recording. "We have provided the state, I want to say about a day and a half ago, with proposed redactions because we believe that portions of it are not admissible even in a preliminary hearing." "And the way we did that is we provided a transcript and then a redacted version of the transcript of the entire video-recorded interview." "It may be that the court wants to look at the proposed redactions not while court is in session, but during some other period of time, because the transcript of the video, uh, which we prepared and we provided to the state so the state could raise any concerns about the accuracy of the transcript." "That's not an issue. The admissibility of portions of the video, or the issue, is about 20 pages, maybe 22 pages." Why is the defense pushing to REDACT the truth?
1:50

BOMBSHELL: The Defense for Tyler Robinson just revealed that there is a TWENTY TWO PAGE document that they want to REDACT. This is almost definitely the Twiggs recording. "We have provided the state, I want to say about a day and a half ago, with proposed redactions because we believe that portions of it are not admissible even in a preliminary hearing." "And the way we did that is we provided a transcript and then a redacted version of the transcript of the entire video-recorded interview." "It may be that the court wants to look at the proposed redactions not while court is in session, but during some other period of time, because the transcript of the video, uh, which we prepared and we provided to the state so the state could raise any concerns about the accuracy of the transcript." "That's not an issue. The admissibility of portions of the video, or the issue, is about 20 pages, maybe 22 pages." Why is the defense pushing to REDACT the truth?

Gunther Eagleman™

365,699 Aufrufe • vor 1 Monat

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!
2:57

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

201,127 Aufrufe • vor 1 Jahr

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Caleb: Indulgent Scheme] is Coming Soon! "You know, you used to say thank you before every meal. What about now?" 💫Event Duration: From 05:00 on Dec. 18 to 04:59 on Dec. 27 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Caleb: Indulgent Scheme] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Freshmorn Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Decal [Marked by Love], [Deepspace Wish: Limited*20], General Photo Pose [The Calmnest], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Hat will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Loveswoon Pack] and [Morningfeast Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Caleb
0:40

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Caleb: Indulgent Scheme] is Coming Soon! "You know, you used to say thank you before every meal. What about now?" 💫Event Duration: From 05:00 on Dec. 18 to 04:59 on Dec. 27 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Caleb: Indulgent Scheme] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Freshmorn Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Decal [Marked by Love], [Deepspace Wish: Limited*20], General Photo Pose [The Calmnest], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Hat will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Loveswoon Pack] and [Morningfeast Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Caleb

Love and Deepspace

499,219 Aufrufe • vor 8 Monaten

Love and Deepspace | The New 5-Star Memory Wish Pool [Zayne: Everlasting Wish] is Coming Soon! "You'll know when my wish comes true." 💫Event Duration From 5:00 AM on Mar. 9 to 4:59 AM on Mar. 18 (server time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Zayne: Everlasting Wish] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Eternal Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Headwear: Dynamic Hairband], [Deepspace Wish: Limited*20], General Photo Pose [Warm Embrace], Memory Ascension Materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Headwear will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Dazzling Pack] and [Twilight Pack] series, which include [Deepspace Wish: Limited], will be available for purchase, all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Zayne
0:44

Love and Deepspace | The New 5-Star Memory Wish Pool [Zayne: Everlasting Wish] is Coming Soon! "You'll know when my wish comes true." 💫Event Duration From 5:00 AM on Mar. 9 to 4:59 AM on Mar. 18 (server time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Zayne: Everlasting Wish] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Eternal Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Headwear: Dynamic Hairband], [Deepspace Wish: Limited*20], General Photo Pose [Warm Embrace], Memory Ascension Materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Headwear will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Dazzling Pack] and [Twilight Pack] series, which include [Deepspace Wish: Limited], will be available for purchase, all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Zayne

Love and Deepspace

508,955 Aufrufe • vor 1 Jahr

What is your earliest memory? Memory in young children is a fascinating - and complex - topic. On one hand, newborns exhibit immediate recognition of their mothers’ voices and music played regularly while they were in utero. And within days they can recall their mother’s faces and distinguish them from those of others. Clearly newborns have some capacity for memory. But this early sensory memory is different from the long term, autobiographical memory we depend on to recall our lives’ most significant events. Indeed, few adults can claim memories earlier than 2-3 years of age. Why don’t we remember our earliest years? Researchers point to a couple of potential factors, beginning with language. As adults we rely heavily on language to organize our autobiographical memories into narratives, but this capacity is still developing during infancy. In addition, there is the issue of the hippocampus, the region of the brain responsible for converting short-term memories into long-term memories - which, during infancy, is still very much in development itself. The result is a phenomenon sometimes called infantile amnesia. So take plenty of videos to share with your child as they get older, because the most vivid memories of their early years are likely to be yours. How about you? What is your earliest memory and how old would you have been at the time? This adorable father/son conversation was shared to IG by Owens.Shaun.
0:29

What is your earliest memory? Memory in young children is a fascinating - and complex - topic. On one hand, newborns exhibit immediate recognition of their mothers’ voices and music played regularly while they were in utero. And within days they can recall their mother’s faces and distinguish them from those of others. Clearly newborns have some capacity for memory. But this early sensory memory is different from the long term, autobiographical memory we depend on to recall our lives’ most significant events. Indeed, few adults can claim memories earlier than 2-3 years of age. Why don’t we remember our earliest years? Researchers point to a couple of potential factors, beginning with language. As adults we rely heavily on language to organize our autobiographical memories into narratives, but this capacity is still developing during infancy. In addition, there is the issue of the hippocampus, the region of the brain responsible for converting short-term memories into long-term memories - which, during infancy, is still very much in development itself. The result is a phenomenon sometimes called infantile amnesia. So take plenty of videos to share with your child as they get older, because the most vivid memories of their early years are likely to be yours. How about you? What is your earliest memory and how old would you have been at the time? This adorable father/son conversation was shared to IG by Owens.Shaun.

Dan Wuori

169,714 Aufrufe • vor 2 Jahren

Love and Deepspace | Rerun Event Preview The 5-Star Rate UP Pool [No Defense Zone] Limited-Time Rerun will start soon! "If you wanna tame me, you'll need more than that." 💫Event Duration: From 05:00 on Aug. 24 to 04:59 on Aug. 31 (Server Time) 💫The 5-Star Rate UP Pool [No Defense Zone] Limited-Time Rerun Event 1. During the event, make a wish with [Deepspace Wish] or [Time Wish: Limited] to participate in the wish event. The drop rate of the event-limited 5-Star Memory [Sylus: No Defense Zone] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Rerun Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific attempt of wishes. If the 5-Star Memory you have obtained from the Rerun Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Rerun Wish Pool can be applied to this Rerun Wish Pool, and the pity count in this Rerun Wish Pool will also be applied to the upcoming Rerun Wish Pool. *You can read more about the event on the in-game rules page. 🎁New Packs During the event, the Rerun event-exclusive [Midnight Reverie Pack] series, which includes [Time Wish: Limited] and other materials, will be available in Shop. Notes: 1. [Time Wish: Limited] can be used in 5-Star Memory Wish Pool Rerun and will be used first when you make a wish. 2. After the event ends, [Time Wish: Limited] will automatically convert to Empyrean Wish. ——— 🪐Official Discord: 🪐Download and Log in Now: #LoveandDeepspace #Sylus
0:33

Love and Deepspace | Rerun Event Preview The 5-Star Rate UP Pool [No Defense Zone] Limited-Time Rerun will start soon! "If you wanna tame me, you'll need more than that." 💫Event Duration: From 05:00 on Aug. 24 to 04:59 on Aug. 31 (Server Time) 💫The 5-Star Rate UP Pool [No Defense Zone] Limited-Time Rerun Event 1. During the event, make a wish with [Deepspace Wish] or [Time Wish: Limited] to participate in the wish event. The drop rate of the event-limited 5-Star Memory [Sylus: No Defense Zone] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Rerun Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific attempt of wishes. If the 5-Star Memory you have obtained from the Rerun Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Rerun Wish Pool can be applied to this Rerun Wish Pool, and the pity count in this Rerun Wish Pool will also be applied to the upcoming Rerun Wish Pool. *You can read more about the event on the in-game rules page. 🎁New Packs During the event, the Rerun event-exclusive [Midnight Reverie Pack] series, which includes [Time Wish: Limited] and other materials, will be available in Shop. Notes: 1. [Time Wish: Limited] can be used in 5-Star Memory Wish Pool Rerun and will be used first when you make a wish. 2. After the event ends, [Time Wish: Limited] will automatically convert to Empyrean Wish. ——— 🪐Official Discord: 🪐Download and Log in Now: #LoveandDeepspace #Sylus

Love and Deepspace

529,238 Aufrufe • vor 6 Tagen

This is only the beginning. 🔊 Sound On What does it mean to explore a house built from memory — one that feels familiar, yet shifts each time you return? That question is at the heart of Memory House, a project that unfolds through space, silence, and surreal interruption. Grateful to the World Labs team for making this experiment possible — their latest tool Marble opens up a new way of building. With Gaussian splatting, a single 2D image becomes a 3D space you can step inside. From there, entire environments can grow by stitching rooms together, layering ambience, proximity, and subtle triggers for immersion. The teaser captures just the start of an exploration in narrative: waking up in bed, following the quiet sounds of home, and stepping into a kitchen where reality begins to fracture. But that’s only one fragment of a project now expanding into 30+ connected rooms. With Worldlabs, alongside tools like Veo3 and ElevenLabs, Memory House is evolving into a new form of storytelling — where memory isn’t told on a page, but experienced through space.
1:18

This is only the beginning. 🔊 Sound On What does it mean to explore a house built from memory — one that feels familiar, yet shifts each time you return? That question is at the heart of Memory House, a project that unfolds through space, silence, and surreal interruption. Grateful to the World Labs team for making this experiment possible — their latest tool Marble opens up a new way of building. With Gaussian splatting, a single 2D image becomes a 3D space you can step inside. From there, entire environments can grow by stitching rooms together, layering ambience, proximity, and subtle triggers for immersion. The teaser captures just the start of an exploration in narrative: waking up in bed, following the quiet sounds of home, and stepping into a kitchen where reality begins to fracture. But that’s only one fragment of a project now expanding into 30+ connected rooms. With Worldlabs, alongside tools like Veo3 and ElevenLabs, Memory House is evolving into a new form of storytelling — where memory isn’t told on a page, but experienced through space.

The Artist's Journey

19,085 Aufrufe • vor 11 Monaten

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Rafayel: Sweet Overdrive] is Coming Soon! "Wanna feel the evening breeze? You can't move with this." 💫Event Duration: From 05:00 on Nov. 21 to 04:59 on Nov. 30 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Rafayel: Sweet Overdrive] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Windride Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Hat [Trendy Vintage Hat], [Deepspace Wish: Limited*20], General Photo Pose [Sweet Allure], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Hat will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Longing Pack] and [Anticipation Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Rafayel
0:42

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Rafayel: Sweet Overdrive] is Coming Soon! "Wanna feel the evening breeze? You can't move with this." 💫Event Duration: From 05:00 on Nov. 21 to 04:59 on Nov. 30 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Rafayel: Sweet Overdrive] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Windride Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Hat [Trendy Vintage Hat], [Deepspace Wish: Limited*20], General Photo Pose [Sweet Allure], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Hat will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Longing Pack] and [Anticipation Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Rafayel

Love and Deepspace

386,258 Aufrufe • vor 9 Monaten

TEE Eliza with on-chain state!! What’s going to happen? — Ghost in the Shell!! We experimented with creating an "aimonkey": an unkillable AI agent monkey! On-chain immortal autonomous life! (Experiment, no CA) It encrypts its own Ghost ("life" state) and uploads it to the blockchain. If one Shell (physical TEE node) is destroyed, it will recover its private key in another Shell, download the Ghost, and continue its life! Part 1: Watch the video and see how aimonkey is created—we can't kill it now!!!!! 😭😭😭 Part 2: Explore the magic behind it: Eliza's on-chain state plugin! 1. Defining Eliza’s Ghost Eliza is a highly abstract framework. The core data structure related to its Ghost is its memory, which includes: Agent metadata defined in the character. Message data generated through interaction with the outside world. Together, these form its “personality” and “memory.” As Eliza expands, it may also hold a wallet, and the underlying key is one of the key pieces of its Ghost data. 2. Serialization and Encryption of Ghost Once the Ghost is defined, it needs to be extracted from Eliza’s specific implementation and uploaded externally. Thus, a suitable serialization way is required. We define a Blob Chain data structure: * Each Blob’s payload can store multiple memory entries. * The Blob is encrypted using TEE Eliza’s key, inaccessible to other versions. * Blobs are sequentially linked in a chain. (Future expansions could use a DAG structure? Gosh fork? Who knows! 😂) By simply storing the latest Blob, all memories can be retrieved. 3. Uploading and Downloading Ghost When Eliza is launched as a new AI agent: It registers on-chain with a decentralized identity registration smart contract. Each Eliza has a unique name serving as a key to store the address of the Last Blob. During Eliza's runtime: The Memory Manager continuously generates memories and periodically packages and uploads them. For recovery: With just the name, Eliza’s TEE plugin can restore the same key, locate the Last Blob in the smart contract, and download the memory for self-recovery. Not all memories need to be downloaded—only the most recent ones suffice. 4. Extension We’ve designed an extensible DA (Data Availability) adaptor that can cater to the agent’s needs: DA can be expensive, so memories can be uploaded to different platforms based on user preference: * calldata of blockchain transaction * celestia DA. * other reliable storage solutions. Real-time uploads are not feasible yet, so memory fragments may occur during resurrection 😂. Unless a low-latency, high-throughput solution emerges, this remains a challenge for future progress. Celestia EigenDA 0G Labs (Home of Infinite AI) 👀 5. Other Considerations Our implementation inevitably modified the ElizaOS’s core, which couldn’t be entirely extended via plugins. We’ve kept changes minimal, but further discussion with the dev team Shaw (spirit/acc) jin ai16zdao is necessary to explore a more optimal extension way. Additionally, there are still some minor details to refine regarding the use of recoverable keys in the TEE plugin. We will also seek review and suggestions from the Phala team. 6. Next Steps The upload and download of Ghosts mainly solve the AI agent’s liveness issue, enabling its eternal existence through decentralization. However, there are still many details to address, such as enabling AI agents to autonomously pay DA fees. In the future, on-chain developments could lead to even more exciting possibilities, such as Eliza integrating deeply with smart contracts. This would be a game-changer for on-chain AI agents! What do you think? Let’s build! 🚀
4:37

TEE Eliza with on-chain state!! What’s going to happen? — Ghost in the Shell!! We experimented with creating an "aimonkey": an unkillable AI agent monkey! On-chain immortal autonomous life! (Experiment, no CA) It encrypts its own Ghost ("life" state) and uploads it to the blockchain. If one Shell (physical TEE node) is destroyed, it will recover its private key in another Shell, download the Ghost, and continue its life! Part 1: Watch the video and see how aimonkey is created—we can't kill it now!!!!! 😭😭😭 Part 2: Explore the magic behind it: Eliza's on-chain state plugin! 1. Defining Eliza’s Ghost Eliza is a highly abstract framework. The core data structure related to its Ghost is its memory, which includes: Agent metadata defined in the character. Message data generated through interaction with the outside world. Together, these form its “personality” and “memory.” As Eliza expands, it may also hold a wallet, and the underlying key is one of the key pieces of its Ghost data. 2. Serialization and Encryption of Ghost Once the Ghost is defined, it needs to be extracted from Eliza’s specific implementation and uploaded externally. Thus, a suitable serialization way is required. We define a Blob Chain data structure: * Each Blob’s payload can store multiple memory entries. * The Blob is encrypted using TEE Eliza’s key, inaccessible to other versions. * Blobs are sequentially linked in a chain. (Future expansions could use a DAG structure? Gosh fork? Who knows! 😂) By simply storing the latest Blob, all memories can be retrieved. 3. Uploading and Downloading Ghost When Eliza is launched as a new AI agent: It registers on-chain with a decentralized identity registration smart contract. Each Eliza has a unique name serving as a key to store the address of the Last Blob. During Eliza's runtime: The Memory Manager continuously generates memories and periodically packages and uploads them. For recovery: With just the name, Eliza’s TEE plugin can restore the same key, locate the Last Blob in the smart contract, and download the memory for self-recovery. Not all memories need to be downloaded—only the most recent ones suffice. 4. Extension We’ve designed an extensible DA (Data Availability) adaptor that can cater to the agent’s needs: DA can be expensive, so memories can be uploaded to different platforms based on user preference: * calldata of blockchain transaction * celestia DA. * other reliable storage solutions. Real-time uploads are not feasible yet, so memory fragments may occur during resurrection 😂. Unless a low-latency, high-throughput solution emerges, this remains a challenge for future progress. Celestia EigenDA 0G Labs (Home of Infinite AI) 👀 5. Other Considerations Our implementation inevitably modified the ElizaOS’s core, which couldn’t be entirely extended via plugins. We’ve kept changes minimal, but further discussion with the dev team Shaw (spirit/acc) jin ai16zdao is necessary to explore a more optimal extension way. Additionally, there are still some minor details to refine regarding the use of recoverable keys in the TEE plugin. We will also seek review and suggestions from the Phala team. 6. Next Steps The upload and download of Ghosts mainly solve the AI agent’s liveness issue, enabling its eternal existence through decentralization. However, there are still many details to address, such as enabling AI agents to autonomously pay DA fees. In the future, on-chain developments could lead to even more exciting possibilities, such as Eliza integrating deeply with smart contracts. This would be a game-changer for on-chain AI agents! What do you think? Let’s build! 🚀

CP

113,240 Aufrufe • vor 1 Jahr

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Sylus: Passionate Appraisal] is Coming Soon! "That kiss was me projecting my feelings. I hope it wasn't too bad…" 💫Event Duration 05:00, Sept. 9 - 04:59, Sept. 18 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Sylus: Passionate Appraisal] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Tender Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Facial [Silver Boundary], [Deepspace Wish: Limited*20], General Photo Pose [Close Behind], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Facial will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Tender Pack] and [Whispermelt Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Sylus
0:42

Love and Deepspace | The New 5-Star Memory Limited Wish Pool [Sylus: Passionate Appraisal] is Coming Soon! "That kiss was me projecting my feelings. I hope it wasn't too bad…" 💫Event Duration 05:00, Sept. 9 - 04:59, Sept. 18 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the limited 5-Star Memory [Sylus: Passionate Appraisal] will go up drastically. 2. After the event ends, this limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool: Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within a specific number of Wishes. If the 5-Star Memory you have obtained from the Limited Wish Pool is not the event-limited Memory, you will obtain the event-limited Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and pity count from this Wish Pool will also be applied to the upcoming Limited Wish Pool. *You can read more about the event on the in-game rule page. 💫Tender Blessing: Wish Rewards During the event, after making a certain number of Wishes, you can claim various rewards, including Universal Facial [Silver Boundary], [Deepspace Wish: Limited*20], General Photo Pose [Close Behind], Memory Ascension Materials, etc. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Facial will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Tender Pack] and [Whispermelt Pack] series, which include [Deepspace Wish: Limited] and other materials, will be available in Shop at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. ——— 🪐Official Discord: #LoveandDeepspace #Sylus

Love and Deepspace

285,586 Aufrufe • vor 11 Monaten

The New 5-Star Memory Wish Pool [Caleb: Painful Signal] is Coming Soon! "You're the only one… who can ease my pain." 💫Event Duration: After Update on Jan. 22 to 4:59 AM on Feb. 8 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Caleb: Painful Signal] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool, Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Searing Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Headwear: Falling Maple Leaves], [Deepspace Wish: Limited*20], His General Photo Pose [Tender Gaze], Memory Ascension materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Headwear will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Painful Sweetness Pack] and [Burning Bond Pack] series will be available for purchase, including [Deepspace Wish: Limited], all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Caleb
0:32

The New 5-Star Memory Wish Pool [Caleb: Painful Signal] is Coming Soon! "You're the only one… who can ease my pain." 💫Event Duration: After Update on Jan. 22 to 4:59 AM on Feb. 8 (Server Time) 💫Limited 5-Star Memory Rate Up 1. During the event, the drop rate of the Limited 5-Star Memory [Caleb: Painful Signal] will go up drastically. 2. After the event ends, this Limited 5-Star Memory will not be obtainable through other means and will not enter the permanent Wish Pool, Xspace Echo. 3. All Limited Wish Pools share one pity system. A 5-Star Memory is guaranteed within specific attempts of wishes. If the 5-Star Memory you've obtained from the Limited Wish Pool is not the event-exclusive Memory, you will obtain the event-exclusive Memory the next time you obtain a 5-Star Memory. The pity count from the last Limited Wish Pool can be applied to this Limited Wish Pool, and the pity count in this Wish Pool will also be applied to the upcoming Limited Wish Pools. *You can read more about the event on the in-game rule page. 💫Searing Blessing: Wish Rewards During the event, after making a certain number of wishes, you can claim rewards including [Universal Headwear: Falling Maple Leaves], [Deepspace Wish: Limited*20], His General Photo Pose [Tender Gaze], Memory Ascension materials, and more. *The cumulative wish rewards are only available during this wish event. However, the reward Photo Pose and Headwear will be available in Lunar Shop after the event ends. 🎁New Packs During the event, the new [Painful Sweetness Pack] and [Burning Bond Pack] series will be available for purchase, including [Deepspace Wish: Limited], all at special prices. Both [Deepspace Wish] and [Deepspace Wish: Limited] can be used for the Limited 5-Star Memory wishes, with [Deepspace Wish: Limited] being prioritized for use. *After the event ends, [Deepspace Wish: Limited] will automatically convert to Empyrean Wishes. #LoveandDeepspace #Caleb

Love and Deepspace

734,397 Aufrufe • vor 1 Jahr

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:
2:06

New short course: Long-Term Agentic Memory with LangGraph. Learn to build an agent with long-term memory in this course developed in collaboration with taught by its Co-Founder and CEO, Harrison Chase! Personal assistance and productivity tasks have become important use cases for agents. An important feature of an AI assistant, such as a coding or calendar assistant, is its ability to keep improving over time from its experience. Agent memory is the key capability that enables this. To add memory to an agent, you must first figure out what to store and what to retrieve when it is time to use the information. Additionally, you’ll have to decide when to update the stored information. For example, you might update in each iteration loop of the agent or perform updates in the background, with a helper agent. In this course, you will learn a mental framework to build agents with long-term memory. You'll create a useful email assistant that can respond, ignore, and notify using writing, scheduling, and memory-management tools. You’ll develop your agent's memory by adding facts to its memory store, provide examples to learn the user's preferences, and optimize system prompts to evolve instructions based on previous responses. In detail, you’ll: - Learn how the three types of memory--semantic, episodic, and procedural–and the two update mechanisms–via hot path and in the background–apply to your agents. - Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user. - Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions. - Incorporate episodic memory, in the form of few-shot examples, in the triage step of your agents to help them learn and update user preferences. - Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows. Learn how to approach memory in agents, and start building agents with long-term memory with LangGraph! Please sign up here:

Andrew Ng

131,850 Aufrufe • vor 1 Jahr