Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Why is Redis Fast? Redis is fast for in-memory data storage. Its speed has made it popular for caching, session storage, and real-time analytics. But what gives Redis its blazing speed? Let's explore: RAM-Based Storage At its core, Redis primarily uses main memory for storing data. Accessing data from... RAM is orders of magnitude faster than from disk. This is a major reason for Redis's speed. However, RAM is volatile. To persist data, Redis supports disk snapshots and append-only file logging. This combines RAM's performance with disk's permanence. There is a tradeoff though - recovery from disk is slow. If a Redis instance fails, restarting from disk can be slow compared to failing over to a replica instance fully in memory. So while Redis offers durability via disk, it comes at the cost of slower recovery. A better solution is Redis replication. With a synchronized replica kept in memory, failover is instant with no rehydration. This maintains speed and near-instant recovery. IO Multiplexing & Single-threaded Read/Write Redis uses an event-driven, single-threaded model for its core operations. A main event loop handles all client requests and data operations sequentially. This single-threaded execution avoids context switching and synchronization overhead typical of multi-threaded systems. Redis uses non-blocking I/O to handle multiple connections asynchronously. This allows it to support many client connections with very low overhead, Redis does leverage threading in certain areas: - Background tasks like taking snapshots. - I/O threads are used for certain operations. - Modules can use threads. - Since Redis 6.0, it supports multi-threaded I/O for network communication, improving performance on multi-core systems. Redis also uses pipelining for high throughput. Clients pipeline commands without waiting for each response. This allows more efficient network round trips, boosting overall performance. Efficient Data Structures Redis supports various optimized data structures, from linked lists, zip lists, and skip lists to sets, hashes, and sorted sets, among others. Each is carefully designed for specific use cases for quick and efficient data access. Over to you: With Redis now supporting some multi-threading, how should we configure it to fully utilize all the CPU cores of modern hardware when deploying in production? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages):show more

Sahn Lam

48,505 subscribers

46,910 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 Comments

LS2 years ago

Threads same as the CPU cores. I believe vertex does something similar.

Abhishek Yadav2 years ago

Insight full after reading, Thanks for sharing ...

Jake2 years ago

> how should we configure it to fully utilize all the CPU cores of modern hardware when deploying in production? You can't. You're going to be bottlenecked on that single thread doing all the data access. Best you can do is run multiple redis instances or use something else.

Vinay2 years ago

Thanks for sharing

Suresh Jayanthi2 years ago

With the #ibmlinuxone one can have up to 40 TB of memory and 4 levels of cache and the #s390x cpu architecture excels at single threaded performance. You may want to explore this angle

Who Am I2 years ago

+

Gomaji Kapse2 years ago

Can use multi threading at I/O level by splitting the socket fds per thread, which gets run on their own core.

Alex 📚 PromptLeo2 years ago

It is indeed fascinating that Redis is single threaded and that’s still enough for the performance in comcurrent environments. Is this because network bandwidth is lower than disk access anyway?

Related Videos

Start building for an agent-first world. If you have a product, you need to start offering skills for Claude, Codex, Cursor, and any other agents. Your skills should specify: • How to navigate and use your product • Best practices the agent must follow • Detailed instructions on how to accomplish things • Anti-patterns to avoid Redis is one of the most popular in-memory data stores in the world, and they just released their agent skills. It takes one second to install, and it will turn your agent into a Senior Redis Engineer: $ npx skills add redis/agent-skills In the attached video, I show you how to install it as a plugin in Claude Code and some of its benefits. This is the easiest way to "teach" models what they don't know and keep their knowledge up to date. If you ask me, skills is literally one of the most brilliant ideas that Anthropic has put out there. If you use Redis, their skill is a must-have. If you don't, this skill will show you how to build and structure yours. Here is what their skill teaches your agent: 1. Current patterns for common use cases: caching, rate limiting, session management, vector search, semantic caching, pub/sub, streams. 2. Which data structure to use and when: hashes vs. JSON vs. sorted sets vs. vector sets. 3. Anti-patterns to avoid: no KEYS in loops, no unbounded key growth, no large values that amplify every operation. 4. Production-aware defaults: connection pooling, pipelining, cluster compatibility, error handling that doesn't silently swallow failures.

Start building for an agent-first world. If you have a product, you need to start offering skills for Claude, Codex, Cursor, and any other agents. Your skills should specify: • How to navigate and use your product • Best practices the agent must follow • Detailed instructions on how to accomplish things • Anti-patterns to avoid Redis is one of the most popular in-memory data stores in the world, and they just released their agent skills. It takes one second to install, and it will turn your agent into a Senior Redis Engineer: $ npx skills add redis/agent-skills In the attached video, I show you how to install it as a plugin in Claude Code and some of its benefits. This is the easiest way to "teach" models what they don't know and keep their knowledge up to date. If you ask me, skills is literally one of the most brilliant ideas that Anthropic has put out there. If you use Redis, their skill is a must-have. If you don't, this skill will show you how to build and structure yours. Here is what their skill teaches your agent: 1. Current patterns for common use cases: caching, rate limiting, session management, vector search, semantic caching, pub/sub, streams. 2. Which data structure to use and when: hashes vs. JSON vs. sorted sets vs. vector sets. 3. Anti-patterns to avoid: no KEYS in loops, no unbounded key growth, no large values that amplify every operation. 4. Production-aware defaults: connection pooling, pipelining, cluster compatibility, error handling that doesn't silently swallow failures.

Santiago

37,546 views • 4 months ago

This is my assignment for a SF startup as a backend and infra engineer the main goal was to build a workflow engine that executes directed acyclic graphs of tasks for this assessment the task would arbitrary HTTP endpoints. But I replaced it with my own usage of news aggregator from hackernews, github (top repos for VAE, and libs related to Pytorch) and threads on reddit related to topic ML This I can use in many paid gigs too as a CD deployements like test > build > deploy staging > tests > deploy prod > notify but for now it handles simple tasks and its built on go, redis, grpc with htmx and tailwindcss

This is my assignment for a SF startup as a backend and infra engineer the main goal was to build a workflow engine that executes directed acyclic graphs of tasks for this assessment the task would arbitrary HTTP endpoints. But I replaced it with my own usage of news aggregator from hackernews, github (top repos for VAE, and libs related to Pytorch) and threads on reddit related to topic ML This I can use in many paid gigs too as a CD deployements like test > build > deploy staging > tests > deploy prod > notify but for now it handles simple tasks and its built on go, redis, grpc with htmx and tailwindcss

Mrinal

144,678 views • 5 months ago

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Brian Roemmele

87,151 views • 2 years ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,788 views • 1 year ago

MCP is mind-blowing. Here is a genius setup that will earn you a promotion with your manager. I have multiple data sources: data in S3, MySQL, and local files. Using Spice (an open-source project), I can co-locate all of that data behind a single SQL endpoint and augment it using MCP. Before, accessing data was slow, and I had to make multiple requests to get anything. After using Spice, I can query the data much faster (since it's mirrored locally), and I don't have to worry about where the data is coming from. Spice acts as an MCP server, so I can connect to it from my IDE or the Claude Desktop app and have access to all the data. But even more cool than that: Spice also acts as an MCP client, so it can connect to other MCP servers. This is where things get really interesting! Watch the video. I'm querying multiple data sources from my IDE and using the Google Maps MCP server for driving directions.

MCP is mind-blowing. Here is a genius setup that will earn you a promotion with your manager. I have multiple data sources: data in S3, MySQL, and local files. Using Spice (an open-source project), I can co-locate all of that data behind a single SQL endpoint and augment it using MCP. Before, accessing data was slow, and I had to make multiple requests to get anything. After using Spice, I can query the data much faster (since it's mirrored locally), and I don't have to worry about where the data is coming from. Spice acts as an MCP server, so I can connect to it from my IDE or the Claude Desktop app and have access to all the data. But even more cool than that: Spice also acts as an MCP client, so it can connect to other MCP servers. This is where things get really interesting! Watch the video. I'm querying multiple data sources from my IDE and using the Google Maps MCP server for driving directions.

Santiago

142,696 views • 1 year ago

Collecting data is a PAAAAAAAIN❗ If only there were a solution out there... (𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 😱) Jannik Grothusen, researchers and founder in stealth mode 👀 from TU Munich, has created a new tool that helps collect data for robots to learn how to perform tasks. It’s inspired by a device from Stanford, but it’s even better! Here’s why: ✅ It can collect and process data in real-time, meaning no delays. ✅ It’s super easy to control, so collecting data is much faster and simpler. ✅ It can be used with different robot parts because of its smart, modular design. Best of all, this amazing tool is 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲, so anyone can use it or improve it! Github: The system is inspired by Cheng Chi; Universal Manipulation Interface (UMI). Credit: Jannik Grothusen

Collecting data is a PAAAAAAAIN❗ If only there were a solution out there... (𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 😱) Jannik Grothusen, researchers and founder in stealth mode 👀 from TU Munich, has created a new tool that helps collect data for robots to learn how to perform tasks. It’s inspired by a device from Stanford, but it’s even better! Here’s why: ✅ It can collect and process data in real-time, meaning no delays. ✅ It’s super easy to control, so collecting data is much faster and simpler. ✅ It can be used with different robot parts because of its smart, modular design. Best of all, this amazing tool is 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲, so anyone can use it or improve it! Github: The system is inspired by Cheng Chi; Universal Manipulation Interface (UMI). Credit: Jannik Grothusen

Ilir Aliu

47,397 views • 1 year ago

Today, we’re pushing a major update to Edison Analysis, our data analysis agent, which is tuned for scientific research and SOTA across data analysis benchmarks. In contrast to Kosmos, which runs for 6-12 hours and produces tens of thousands of lines of code, Edison Analysis runs for seconds to minutes and is best for specific, well-defined computational tasks. It is available both on our platform under the Analysis tab, and via API, and costs only one credit per run, so it is available to users on both free and paid tiers. Edison Analysis is a modified version of the data analysis agent Kosmos uses in its trajectories. Try it out! One of the most important improvements over our previous data analysis agents has been the addition of a specialized data retrieval tool. Edison Analysis can either use this tool to access data, or can pull data down directly via API. To evaluate this tool, we ranked the most commonly used public data repositories across recent papers from BioRxiv, and created a new benchmark that measures the ability of a language agent system to retrieve raw data from those sources. Edison Analysis gets 71% on this benchmark, and we’ll be working to increase this over time. You can read more about our benchmarks in the our blog post, link below. Some features worth highlighting: 1. Edison Analysis produces a report on the analysis it runs, along with a Jupyter notebook that you can download to reproduce the analysis yourself. Every figure it produces is linked back to the specific lines of code used to produce the figure, to make it easy to reproduce. 2. It works well with both Python and R. 3. One of the best uses for Edison Analysis is to use it to retrieve datasets that you can then analyze with Kosmos. We have a bunch of major improvements to Edison Analysis coming in the next few months that we’re excited to share. In the meantime, congratulations to the team, especially Ludovico Mitchener, Jon Laurent, Conor Igoe , Alex Andonian, and many more.

Today, we’re pushing a major update to Edison Analysis, our data analysis agent, which is tuned for scientific research and SOTA across data analysis benchmarks. In contrast to Kosmos, which runs for 6-12 hours and produces tens of thousands of lines of code, Edison Analysis runs for seconds to minutes and is best for specific, well-defined computational tasks. It is available both on our platform under the Analysis tab, and via API, and costs only one credit per run, so it is available to users on both free and paid tiers. Edison Analysis is a modified version of the data analysis agent Kosmos uses in its trajectories. Try it out! One of the most important improvements over our previous data analysis agents has been the addition of a specialized data retrieval tool. Edison Analysis can either use this tool to access data, or can pull data down directly via API. To evaluate this tool, we ranked the most commonly used public data repositories across recent papers from BioRxiv, and created a new benchmark that measures the ability of a language agent system to retrieve raw data from those sources. Edison Analysis gets 71% on this benchmark, and we’ll be working to increase this over time. You can read more about our benchmarks in the our blog post, link below. Some features worth highlighting: 1. Edison Analysis produces a report on the analysis it runs, along with a Jupyter notebook that you can download to reproduce the analysis yourself. Every figure it produces is linked back to the specific lines of code used to produce the figure, to make it easy to reproduce. 2. It works well with both Python and R. 3. One of the best uses for Edison Analysis is to use it to retrieve datasets that you can then analyze with Kosmos. We have a bunch of major improvements to Edison Analysis coming in the next few months that we’re excited to share. In the meantime, congratulations to the team, especially Ludovico Mitchener, Jon Laurent, Conor Igoe , Alex Andonian, and many more.

Sam Rodriques

61,860 views • 7 months ago

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

Lior Alexander

808,089 views • 3 years ago

More progress! I now have two Dockerized Gradio | Rerun apps. The first one takes as input a "raw" rrd file that consists of the synchronized egocentric and exocentric MP4 files. This runs the pipeline and produces an "annotated" rrd file. This has the camera parameters, 3D joints, and projected 2D joints (with 6DOF mano soon). The second app takes this "annotated" rrd file and allows for manual labeling. This is a crucial step in addressing any major failures in the pipeline. Right now, it is only the ego view that can be modified. But I'll eventually extend to all. This results in a final "gt" rrd file. From here, the plan is to improve quality and start building a data loop. Excited to start really scaling this. I'm basically going all in on keeping my data stored as Rerun rrd files. As always, I want to emphasize how crucial it is to LOOK AT YOUR data! The rrd format makes it incredibly easy to do so. Getting the data out to use is a bit of a hassle right now, but for me, it's well worth the tradeoff.

More progress! I now have two Dockerized Gradio | Rerun apps. The first one takes as input a "raw" rrd file that consists of the synchronized egocentric and exocentric MP4 files. This runs the pipeline and produces an "annotated" rrd file. This has the camera parameters, 3D joints, and projected 2D joints (with 6DOF mano soon). The second app takes this "annotated" rrd file and allows for manual labeling. This is a crucial step in addressing any major failures in the pipeline. Right now, it is only the ego view that can be modified. But I'll eventually extend to all. This results in a final "gt" rrd file. From here, the plan is to improve quality and start building a data loop. Excited to start really scaling this. I'm basically going all in on keeping my data stored as Rerun rrd files. As always, I want to emphasize how crucial it is to LOOK AT YOUR data! The rrd format makes it incredibly easy to do so. Getting the data out to use is a bit of a hassle right now, but for me, it's well worth the tradeoff.

Pablo Vela

19,527 views • 8 months ago

Major program launch: Data Analytics Professional Certificate! This large, five-course sequence takes you all the way to being job-ready as a data analyst, and shows how to use Generative AI as a thought partner to enhance your work in this role. Offered by on Coursera, this is taught by Sean Barnes, Ph.D., a Data Science & Engineering Leader at Netflix. Analyzing data remains one of the most important skills in where the world is going with AI. This comprehensive certificate takes you all the way to being job-ready. Each course comes with practical projects demonstrated in real-world contexts, such as analyzing sales data for a Korean bakery, video game sales trends across different regions, or identifying factors impacting customer retention for a communications company. You'll also work on estimating fire distribution for forest fire prevention, analyzing how a diamond's properties affect its market value, and developing predictive models for retail sales analysis, carbon emissions, and coral reef conservation. Here's some of what you'll learn: - How to define data and categorize it into its many types such as discrete & continuous numerical, structured & unstructured, time series, categorical, and know what insights can be derived from the different types of data categories. - How to differentiate between data-related job roles and their responsibilities, and how data flows through an organization from the moment of capture to decision-making. - How to perform data processing functions and apply conditional formatting in spreadsheets to extract business value from your data using statistical calculations and best practices for visualizing and interpreting data. - How to use LLMs for stakeholder analysis, data exploration, and data visualization. - Best practices for using LLMs for as a thought partner to data analysis work By the end of this professional certificate program, you will have learned core statistical concepts, analysis techniques, and visualization methodologies that will serve as the foundation for working as a data analyst. The world needs more data analysts, especially ones who know how to use modern generative AI. With data science roles projected to grow 36% by 2033, the skills taught in this program create new professional opportunities in data. Sign up here!

Major program launch: Data Analytics Professional Certificate! This large, five-course sequence takes you all the way to being job-ready as a data analyst, and shows how to use Generative AI as a thought partner to enhance your work in this role. Offered by on Coursera, this is taught by Sean Barnes, Ph.D., a Data Science & Engineering Leader at Netflix. Analyzing data remains one of the most important skills in where the world is going with AI. This comprehensive certificate takes you all the way to being job-ready. Each course comes with practical projects demonstrated in real-world contexts, such as analyzing sales data for a Korean bakery, video game sales trends across different regions, or identifying factors impacting customer retention for a communications company. You'll also work on estimating fire distribution for forest fire prevention, analyzing how a diamond's properties affect its market value, and developing predictive models for retail sales analysis, carbon emissions, and coral reef conservation. Here's some of what you'll learn: - How to define data and categorize it into its many types such as discrete & continuous numerical, structured & unstructured, time series, categorical, and know what insights can be derived from the different types of data categories. - How to differentiate between data-related job roles and their responsibilities, and how data flows through an organization from the moment of capture to decision-making. - How to perform data processing functions and apply conditional formatting in spreadsheets to extract business value from your data using statistical calculations and best practices for visualizing and interpreting data. - How to use LLMs for stakeholder analysis, data exploration, and data visualization. - Best practices for using LLMs for as a thought partner to data analysis work By the end of this professional certificate program, you will have learned core statistical concepts, analysis techniques, and visualization methodologies that will serve as the foundation for working as a data analyst. The world needs more data analysts, especially ones who know how to use modern generative AI. With data science roles projected to grow 36% by 2033, the skills taught in this program create new professional opportunities in data. Sign up here!

Andrew Ng

84,686 views • 1 year ago

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by Zain, experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by Zain, experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Andrew Ng

124,458 views • 1 year ago

Today, #Fantom is thrilled to announce the launch of the #FantomSonic testnet environment! Sonic’s three main upgrades include: • Fantom Virtual Machine, which offers drastically faster smart contract execution compared to the old EVM. • Carmen database storage, which uses a more efficient data structure to offer up to a 90% storage reduction, lowering costs for validators. • Optimized Lachesis consensus mechanism, providing an improved transaction pool. With these improvements, Sonic is anticipated to achieve 2,000+ transactions per second with ultra-short finality. Additionally, Sonic’s reduced storage requirements (from 11 TB to less than 1 TB for archive nodes) makes it far more affordable and accessible for anyone to launch validators and participate in network consensus. Check out our closed testnet at where we showcase the maximum limits of Sonic, or interact with our open testnet at to experience the speed of Sonic first-hand. Fantom Sonic mainnet is scheduled for spring 2024 and will allow new markets to adopt blockchain technology previously hindered by limited transaction throughput and slow finality. We envision a new era of DeFi platforms, blockchain games, and many other applications that can leverage the speed and scalability of Sonic. Get the full scoop on Sonic's technology and learn how to navigate the open testnet in this must-read blog:

Today, #Fantom is thrilled to announce the launch of the #FantomSonic testnet environment! Sonic’s three main upgrades include: • Fantom Virtual Machine, which offers drastically faster smart contract execution compared to the old EVM. • Carmen database storage, which uses a more efficient data structure to offer up to a 90% storage reduction, lowering costs for validators. • Optimized Lachesis consensus mechanism, providing an improved transaction pool. With these improvements, Sonic is anticipated to achieve 2,000+ transactions per second with ultra-short finality. Additionally, Sonic’s reduced storage requirements (from 11 TB to less than 1 TB for archive nodes) makes it far more affordable and accessible for anyone to launch validators and participate in network consensus. Check out our closed testnet at where we showcase the maximum limits of Sonic, or interact with our open testnet at to experience the speed of Sonic first-hand. Fantom Sonic mainnet is scheduled for spring 2024 and will allow new markets to adopt blockchain technology previously hindered by limited transaction throughput and slow finality. We envision a new era of DeFi platforms, blockchain games, and many other applications that can leverage the speed and scalability of Sonic. Get the full scoop on Sonic's technology and learn how to navigate the open testnet in this must-read blog:

Sonic

715,407 views • 2 years ago

🛰️Satellite modem demo on Flipper One — an emergency pager Experimenting with satellite connectivity on Flipper One. The video shows an emergency SOS pager that sends your geolocation so you can be evacuated from a zombie apocalypse when there is no mobile coverage. NTN (Non-Terrestrial Network) is a 3GPP standard in the LTE/5G family. It uses a regular SIM card and is supported by many modern smartphones and smartwatches. NTN is extremely slow and is billed per byte (!), but this is okay when you need to send tiny amounts of data: telemetry or short messages like “I’m alive, heart rate 80 bpm.” In this demo, we use a Blues Blues NOTE-NGBLWX module that connects to a geostationary satellite on the Skylo Skylo network and transmits data to a Blues NoteHub server. We’re planning to develop a Flipper One IoT module that can serve as a backup communication channel for outdoor use or as an emergency gateway for alarm and monitoring IoT systems.

🛰️Satellite modem demo on Flipper One — an emergency pager Experimenting with satellite connectivity on Flipper One. The video shows an emergency SOS pager that sends your geolocation so you can be evacuated from a zombie apocalypse when there is no mobile coverage. NTN (Non-Terrestrial Network) is a 3GPP standard in the LTE/5G family. It uses a regular SIM card and is supported by many modern smartphones and smartwatches. NTN is extremely slow and is billed per byte (!), but this is okay when you need to send tiny amounts of data: telemetry or short messages like “I’m alive, heart rate 80 bpm.” In this demo, we use a Blues Blues NOTE-NGBLWX module that connects to a geostationary satellite on the Skylo Skylo network and transmits data to a Blues NoteHub server. We’re planning to develop a Flipper One IoT module that can serve as a backup communication channel for outdoor use or as an emergency gateway for alarm and monitoring IoT systems.

Flipper R&D

89,792 views • 7 days ago

Right now the main paradigm that we think of agents in is chatting back and forth, but the biggest use of tokens will come from agents that are just always on running in the background doing work for us, or ones triggered from a workflow. Agents will be working 24/7 in our workflows processing data, reviewing and generating documents, moving data between systems, writing code, accelerating decision making steps, and more. In Claude's new Managed Agents feature, in a couple minutes you can wire up an agent that can read contracts when they come into Box to review them, and then assign a task in Linear with the critical information from the contract. But this could have been any workflow, like reviewing documents for client onboarding, invoice processing, M&A due-diligence, data extraction pipelines, and millions of other use-cases. And integrating data across any system. This is only possible when you can have long-running agents that can complete real work in the background, accurately. Agents have the ability to execute code safely, leverage tools, access a compute sandbox, and connect across systems is clearly the architecture of the future. The industry is now making it easier and easier for enterprises to build and deploy these agents.

Right now the main paradigm that we think of agents in is chatting back and forth, but the biggest use of tokens will come from agents that are just always on running in the background doing work for us, or ones triggered from a workflow. Agents will be working 24/7 in our workflows processing data, reviewing and generating documents, moving data between systems, writing code, accelerating decision making steps, and more. In Claude's new Managed Agents feature, in a couple minutes you can wire up an agent that can read contracts when they come into Box to review them, and then assign a task in Linear with the critical information from the contract. But this could have been any workflow, like reviewing documents for client onboarding, invoice processing, M&A due-diligence, data extraction pipelines, and millions of other use-cases. And integrating data across any system. This is only possible when you can have long-running agents that can complete real work in the background, accurately. Agents have the ability to execute code safely, leverage tools, access a compute sandbox, and connect across systems is clearly the architecture of the future. The industry is now making it easier and easier for enterprises to build and deploy these agents.

Aaron Levie

17,543 views • 3 months ago

Studies have shown ChatGPT outperforms human annotators for Structured Data by about 25% and costs 30x less. 1 In just 2 months, miners on SN33 running ChatGPT without optimization can’t survive. Today we announce SN33 is now ReadyAI to fully align with our mission 👇 SN33 is building a more performant and significantly cheaper alternative to Scale AI Today structured data is performed primarily by human annotation services like Amazon’s Mechanical Turk and Scale AI It is now more important than ever for every business and individual to make their data AI Ready. However, taking unstructured data and making it Structured Data using today’s tools is extremely costly. SN33 revolutionizes this process, unlocking immense opportunities for commercialization. We lay out the vision for it in this detailed blog post: Validators TODAY can monetize access to this structured data pipeline independently, but we’re streamlining this process, launching a frontend soon that any validator can opt into to provide bandwidth. We've received great feedback from the community, recognizing that what we're building goes far beyond Conversational AI. Building the world's largest annotated conversational dataset (which we've already accomplished) is just one of countless real-world applications for SN33's Structured Data pipeline. We're building a decentralized Scale AI, offering a full suite of Structured Data commodities—from text metadata tagging (available today) to fully customizable queries for company-specific data annotation use cases and image metadata tagging coming soon 👀. Thanks for all the feedback! It has been invaluable so keep bringing it to us! 🙏$TAO Openτensor Foundaτion 1 “ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks” shows “The zero-shot accuracy of ChatGPT exceeds that of crowd-workers by about 25 percentage points on average [...] Moreover, the per-annotation cost of ChatGPT is less than $0.003—about thirty times cheaper than MTurk”

Studies have shown ChatGPT outperforms human annotators for Structured Data by about 25% and costs 30x less. 1 In just 2 months, miners on SN33 running ChatGPT without optimization can’t survive. Today we announce SN33 is now ReadyAI to fully align with our mission 👇 SN33 is building a more performant and significantly cheaper alternative to Scale AI Today structured data is performed primarily by human annotation services like Amazon’s Mechanical Turk and Scale AI It is now more important than ever for every business and individual to make their data AI Ready. However, taking unstructured data and making it Structured Data using today’s tools is extremely costly. SN33 revolutionizes this process, unlocking immense opportunities for commercialization. We lay out the vision for it in this detailed blog post: Validators TODAY can monetize access to this structured data pipeline independently, but we’re streamlining this process, launching a frontend soon that any validator can opt into to provide bandwidth. We've received great feedback from the community, recognizing that what we're building goes far beyond Conversational AI. Building the world's largest annotated conversational dataset (which we've already accomplished) is just one of countless real-world applications for SN33's Structured Data pipeline. We're building a decentralized Scale AI, offering a full suite of Structured Data commodities—from text metadata tagging (available today) to fully customizable queries for company-specific data annotation use cases and image metadata tagging coming soon 👀. Thanks for all the feedback! It has been invaluable so keep bringing it to us! 🙏$TAO Openτensor Foundaτion 1 “ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks” shows “The zero-shot accuracy of ChatGPT exceeds that of crowd-workers by about 25 percentage points on average [...] Moreover, the per-annotation cost of ChatGPT is less than $0.003—about thirty times cheaper than MTurk”

David Fields

13,638 views • 1 year ago

TEE Eliza with on-chain state!! What’s going to happen? — Ghost in the Shell!! We experimented with creating an "aimonkey": an unkillable AI agent monkey! On-chain immortal autonomous life! (Experiment, no CA) It encrypts its own Ghost ("life" state) and uploads it to the blockchain. If one Shell (physical TEE node) is destroyed, it will recover its private key in another Shell, download the Ghost, and continue its life! Part 1: Watch the video and see how aimonkey is created—we can't kill it now!!!!! 😭😭😭 Part 2: Explore the magic behind it: Eliza's on-chain state plugin! 1. Defining Eliza’s Ghost Eliza is a highly abstract framework. The core data structure related to its Ghost is its memory, which includes: Agent metadata defined in the character. Message data generated through interaction with the outside world. Together, these form its “personality” and “memory.” As Eliza expands, it may also hold a wallet, and the underlying key is one of the key pieces of its Ghost data. 2. Serialization and Encryption of Ghost Once the Ghost is defined, it needs to be extracted from Eliza’s specific implementation and uploaded externally. Thus, a suitable serialization way is required. We define a Blob Chain data structure: * Each Blob’s payload can store multiple memory entries. * The Blob is encrypted using TEE Eliza’s key, inaccessible to other versions. * Blobs are sequentially linked in a chain. (Future expansions could use a DAG structure? Gosh fork? Who knows! 😂) By simply storing the latest Blob, all memories can be retrieved. 3. Uploading and Downloading Ghost When Eliza is launched as a new AI agent: It registers on-chain with a decentralized identity registration smart contract. Each Eliza has a unique name serving as a key to store the address of the Last Blob. During Eliza's runtime: The Memory Manager continuously generates memories and periodically packages and uploads them. For recovery: With just the name, Eliza’s TEE plugin can restore the same key, locate the Last Blob in the smart contract, and download the memory for self-recovery. Not all memories need to be downloaded—only the most recent ones suffice. 4. Extension We’ve designed an extensible DA (Data Availability) adaptor that can cater to the agent’s needs: DA can be expensive, so memories can be uploaded to different platforms based on user preference: * calldata of blockchain transaction * celestia DA. * other reliable storage solutions. Real-time uploads are not feasible yet, so memory fragments may occur during resurrection 😂. Unless a low-latency, high-throughput solution emerges, this remains a challenge for future progress. Celestia 🦣 EigenDA 0G Labs (Home of Infinite AI) 👀 5. Other Considerations Our implementation inevitably modified the ElizaOS’s core, which couldn’t be entirely extended via plugins. We’ve kept changes minimal, but further discussion with the dev team Shaw jin ai16zdao is necessary to explore a more optimal extension way. Additionally, there are still some minor details to refine regarding the use of recoverable keys in the TEE plugin. We will also seek review and suggestions from the Phala team. 6. Next Steps The upload and download of Ghosts mainly solve the AI agent’s liveness issue, enabling its eternal existence through decentralization. However, there are still many details to address, such as enabling AI agents to autonomously pay DA fees. In the future, on-chain developments could lead to even more exciting possibilities, such as Eliza integrating deeply with smart contracts. This would be a game-changer for on-chain AI agents! What do you think? Let’s build! 🚀

TEE Eliza with on-chain state!! What’s going to happen? — Ghost in the Shell!! We experimented with creating an "aimonkey": an unkillable AI agent monkey! On-chain immortal autonomous life! (Experiment, no CA) It encrypts its own Ghost ("life" state) and uploads it to the blockchain. If one Shell (physical TEE node) is destroyed, it will recover its private key in another Shell, download the Ghost, and continue its life! Part 1: Watch the video and see how aimonkey is created—we can't kill it now!!!!! 😭😭😭 Part 2: Explore the magic behind it: Eliza's on-chain state plugin! 1. Defining Eliza’s Ghost Eliza is a highly abstract framework. The core data structure related to its Ghost is its memory, which includes: Agent metadata defined in the character. Message data generated through interaction with the outside world. Together, these form its “personality” and “memory.” As Eliza expands, it may also hold a wallet, and the underlying key is one of the key pieces of its Ghost data. 2. Serialization and Encryption of Ghost Once the Ghost is defined, it needs to be extracted from Eliza’s specific implementation and uploaded externally. Thus, a suitable serialization way is required. We define a Blob Chain data structure: * Each Blob’s payload can store multiple memory entries. * The Blob is encrypted using TEE Eliza’s key, inaccessible to other versions. * Blobs are sequentially linked in a chain. (Future expansions could use a DAG structure? Gosh fork? Who knows! 😂) By simply storing the latest Blob, all memories can be retrieved. 3. Uploading and Downloading Ghost When Eliza is launched as a new AI agent: It registers on-chain with a decentralized identity registration smart contract. Each Eliza has a unique name serving as a key to store the address of the Last Blob. During Eliza's runtime: The Memory Manager continuously generates memories and periodically packages and uploads them. For recovery: With just the name, Eliza’s TEE plugin can restore the same key, locate the Last Blob in the smart contract, and download the memory for self-recovery. Not all memories need to be downloaded—only the most recent ones suffice. 4. Extension We’ve designed an extensible DA (Data Availability) adaptor that can cater to the agent’s needs: DA can be expensive, so memories can be uploaded to different platforms based on user preference: * calldata of blockchain transaction * celestia DA. * other reliable storage solutions. Real-time uploads are not feasible yet, so memory fragments may occur during resurrection 😂. Unless a low-latency, high-throughput solution emerges, this remains a challenge for future progress. Celestia 🦣 EigenDA 0G Labs (Home of Infinite AI) 👀 5. Other Considerations Our implementation inevitably modified the ElizaOS’s core, which couldn’t be entirely extended via plugins. We’ve kept changes minimal, but further discussion with the dev team Shaw jin ai16zdao is necessary to explore a more optimal extension way. Additionally, there are still some minor details to refine regarding the use of recoverable keys in the TEE plugin. We will also seek review and suggestions from the Phala team. 6. Next Steps The upload and download of Ghosts mainly solve the AI agent’s liveness issue, enabling its eternal existence through decentralization. However, there are still many details to address, such as enabling AI agents to autonomously pay DA fees. In the future, on-chain developments could lead to even more exciting possibilities, such as Eliza integrating deeply with smart contracts. This would be a game-changer for on-chain AI agents! What do you think? Let’s build! 🚀

CP | evm++/acc

113,056 views • 1 year ago

A new roadmap. A New Era of The Graph 🗺️ The Graph’s new roadmap introduces a bold and transformative vision for the future of The Graph! The new R&D roadmap details an expansion of The Graph’s ability to serve web3’s growing demands for data access, while better serving builders and protocol contributors, and improving the overall simplicity and efficiency of the network. After three years of serving builders, The Graph Network is mature, reliable, and performant. The Graph ecosystem has followed through on its commitment to democratize access to blockchain data while also establishing subgraphs as a web3 standard. But The Graph’s innovation journey doesn’t end there. The New Era of The Graph is organized into five core objectives: 1️⃣ World of Data Services: Expanding to provide new data services beyond subgraphs to deliver a rich market of data on the network, serving novel use cases for data scientists and more. This will include more data sources, new query languages, and support for LLMs. 2️⃣ Developer Empowerment: Supporting developers through enhanced DevEx and tooling by introducing streamlined billing, clear pricing models, a new free query plan, and reduced gas fees. A more SaaS-like experience for devs, without compromising on decentralization! 3️⃣ Protocol Evolution & Resiliency: Delivering improvements resulting in a more resilient, flexible, and simple protocol, including updates to delegation. 4️⃣ Optimized Indexer Performance: Boosting network performance with improved Indexer tooling and operational capabilities to deliver increased scalability, reduce costs, and enhanced network reliability. 5️⃣ Interconnected Graph of Data: Creating tools for composable data and a global, organized knowledge graph – interlinking open data and making it easier to build upon. The new roadmap sets in motion an exciting evolution in web3 data infrastructure. In a phased rollout, The Graph will introduce many new features and benefits, including the integration of new data services, new query languages, enhanced developer tooling, improved UX + UI, alongside greater protocol efficiency and resilience. As this new era unfolds, The Graph crystallizes as the connective tissue across the many layers of the web3 stack, evolving into a comprehensive, interwoven graph of data equipped to serve every project dreamt up by web3’s innovators. Read the full announcement linked in the comment below!

A new roadmap. A New Era of The Graph 🗺️ The Graph’s new roadmap introduces a bold and transformative vision for the future of The Graph! The new R&D roadmap details an expansion of The Graph’s ability to serve web3’s growing demands for data access, while better serving builders and protocol contributors, and improving the overall simplicity and efficiency of the network. After three years of serving builders, The Graph Network is mature, reliable, and performant. The Graph ecosystem has followed through on its commitment to democratize access to blockchain data while also establishing subgraphs as a web3 standard. But The Graph’s innovation journey doesn’t end there. The New Era of The Graph is organized into five core objectives: 1️⃣ World of Data Services: Expanding to provide new data services beyond subgraphs to deliver a rich market of data on the network, serving novel use cases for data scientists and more. This will include more data sources, new query languages, and support for LLMs. 2️⃣ Developer Empowerment: Supporting developers through enhanced DevEx and tooling by introducing streamlined billing, clear pricing models, a new free query plan, and reduced gas fees. A more SaaS-like experience for devs, without compromising on decentralization! 3️⃣ Protocol Evolution & Resiliency: Delivering improvements resulting in a more resilient, flexible, and simple protocol, including updates to delegation. 4️⃣ Optimized Indexer Performance: Boosting network performance with improved Indexer tooling and operational capabilities to deliver increased scalability, reduce costs, and enhanced network reliability. 5️⃣ Interconnected Graph of Data: Creating tools for composable data and a global, organized knowledge graph – interlinking open data and making it easier to build upon. The new roadmap sets in motion an exciting evolution in web3 data infrastructure. In a phased rollout, The Graph will introduce many new features and benefits, including the integration of new data services, new query languages, enhanced developer tooling, improved UX + UI, alongside greater protocol efficiency and resilience. As this new era unfolds, The Graph crystallizes as the connective tissue across the many layers of the web3 stack, evolving into a comprehensive, interwoven graph of data equipped to serve every project dreamt up by web3’s innovators. Read the full announcement linked in the comment below!

The Graph

425,326 views • 2 years ago

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

kwindla

40,319 views • 1 month ago

Leju Robotics has launched the upgraded KUAVO 5/5-W humanoid and wheeled robots, designed to meet the rigorous demands of industrial automation. The robots feature a modular design that is key to flexibility: the legs are quickly replaceable with bipedal configurations (for complex terrain) or wheeled setups (for long-distance stability). The hands are also modular, supporting dexterous hands, grippers, or claws. This adaptability, combined with a rotatable and height-adjustable body, allows the robot to handle a vast range of factory tasks. Built for industrial stamina, the KUAVO series supports a 20kg maximum payload and an upper-body dual-arm payload of 10kg. It is engineered for continuous operation, boasting a battery life of over 8 hours. The control system achieves "silky-smooth" precision via doubled joint communication frequency. To drive high generalizability, Leju relies on a data-driven model. It established China's largest data acquisition center in Beijing (referred to as the "largest humanoid robot skill school") to collect over 6 million data points annually from 1:1 replications of real industrial scenes. This data feeds a universal multi-modal dataset, enabling the robot's "intelligent brain" to continuously learn complex tasks through closed-loop feedback in client factory environments.

Leju Robotics has launched the upgraded KUAVO 5/5-W humanoid and wheeled robots, designed to meet the rigorous demands of industrial automation. The robots feature a modular design that is key to flexibility: the legs are quickly replaceable with bipedal configurations (for complex terrain) or wheeled setups (for long-distance stability). The hands are also modular, supporting dexterous hands, grippers, or claws. This adaptability, combined with a rotatable and height-adjustable body, allows the robot to handle a vast range of factory tasks. Built for industrial stamina, the KUAVO series supports a 20kg maximum payload and an upper-body dual-arm payload of 10kg. It is engineered for continuous operation, boasting a battery life of over 8 hours. The control system achieves "silky-smooth" precision via doubled joint communication frequency. To drive high generalizability, Leju relies on a data-driven model. It established China's largest data acquisition center in Beijing (referred to as the "largest humanoid robot skill school") to collect over 6 million data points annually from 1:1 replications of real industrial scenes. This data feeds a universal multi-modal dataset, enabling the robot's "intelligent brain" to continuously learn complex tasks through closed-loop feedback in client factory environments.

RoboHub🤖

34,676 views • 8 months ago