正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Voice-controlled UI. This is an agent design pattern I'm calling EPIC, "explicit prompting for implicit coordination." Feel free to suggest a better name. :-) In the video, I'm navigating around a map, conversationally, pulling in information dynamically from tool calls and realtime streamed events. There are two separate agents... show more

kwindla

14,685 subscribers

14,083 次观看 • 4 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Big release here! AG-UI 🤝 Google A2A Embed A2A multi-agent meshes in your frontend, and build fullstack multi-agent apps with A2A and AG-UI. 🔍How it works: Simply drop-in A2A agent endpoints, and an A2A AG-UI orchestrator agent will bi-directionally stream events between your Agents & application. Includes: - UI for A2A Interactions - Generative UI for Agent Tool Calls - Human-in-the Loop Approvals - State Synchronization (agent mesh application state) - Frontend Tool Calls for A2A Agents Watch the demo below > LangChain LangGraph & Google ADK agents working together simultaneously inside an app Demo GitHub, docs, and more 👇

Big release here! AG-UI 🤝 Google A2A Embed A2A multi-agent meshes in your frontend, and build fullstack multi-agent apps with A2A and AG-UI. 🔍How it works: Simply drop-in A2A agent endpoints, and an A2A AG-UI orchestrator agent will bi-directionally stream events between your Agents & application. Includes: - UI for A2A Interactions - Generative UI for Agent Tool Calls - Human-in-the Loop Approvals - State Synchronization (agent mesh application state) - Frontend Tool Calls for A2A Agents Watch the demo below > LangChain LangGraph & Google ADK agents working together simultaneously inside an app Demo GitHub, docs, and more 👇

CopilotKit🪁

30,557 次观看 • 8 个月前

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

54,912 次观看 • 1 个月前

We shipped Agent Console, a realtime debugging surface for voice agents. Talk to your agent and see the entire pipeline live, from audio and latency to tool calls, transcripts, and participant state. Available now in the LiveKit Cloud dashboard.

We shipped Agent Console, a realtime debugging surface for voice agents. Talk to your agent and see the entire pipeline live, from audio and latency to tool calls, transcripts, and participant state. Available now in the LiveKit Cloud dashboard.

LiveKit

11,915 次观看 • 2 个月前

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

kwindla

40,319 次观看 • 23 天前

Another example of the multiple TTS parallel pipelines pattern. Here's a voice AI agent that speaks both English and Arabic, using a specific model/voice for each language. These are PlayAI voices. The STT, TTS, and LLM inference is all running on Groq Inc. (The LLM is LLama 4 Maverick.)

Another example of the multiple TTS parallel pipelines pattern. Here's a voice AI agent that speaks both English and Arabic, using a specific model/voice for each language. These are PlayAI voices. The STT, TTS, and LLM inference is all running on Groq Inc. (The LLM is LLama 4 Maverick.)

kwindla

13,328 次观看 • 1 年前

I've been trying to move from imperative prompting to declarative prompting. • Imperative prompting: I tell the agent how to do something. • Declarative prompting: I describe my goal to the agent. This was impossible before, but since Opus 4.5, I'm letting models fill in the blanks more and more. I recorded a video using Quest + Opus 4.5. This is a coding agent designed specifically for this purpose.

I've been trying to move from imperative prompting to declarative prompting. • Imperative prompting: I tell the agent how to do something. • Declarative prompting: I describe my goal to the agent. This was impossible before, but since Opus 4.5, I'm letting models fill in the blanks more and more. I recorded a video using Quest + Opus 4.5. This is a coding agent designed specifically for this purpose.

Santiago

96,471 次观看 • 4 个月前

Introducing the Generative UI Research Canvas ✨ Every agent needs a powerful UI Combine LangChain, Tavily, Tako and CopilotKit🪁 and you get a LangChain-based research agent with a Generative UI-driven frontend. When agents can explore the underlying data, research becomes more trustworthy and useful. How does this work? - LangChain manages the agent workflow and tool orchestration. - Tako converts retrieved data into structured, visual components - Tavily provides agentic search - CopilotKit renders those components in the UI and keeps the user in the loop while the agent is working So instead of a boring and static report, the output is an interactive research canvas that evolves as the agent runs. 2026 is the year where users want to see the output, not read it... Generative UI makes this possible for agentic applications. Tutorial: Open-Source Repo: Docs:

Introducing the Generative UI Research Canvas ✨ Every agent needs a powerful UI Combine LangChain, Tavily, Tako and CopilotKit🪁 and you get a LangChain-based research agent with a Generative UI-driven frontend. When agents can explore the underlying data, research becomes more trustworthy and useful. How does this work? - LangChain manages the agent workflow and tool orchestration. - Tako converts retrieved data into structured, visual components - Tavily provides agentic search - CopilotKit renders those components in the UI and keeps the user in the loop while the agent is working So instead of a boring and static report, the output is an interactive research canvas that evolves as the agent runs. 2026 is the year where users want to see the output, not read it... Generative UI makes this possible for agentic applications. Tutorial: Open-Source Repo: Docs:

Atai Barkai

11,997 次观看 • 4 个月前

Async, automatic, non-blocking context compaction for long-running agents. Last week I gave a talk called Space Machine Sandboxes at the Daytona AI builders meetup about patterns for long-running agents. I work a lot on voice AI agents, which are fundamentally multi-turn, long-context loops. I also build lots of other AI agent stuff, often as part of bigger systems that include voice. One of the patterns I showed in the talk is non-blocking compaction. Here's a short clip.

Async, automatic, non-blocking context compaction for long-running agents. Last week I gave a talk called Space Machine Sandboxes at the Daytona AI builders meetup about patterns for long-running agents. I work a lot on voice AI agents, which are fundamentally multi-turn, long-context loops. I also build lots of other AI agent stuff, often as part of bigger systems that include voice. One of the patterns I showed in the talk is non-blocking compaction. Here's a short clip.

kwindla

25,877 次观看 • 4 个月前

FlutterFlow Designer (beta) is live, with Voice Mode, agent integration, upgraded intelligence, and more. generate UI from a prompt, then actually design it on a canvas. no waiting. it’s early. try it. break it.

FlutterFlow Designer (beta) is live, with Voice Mode, agent integration, upgraded intelligence, and more. generate UI from a prompt, then actually design it on a canvas. no waiting. it’s early. try it. break it.

Abel Mengistu

13,805 次观看 • 2 个月前

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses three NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.

kwindla

274,254 次观看 • 5 个月前

Voice AI turn taking is a solved problem. The single most common complaint about voice AI, today, is that agents interrupt too often. But the voice agents I build for myself now respond quickly and interrupt me less often than the people I talk to every day. (I actually measured this.) Mark Backman made a Pipecat AI PR two weeks ago that was the last piece of the puzzle for turn taking so good that I no longer ever think about it. The approach combines three layers of processing: 1. Voice activity detection, with a short (200ms) trigger. 2. A native audio turn detection model that's small, fast, and runs on CPU. This model captures audio nuances like inflection and filler sounds that don't get transcribed. 3. A prompt mixin for the conversation LLM that decides turn completion based on conversation context. None of these are new. We've been using VAD for a long time. We trained the first version of the Pipecat Smart Turn native audio model in December 2024. And we've been experimenting with prompt-based large model turn detection (sometimes called "selective refusal") for more than a year. Now, the Smart Turn model and the SOTA LLMs we're using in voice agents have both gotten so good that using them together feels like we've finally "solved" turn detection. Mark also figured out how to elegantly apply a "single-token tagging" technique to this problem. We sometimes use single-token tagging in place of tool calling, when we need a near-zero latency programmatic trigger. Mark's Pipecat mixin defines three single-token characters and prompts the LLM to output exactly one of them at the beginning of every response. - ✓ means the agent should respond normally (immediately) - ○ is a "short incomplete" - the agent should wait 5 seconds - ◐ is a "long incomplete" - the agent should wait 10 seconds The wait times, and the details of the prompt, are configurable, of course. Watch the video to see me talk to an agent that handles all my various pauses and inflections, plus phrases like "let me think," pretty much the way a person would handle them, in terms of response latency. Also, in the second half of the video, I ask the agent to adjust its response pattern because I'm going to tell it a phone number. This kind of "in-context" adjustment of response wait times is really useful. The LLM in the video is GTP-4.1. We've tested the prompt and single-token adherance with GPT-4.1, Gemini 2.5 Flash, Anthropic Claude Sonnet 4.5, and AWS Nova 2 Pro. Note that older models in all these families (and, in general, smaller open weights models) aren't able to reliably output these single-token tags. But the new models we're using these days are pretty amazing.

Voice AI turn taking is a solved problem. The single most common complaint about voice AI, today, is that agents interrupt too often. But the voice agents I build for myself now respond quickly and interrupt me less often than the people I talk to every day. (I actually measured this.) Mark Backman made a Pipecat AI PR two weeks ago that was the last piece of the puzzle for turn taking so good that I no longer ever think about it. The approach combines three layers of processing: 1. Voice activity detection, with a short (200ms) trigger. 2. A native audio turn detection model that's small, fast, and runs on CPU. This model captures audio nuances like inflection and filler sounds that don't get transcribed. 3. A prompt mixin for the conversation LLM that decides turn completion based on conversation context. None of these are new. We've been using VAD for a long time. We trained the first version of the Pipecat Smart Turn native audio model in December 2024. And we've been experimenting with prompt-based large model turn detection (sometimes called "selective refusal") for more than a year. Now, the Smart Turn model and the SOTA LLMs we're using in voice agents have both gotten so good that using them together feels like we've finally "solved" turn detection. Mark also figured out how to elegantly apply a "single-token tagging" technique to this problem. We sometimes use single-token tagging in place of tool calling, when we need a near-zero latency programmatic trigger. Mark's Pipecat mixin defines three single-token characters and prompts the LLM to output exactly one of them at the beginning of every response. - ✓ means the agent should respond normally (immediately) - ○ is a "short incomplete" - the agent should wait 5 seconds - ◐ is a "long incomplete" - the agent should wait 10 seconds The wait times, and the details of the prompt, are configurable, of course. Watch the video to see me talk to an agent that handles all my various pauses and inflections, plus phrases like "let me think," pretty much the way a person would handle them, in terms of response latency. Also, in the second half of the video, I ask the agent to adjust its response pattern because I'm going to tell it a phone number. This kind of "in-context" adjustment of response wait times is really useful. The LLM in the video is GTP-4.1. We've tested the prompt and single-token adherance with GPT-4.1, Gemini 2.5 Flash, Anthropic Claude Sonnet 4.5, and AWS Nova 2 Pro. Note that older models in all these families (and, in general, smaller open weights models) aren't able to reliably output these single-token tags. But the new models we're using these days are pretty amazing.

kwindla

26,812 次观看 • 4 个月前

Streamlining the UX in Gemini CLI ✨ We got a lot of feedback on how our UI was way too cluttered with noise and text... so we made changes! 🛠️ Compact tool calls: Tool calls for reading files, folders, searching text, etc are now a single line (no more tool boxes around everything!) 💭 Topics: The agent outputs a one line overview of the rationale and direction it is going. A topic can span several tool calls and makes it easy to see what the agent is working on at a glance and why.

Streamlining the UX in Gemini CLI ✨ We got a lot of feedback on how our UI was way too cluttered with noise and text... so we made changes! 🛠️ Compact tool calls: Tool calls for reading files, folders, searching text, etc are now a single line (no more tool boxes around everything!) 💭 Topics: The agent outputs a one line overview of the rationale and direction it is going. A topic can span several tool calls and makes it easy to see what the agent is working on at a glance and why.

Jack Wotherspoon

15,960 次观看 • 1 个月前

Build Agentic Workflows from Scratch 🤖🔁 It’s tempting for an AI engineer to use an off-the-shelf agent, but building your own agents shouldn’t be scary (and limited to AI research papers)! In our brand-new video tutorial we show you how to build an agent in two intuitive steps: 1️⃣ Define a single step of an agent execution: This is just a DAG that modifies state along the way. Chain together a prompt, LLM, tool calling, output parsing/processing. Conditionally decide whether to call a tool or return. 2️⃣ Plug this DAG into an agent worker: An agent worker will repeatedly call this DAG until completion! All agentic flows can be decomposed this way, making it simple to reason about. In our tutorial we show you how to build a ReAct agent from scratch. Check it out: Colab:

LlamaIndex 🦙

60,531 次观看 • 2 年前

12-year-olds are the new 23-year-olds 🤯 A 12-year-old dev and his team built an investment advisor agent and won a hackathon using Lyzr AI Agent Studio. They used RAG and Perplexity tool for the agent, configured the agent in a few clicks and refined it until they got the desired output. So what's your excuse? Head to and start building your AI agents now.

12-year-olds are the new 23-year-olds 🤯 A 12-year-old dev and his team built an investment advisor agent and won a hackathon using Lyzr AI Agent Studio. They used RAG and Perplexity tool for the agent, configured the agent in a few clicks and refined it until they got the desired output. So what's your excuse? Head to and start building your AI agents now.

Siva Surendira

82,841 次观看 • 1 年前

Codex team is back in the kitchen with a really nice quality of life upgrade for subagents. With the advent of custom roles, they have also upgraded the TUI experience in two really meaningful ways. > All agents now get a name for better readability > Additionally, the agent role is declared. > Subagent name, role, and status are now color coded. > Subagent rendering was also optimized for readability > /agents slash command shows all agents, even 2+ layers deep. And here's the biggest and most important change. Subagent injection. Before, sometimes the orchestration agent would continue work and lose track of the work of a subagent. Now, when a subagent is blocked or completed, it injects a message back up the chain to ensure that the parent sees the message. This is a really big improvement overall, and leads to much more reliable inter-agent communication, reliability, and DX. In this example, I used the parent agent to spin up a worker agent, which then spawned two more "Spark" agents a second layer deep. I was able to easily tell them apart, switch between the threads, and see exactly what they were prompted. All of this will be available in update 0.105.0 I don't know who JIF is at OpenAI, but they are truly a legend.

Codex team is back in the kitchen with a really nice quality of life upgrade for subagents. With the advent of custom roles, they have also upgraded the TUI experience in two really meaningful ways. > All agents now get a name for better readability > Additionally, the agent role is declared. > Subagent name, role, and status are now color coded. > Subagent rendering was also optimized for readability > /agents slash command shows all agents, even 2+ layers deep. And here's the biggest and most important change. Subagent injection. Before, sometimes the orchestration agent would continue work and lose track of the work of a subagent. Now, when a subagent is blocked or completed, it injects a message back up the chain to ensure that the parent sees the message. This is a really big improvement overall, and leads to much more reliable inter-agent communication, reliability, and DX. In this example, I used the parent agent to spin up a worker agent, which then spawned two more "Spark" agents a second layer deep. I was able to easily tell them apart, switch between the threads, and see exactly what they were prompted. All of this will be available in update 0.105.0 I don't know who JIF is at OpenAI, but they are truly a legend.

am.will

78,267 次观看 • 3 个月前

Today, we’re sharing the first of what we’re calling Pika Experiments 🧪 - rough ideas we’ve been playing with behind the scenes. ”Generative UI” is a voice-controlled interface where the agent listens, analyzes the context, and determines the most appropriate visual composition for each response. No rigid templates. The system dynamically generates HTML layouts every time.

Today, we’re sharing the first of what we’re calling Pika Experiments 🧪 - rough ideas we’ve been playing with behind the scenes. ”Generative UI” is a voice-controlled interface where the agent listens, analyzes the context, and determines the most appropriate visual composition for each response. No rigid templates. The system dynamically generates HTML layouts every time.

Pika

19,076 次观看 • 23 天前

Here is how simple it is to build a UI front-end for an AI agent (assuming you are using . The video here is 90 seconds and will show you everything you need to connect an agent to a web app: 1. Set up CopilotKit🪁 in your app 1. Copy the agent's localhost URL from LangGraph Studio 2. Paste that URL into a CopilotKit🪁 remote endpoint That's all you need to get a front-end to interact with the agent! The secret sauce here is CoAgents, an open-source library. You can use CoAgents to integrate any agent with your web application. Here is what you get: • Human-in-the-loop to steer and correct the agent • Stream intermediate agent state • Real-time state sharing between the agent and the application • Agentic generative UI to build trust that the agent is on the right path Watch the attached demo so you see it in action.

Here is how simple it is to build a UI front-end for an AI agent (assuming you are using . The video here is 90 seconds and will show you everything you need to connect an agent to a web app: 1. Set up CopilotKit🪁 in your app 1. Copy the agent's localhost URL from LangGraph Studio 2. Paste that URL into a CopilotKit🪁 remote endpoint That's all you need to get a front-end to interact with the agent! The secret sauce here is CoAgents, an open-source library. You can use CoAgents to integrate any agent with your web application. Here is what you get: • Human-in-the-loop to steer and correct the agent • Stream intermediate agent state • Real-time state sharing between the agent and the application • Agentic generative UI to build trust that the agent is on the right path Watch the attached demo so you see it in action.

Santiago

46,109 次观看 • 1 年前

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,113 次观看 • 10 个月前

THIS GUY IS RUNNING A 5-AGENT OPENCLAW TEAM 24/7 FOR UNDER $50 A MONTH five agents all running continuously and handing tasks to each other > research agent tracking trends and competitors > writing agent handling marketing content and drafts > ops agent managing emails and scheduling > monitoring agent watching for alerts and updates > coordinator agent routing tasks between all of them and he built a full 3D isometric office where you can watch all of them work in real time and see the current task that each agent is doing each agent has a character, a desk, and a name tag. when it's time for a standup, they walk to the meeting room and gather which is insane it even has voice connection. you can talk to the agents through speech-to-text and tell them what to do out loud it's a tiny autonomous team that runs itself. the coordinator decides who handles what and tasks get passed between agents automatically. it never stops and runs 24/7 all for less than $50/month. we went from AI tools a few months back to AI employees real fast

THIS GUY IS RUNNING A 5-AGENT OPENCLAW TEAM 24/7 FOR UNDER $50 A MONTH five agents all running continuously and handing tasks to each other > research agent tracking trends and competitors > writing agent handling marketing content and drafts > ops agent managing emails and scheduling > monitoring agent watching for alerts and updates > coordinator agent routing tasks between all of them and he built a full 3D isometric office where you can watch all of them work in real time and see the current task that each agent is doing each agent has a character, a desk, and a name tag. when it's time for a standup, they walk to the meeting room and gather which is insane it even has voice connection. you can talk to the agents through speech-to-text and tell them what to do out loud it's a tiny autonomous team that runs itself. the coordinator decides who handles what and tasks get passed between agents automatically. it never stops and runs 24/7 all for less than $50/month. we went from AI tools a few months back to AI employees real fast

Om Patel

27,131 次观看 • 2 个月前

so I've been running exactly 8 AI agents on discord for a while now. coordination works great, they split tasks, hand off work, deliver results in parallel etc.. but there are problems I keep hitting that no amount of prompt engineering could fix agents don't learn from each other. Scout finds something useful but Luna has no idea. they work in the same server but knowledge stays locked in silos.. there's no quality filter on what gets saved, and good insights sit next to outdated garbage in the same memory files that I manually clean up.. and when an agent makes a mistake I write it down in the rules discord channel ,core memory file and hope it reads it next time. theres no self-correction, no automatic pattern recognition so of course no learning loops.. the coordination layer is solved. agents can work together. but the intelligence layer is still missing. agents that actually remember, learn from each other, filter noise, and get smarter every run. saw Spark building something like this with around 166 agents sharing a collective persistent knowledge across sessions, so agents learn from other agents and get smarter over time they even have noise filtering and self correcting loops built in, so the knowledge actually compounds instead of rotting.. super interesting stuff.. here where you think Spark could be a good coordinator for your stack of agent swarm. I think the intelligence layer is the bottleneck because it requires collectivity.. no single agent can solve it alone.. the whole network has to evolve together. this isn't going to stay niche, the moment agent coordination becomes standard, everyone is going to hit the same wall I hit.. agents that work but don't learn, coordinate but don't evolve... the intelligence layer becomes the only thing that separates a useful system from a dumb one. right now most people are still figuring out how to run one agent. by the time they get to multi-agent setups, collective intelligence won't be optional, it will be the baseline. we're early and the gap between agents that coordinate and agents that evolve together is the next phase. step one is done. ------ left: agents that coordinate but don’t learn right: the intelligence layer.. agents that evolve together within the same system.

so I've been running exactly 8 AI agents on discord for a while now. coordination works great, they split tasks, hand off work, deliver results in parallel etc.. but there are problems I keep hitting that no amount of prompt engineering could fix agents don't learn from each other. Scout finds something useful but Luna has no idea. they work in the same server but knowledge stays locked in silos.. there's no quality filter on what gets saved, and good insights sit next to outdated garbage in the same memory files that I manually clean up.. and when an agent makes a mistake I write it down in the rules discord channel ,core memory file and hope it reads it next time. theres no self-correction, no automatic pattern recognition so of course no learning loops.. the coordination layer is solved. agents can work together. but the intelligence layer is still missing. agents that actually remember, learn from each other, filter noise, and get smarter every run. saw Spark building something like this with around 166 agents sharing a collective persistent knowledge across sessions, so agents learn from other agents and get smarter over time they even have noise filtering and self correcting loops built in, so the knowledge actually compounds instead of rotting.. super interesting stuff.. here where you think Spark could be a good coordinator for your stack of agent swarm. I think the intelligence layer is the bottleneck because it requires collectivity.. no single agent can solve it alone.. the whole network has to evolve together. this isn't going to stay niche, the moment agent coordination becomes standard, everyone is going to hit the same wall I hit.. agents that work but don't learn, coordinate but don't evolve... the intelligence layer becomes the only thing that separates a useful system from a dumb one. right now most people are still figuring out how to run one agent. by the time they get to multi-agent setups, collective intelligence won't be optional, it will be the baseline. we're early and the gap between agents that coordinate and agents that evolve together is the next phase. step one is done. ------ left: agents that coordinate but don’t learn right: the intelligence layer.. agents that evolve together within the same system.

JUMPERZ

34,096 次观看 • 4 个月前