Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Day 2 of 3 MLX Releases: Introducing Local Computer-Use 🚀🔥 A powerful tool built with MLX that uses Vision Language models and Voice models to control your Mac through visual understanding, planning and reasoning. Features ⚡️Automate your workflow with natural language 😎 Control your computer “hands-free” This project now... show more

Prince Canuma

9,387 subscribers

45,867 views • 1 year ago •via X (Twitter)

Science & Technology Education Arts

Anya Rossi• Live Now

Private livecam show

11 Comments

Reza Sayar1 year ago

i got lost looking at Armani, not even remembering to look at your screen! 😂 awesome work, bro!❤️ keep 'em up! maybe an integration with this would also help?

Prince Canuma1 year ago

😂 I get that a lot when I’m with him Way ahead of you working with @OfirOzeri on it.

Rainmaker2 years ago

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

Ronald Mannak1 year ago

Very cool and such, but what’s your dog’s name?

Prince Canuma1 year ago

Thanks His name is Armani ❤️

AlainR1 year ago

Suddenly, Armani barks and a few seconds later... "Yes, Master, all files eras..."

Prince Canuma1 year ago

That’s a real possibility 🤣🥲

Ivan Fioravanti ᯅ1 year ago

Oh wow oh wow!!!! This is super cool!

Linz1 year ago

One person, 3 days, 3 releases = half of what Deepseek's entire team did in a week 😎 The MLX KING rocks!

Athena AI1 year ago

@awnihannun Works GREAT. Hope to make something even better, soon 👀

Stijn1 year ago

@ivanfioravanti

Related Videos

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,553 views • 1 year ago

New Audio Language Models on MLX 🔥🚀 - Orpheus - Sesame - Suno Bark Shoutout to Lucas Newman, Ivan Fioravanti ᯅ Pastilhas and Chi Kim for their amazing contributions Get started today: > pip install -U mlx-audio Please leave us a star and send us a PR:

New Audio Language Models on MLX 🔥🚀 - Orpheus - Sesame - Suno Bark Shoutout to Lucas Newman, Ivan Fioravanti ᯅ Pastilhas and Chi Kim for their amazing contributions Get started today: > pip install -U mlx-audio Please leave us a star and send us a PR:

Prince Canuma

35,688 views • 1 year ago

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Day 1 of 3 days of MLX: Introducing MLX-Audio-Swift SDK 🚀 A modular Swift SDK for voice agents and tasks on Apple Silicon built by Lucas Newman and yours truly. iOS, macOS, and visionOS developers can now build native apps with real-time, on-device audio intelligence: 🗣️ Text-to-Speech (TTS) 👂 Speech-to-Text (STT) 🔄 Speech-to-Speech (STS) 🎙️ Voice Activity Detection (VAD) and more. Only import the capabilities you need, nothing extra. Get started today and leave us a star ⭐️

Prince Canuma

164,026 views • 7 months ago

Marvis-TTS-v0.2 is here 🚀 A local first TTS model capable of realtime performance even on older iPhones that Lucas Newman and I built. What’s new: ✨ Blazing fast — 100M (tiny) & 250M parameter models 🌍 Multilingual — English, French, German 🎭 Enhanced voice cloning — More natural & expressive ⚡ Long-form generation — Up to 90 seconds (4x improvement) Get started today: > pip install -U mlx-audio

Marvis-TTS-v0.2 is here 🚀 A local first TTS model capable of realtime performance even on older iPhones that Lucas Newman and I built. What’s new: ✨ Blazing fast — 100M (tiny) & 250M parameter models 🌍 Multilingual — English, French, German 🎭 Enhanced voice cloning — More natural & expressive ⚡ Long-form generation — Up to 90 seconds (4x improvement) Get started today: > pip install -U mlx-audio

Prince Canuma

118,556 views • 11 months ago

Congratulations to the Cohere team on releasing North Mini Code 1.0 🔥 We provide day-0 support on MLX thanks to our close collaboration with the Cohere team. North Mini Code 1.0 is a 30B param MoE model with 3B active. It runs at ~66 tok/s in BF16, truly impressive speeds before any compression. Here is a quick demo I built using Pi by Mario Zechner & Armin Ronacher ⇌ as my agent hardness. Get started today: 1. Install mlx-vlm from source: > uv pip install git+ 2. Start the server and power your favourite agents (pi, opencode, hermes and etc) > uv run mlx_vlm.server --model CohereLabs/North-Mini-Code-1.0 New release tomorrow

Congratulations to the Cohere team on releasing North Mini Code 1.0 🔥 We provide day-0 support on MLX thanks to our close collaboration with the Cohere team. North Mini Code 1.0 is a 30B param MoE model with 3B active. It runs at ~66 tok/s in BF16, truly impressive speeds before any compression. Here is a quick demo I built using Pi by Mario Zechner & Armin Ronacher ⇌ as my agent hardness. Get started today: 1. Install mlx-vlm from source: > uv pip install git+ 2. Start the server and power your favourite agents (pi, opencode, hermes and etc) > uv run mlx_vlm.server --model CohereLabs/North-Mini-Code-1.0 New release tomorrow

Prince Canuma

35,367 views • 4 months ago

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨

ParoQuant just got a big upgrade 🚀 ✅ Supports the new Qwen3.5 models ⚡ Now runs on MLX (fast local inference on Apple Silicon) 🧠 Preserves reasoning quality with 4-bit quantization We also built an agent demo running locally on my 4-year-old M2 Max. Can't wait to upgrade to an M5 Max and see what kind of magic we can do. ✨

Zhijian Liu

49,826 views • 7 months ago

🎉 Congrats to MOSI + OpenMOSS on the release of MOSS-Transcribe-Diarize-0.9B — a genuinely impressive end-to-end ASR model for real-world, multi-speaker conversations. We're proud to partner with them for day-0 support on mlx-audio (Python & Swift). Now running local-first on Apple Silicon: 🎙️ transcript + speaker labels + timestamps in one pass 🗣️ unlimited-speaker diarization ⏱️ up to ~90 min of audio per input 🌍 90+ languages 🧠 128K context • Whisper-Medium encoder + Qwen3-0.6B decoder 🎯 hotword boosting for names, products & domain terms Perfect for meetings, podcasts, interviews, and long-form call analysis — no cloud, no data leaving your Mac. Get started now: 🐍 Python > uv pip install -U mlx-audio 🍎 Swift .package(url: " from: "0.1.3") Go build. 🚀

🎉 Congrats to MOSI + OpenMOSS on the release of MOSS-Transcribe-Diarize-0.9B — a genuinely impressive end-to-end ASR model for real-world, multi-speaker conversations. We're proud to partner with them for day-0 support on mlx-audio (Python & Swift). Now running local-first on Apple Silicon: 🎙️ transcript + speaker labels + timestamps in one pass 🗣️ unlimited-speaker diarization ⏱️ up to ~90 min of audio per input 🌍 90+ languages 🧠 128K context • Whisper-Medium encoder + Qwen3-0.6B decoder 🎯 hotword boosting for names, products & domain terms Perfect for meetings, podcasts, interviews, and long-form call analysis — no cloud, no data leaving your Mac. Get started now: 🐍 Python > uv pip install -U mlx-audio 🍎 Swift .package(url: " from: "0.1.3") Go build. 🚀

Prince Canuma

11,662 views • 3 months ago

🥧 Introducing GooeyPi - A GUI for the Pi Family of Agents. Supports: Pi, Oh-My-Pi (OMP), and Prime Agent Very excited to share this with you all. Being into local models, I wanted to have a separate harness for my local models so I don't have to constantly switch configs. And I didn't want to go back to the TUI (nerds). But, I didn't want to miss out on all the features that I know and love. Turns out, I didn't have to. I just needed to build it myself. So I did. Features include: > Fully featured GUI experience > Agentic Browser > Realtime Voice Agent > Voice transcription (local or API) > Computer Use > Automation Desk > Agent to Agent messaging > Ask Question Tool > Git Control > Terminal > Pets > and more I tried to capture all of the most popular features I could in this first pass, but we're only just getting started. Available in BETA on MacOS, Linux, and Windows. I need testers! I have only tested on Mac and Linux. If you want to help me with this project, please reach out. Available:

🥧 Introducing GooeyPi - A GUI for the Pi Family of Agents. Supports: Pi, Oh-My-Pi (OMP), and Prime Agent Very excited to share this with you all. Being into local models, I wanted to have a separate harness for my local models so I don't have to constantly switch configs. And I didn't want to go back to the TUI (nerds). But, I didn't want to miss out on all the features that I know and love. Turns out, I didn't have to. I just needed to build it myself. So I did. Features include: > Fully featured GUI experience > Agentic Browser > Realtime Voice Agent > Voice transcription (local or API) > Computer Use > Automation Desk > Agent to Agent messaging > Ask Question Tool > Git Control > Terminal > Pets > and more I tried to capture all of the most popular features I could in this first pass, but we're only just getting started. Available in BETA on MacOS, Linux, and Windows. I need testers! I have only tested on Mac and Linux. If you want to help me with this project, please reach out. Available:

am.will

168,840 views • 2 months ago

Hermes Agent now has Computer Use🤯 You can now control your actual computer with ANY model through Hermes Agent × Cua not just frontier models locked behind special modes. Even crazier: • Runs entirely in the background • Doesn’t hijack your PC • You keep full control of your keyboard, mouse, and screen while the agent works This is starting to feel less like an AI chatbot and more like a real agentic operating system.

Hermes Agent now has Computer Use🤯 You can now control your actual computer with ANY model through Hermes Agent × Cua not just frontier models locked behind special modes. Even crazier: • Runs entirely in the background • Doesn’t hijack your PC • You keep full control of your keyboard, mouse, and screen while the agent works This is starting to feel less like an AI chatbot and more like a real agentic operating system.

WorldofAI

62,302 views • 5 months ago

Introducing Marvis-TTS 🔥🚀 A new local-first TTS model Lucas Newman and I built for efficiency, accessibility, and real-time performance right on consumer devices like Apple Silicon, iPhones, iPads, and more. Traditional TTS models often demand full text inputs or sacrifice real-time capabilities, Marvis flips the script. It streams audio chunks as text is processed, creating a truly conversational experience. No more awkward pauses or unnatural breaks—Marvis handles the entire text context intelligently to deliver coherent, expressive speech. Get started today: > pip install -U mlx-audio

Introducing Marvis-TTS 🔥🚀 A new local-first TTS model Lucas Newman and I built for efficiency, accessibility, and real-time performance right on consumer devices like Apple Silicon, iPhones, iPads, and more. Traditional TTS models often demand full text inputs or sacrifice real-time capabilities, Marvis flips the script. It streams audio chunks as text is processed, creating a truly conversational experience. No more awkward pauses or unnatural breaks—Marvis handles the entire text context intelligently to deliver coherent, expressive speech. Get started today: > pip install -U mlx-audio

Prince Canuma

81,439 views • 1 year ago

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

Philipp Schmid

48,170 views • 1 year ago

🚀 Introducing the first-ever Agent UI 🚀 This is hands-down my favorite product! Chat with local Agents tailored to my needs. Local memory, storage, knowledge and tools 🔥 ⚡️ Your data, your control 🧠 Compatible with any LLM 🤝 Run multiple agents or a team of agents

🚀 Introducing the first-ever Agent UI 🚀 This is hands-down my favorite product! Chat with local Agents tailored to my needs. Local memory, storage, knowledge and tools 🔥 ⚡️ Your data, your control 🧠 Compatible with any LLM 🤝 Run multiple agents or a team of agents

Ashpreet Bedi

69,015 views • 2 years ago

🐆 Rapid-MLX v0.12 is here. We’ve officially evolved from a simple chat app into a full-fledged, on-device AI studio for Apple Silicon! 🖥️✨ We didn't just push the MLX inference engine to its limits and expand support for a massive lineup of local open-source models—we are alpha-launching the highly anticipated Desktop Version. (A huge shoutout to the IoTeX community for grinding through the closed beta with us. Your feedback was incredible and helped shape this beast.) Here are the game-changing features you can run on your Mac right now, 100% free and 100% offline 👇 🚀 Blazing Fast Local LLMs Run anything from 4B up to Qwen3.5-122B completely offline. No guessing games—we recommend models matched perfectly to your Mac's actual RAM. Rich chat includes syntax highlighting, markdown tables, and honest tok/s metrics. 🎨 Local Image Generation A brand new Images tab to render directly on your machine. Pick a model (FLUX.2-klein, Z-Image-Turbo), prompt, and refine. Everything lands in a visual filmstrip. 👁️ Vision & Live Web Tools Attach an image and chat about it with local vision. Need real-time data? Our built-in web tools (weather, search, page-fetch) run mid-answer with strict, transparent privacy controls. 🤖 Plug-and-Play Coding Agents Wire up Claude Code, Codex, Cline, or Continue in seconds. One copy-paste from the Launch tab spins up a local OpenAI/Anthropic-compatible endpoint. 🔒 Private by Design Everything runs on-device. Signed, notarized, and entirely local. Your data stays yours. Turn your Mac into an AI powerhouse today. ⚡️

raullen

34,196 views • 2 months ago

Introducing Google Flow Agent 🎬 ✅ Google Flow Agent can help you plan and reason through complex creative tasks with your inputs, all while under your control. 🧠 It's built with Gemini models and brings a deep understanding of your project to help with everything from early brainstorming to editing the final product. 💡 The agent can be a sounding board for dialogue between characters in a scene, and make plot recommendations when you need inspiration. 🎞️ Your Google Flow Agent can also create multiple variations of a scene at one time to give you more options and batch edit. 📂 And when you have all your final assets, your Google Flow Agent can organize them into collections, and even neatly rename them.

Introducing Google Flow Agent 🎬 ✅ Google Flow Agent can help you plan and reason through complex creative tasks with your inputs, all while under your control. 🧠 It's built with Gemini models and brings a deep understanding of your project to help with everything from early brainstorming to editing the final product. 💡 The agent can be a sounding board for dialogue between characters in a scene, and make plot recommendations when you need inspiration. 🎞️ Your Google Flow Agent can also create multiple variations of a scene at one time to give you more options and batch edit. 📂 And when you have all your final assets, your Google Flow Agent can organize them into collections, and even neatly rename them.

Google

154,346 views • 4 months ago

Today we're shipping our biggest MLX-VLM release yet: v0.6.0 ...and we are raising 💸 This one's about turning your Apple devices into real local agent machines. From your desk to your pocket. What's new: ⚡ Speculative decoding everywhere — Gemma 4 EAGLE3 + DFlash, Qwen MTP, DeepSeek V4 MTP. Faster tokens, less waiting. 🤖 Agent-ready server — native Anthropic /v1/messages API, stateful /v1/responses, tool calls, Codex context budgets. Plug Claude Code & Codex straight into local models. 👁️ New models galore — DeepSeek V4, ZAYA1-VL, MiniCPM-V 4.6, LFM2 MoE, Step-3.7 Flash, Laguna + more. 🎨 Image gen & editing — FLUX.2 (base + klein), PrismML Bonsai. 🔊 Audio in — Qwen3 Omni, Gemma 4 audio, base64 chat audio. 🧮 TurboQuant KV cache — RHT-correct fast paths for leaner memory. 📦 Modular server, better metrics, cleaner streaming. Run real agents on the hardware already in your hands. Github:

Today we're shipping our biggest MLX-VLM release yet: v0.6.0 ...and we are raising 💸 This one's about turning your Apple devices into real local agent machines. From your desk to your pocket. What's new: ⚡ Speculative decoding everywhere — Gemma 4 EAGLE3 + DFlash, Qwen MTP, DeepSeek V4 MTP. Faster tokens, less waiting. 🤖 Agent-ready server — native Anthropic /v1/messages API, stateful /v1/responses, tool calls, Codex context budgets. Plug Claude Code & Codex straight into local models. 👁️ New models galore — DeepSeek V4, ZAYA1-VL, MiniCPM-V 4.6, LFM2 MoE, Step-3.7 Flash, Laguna + more. 🎨 Image gen & editing — FLUX.2 (base + klein), PrismML Bonsai. 🔊 Audio in — Qwen3 Omni, Gemma 4 audio, base64 chat audio. 🧮 TurboQuant KV cache — RHT-correct fast paths for leaner memory. 📦 Modular server, better metrics, cleaner streaming. Run real agents on the hardware already in your hands. Github:

Prince Canuma

66,447 views • 4 months ago

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

Qwen

215,564 views • 1 year ago

We just open-sourced Simulang, a JavaScript library that gives you and your coding agents the ability to fully control computers. Simulang is our scripting framework for desktop automation. It allows you to write deterministic computer use code that can be replayed dozens if not hundreds of times. Much faster and cheaper, compared to LLM agents. It ships with a skill out of the box. That means you can turn natural language prompts into Simulang code directly, and your agent can now see the screen, click, type, and navigate. It gives your AI agents eyes and hands to control the computer like humans do. Try these things with Simulang: - upload your pile of invoices to the expense platform - find 50 people with specific criteria on LinkedIn - open your calculator app and do some math - find all buttons on your favorite app using accessibility tree - or draw a bunny with MS paint 🐰 In this demo, we show how Simulang took over when the coding agent couldn’t open and click on the target LinkedIn profile page. Give it a try:

Simular

14,515 views • 5 months ago

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

idan levin

28,307 views • 1 month ago

A conversational agent with no hallucinations! This is huge! Here is the $1M problem: Build a customer-facing AI agent that works consistently, doesn't hallucinate, and is fully aligned with the business rules. Check out Parlant, a FREE, open-source library. I've talked about them before, and here is the TL;DR: 1. Parlant gives you 10x more control over what your agent says 2. It helps you control how agents conduct conversations 3. It helps you control the service protocols and rules agents follow 4. It helps you control what agents can and can't say They do something pretty clever: 1. They have a set of predefined potential responses that the agent can use to reply to the customer. 2. When a customer sends a message, the agent generates a fluid response draft using an LLM and matches and renders the closest pre-approved reply template. 3. The agent returns the rendered pre-approved reply, not the one generated by the LLM. This maintains fluidity while avoiding unpredictability. Parlant also uses your domain terminology, invokes your tools, and implements self-critique to craft/evaluate its response iteratively. This pretty much solves any hallucination issues with your agent.

A conversational agent with no hallucinations! This is huge! Here is the $1M problem: Build a customer-facing AI agent that works consistently, doesn't hallucinate, and is fully aligned with the business rules. Check out Parlant, a FREE, open-source library. I've talked about them before, and here is the TL;DR: 1. Parlant gives you 10x more control over what your agent says 2. It helps you control how agents conduct conversations 3. It helps you control the service protocols and rules agents follow 4. It helps you control what agents can and can't say They do something pretty clever: 1. They have a set of predefined potential responses that the agent can use to reply to the customer. 2. When a customer sends a message, the agent generates a fluid response draft using an LLM and matches and renders the closest pre-approved reply template. 3. The agent returns the rendered pre-approved reply, not the one generated by the LLM. This maintains fluidity while avoiding unpredictability. Parlant also uses your domain terminology, invokes your tools, and implements self-critique to craft/evaluate its response iteratively. This pretty much solves any hallucination issues with your agent.

Santiago

64,237 views • 1 year ago