Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Here is an example Pipecat project for trying PhoneLLM Alpha 1. Deploy the model to a Modal endpoint with one click. Client from the video in repo too (with all that sweet sweet terminal-ish aura.) Next up, a TTS that can pronounce Merve Tan's name correctly? 😅

Jon Taylor

1,167 subscribers

21,006 views • 21 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 Comments

ata20 days ago

@bmervetan tts’e merve’yi okutabiliriz. agent kullanıcıdan yeni bir özel isim duyduğunda son audio turn’de ilgili span üzerinde phoneme extraction çalıştıracak, sonra aynı isim için sonraki tts turn’lerine phoneme override enjekte edecek. yw :)

Tim 🚴‍♂️ Builder20 days ago

@bmervetan Für mich ist das Repo fast wichtiger als der Model-Launch: Client plus Deploy-Pfad schaffen erst die Basis für faire Vergleiche. Als Nächstes würde ich ein deutsches Eval-Set für Unterbrechungen, Tool Calls und Namen bauen. Dort zeigt sich, wie alltagstauglich Voice Agents sind.

yian21 days ago

@bmervetan I tried this model and it didn't know what tomorrow was, (thought today was Thursday, tomorrow was Saturday)

Scott Thomas21 days ago

@bmervetan hi I'm actively building a voice sales agent. I tried phonellm through live kit and it could not tool call very well. Any suggestions? Willing to also give pipecat a try

LukeHalley21 days ago

@bmervetan I'm thinking of doing this for our platform - trying to handle bursty traffic without running a GPU 24/7. We currently use API endpoints to keep costs down, as we don't need 24/7 inference, but we'd love this low latency. Any words of wisdom?

Ahmed Omar.20 days ago

@bmervetan Jon we'd love to partner. please dm

Related Videos

GLM-5, the latest frontier open model from Z.ai, is available now on Modal. We partnered with to release an endpoint that will be free for a limited time.

GLM-5, the latest frontier open model from Z.ai, is available now on Modal. We partnered with to release an endpoint that will be free for a limited time.

Modal

58,785 views • 7 months ago

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

kwindla

329,729 views • 22 days ago

Here is a step-by-step introduction to building a workflow with a custom AI agent that uses MCP. I explain every component in the video: 1. Building the MCP server 2. Building the agent and an MCP client 3. Building a workflow that uses the agent The goal is simple: Generate a dialogue between two people and make one yell and the other answer with sarcasm. Kestra is sponsoring this video. They are an open-source orchestration platform (repo link below), and I used them to build the workflow that connects every component.

Here is a step-by-step introduction to building a workflow with a custom AI agent that uses MCP. I explain every component in the video: 1. Building the MCP server 2. Building the agent and an MCP client 3. Building a workflow that uses the agent The goal is simple: Generate a dialogue between two people and make one yell and the other answer with sarcasm. Kestra is sponsoring this video. They are an open-source orchestration platform (repo link below), and I used them to build the workflow that connects every component.

Santiago

51,527 views • 1 year ago

Wow, TypeSafe AI Jev is a game changer for voice-driven UI. Responsiveness really matters for an experience to 'feel' right, and this certainly does the job! Latency here is not optimal given I'm in London and calling out to us-west. But still, super snappy. In this video, Jamcat is using Pipecat to orchestrate a real-time handover between a command agent (Jev) that drives the UI, and voice agent (PhoneLLM) for handling generations and (not in the video) discussing ideas and session setup with the user.

Wow, TypeSafe AI Jev is a game changer for voice-driven UI. Responsiveness really matters for an experience to 'feel' right, and this certainly does the job! Latency here is not optimal given I'm in London and calling out to us-west. But still, super snappy. In this video, Jamcat is using Pipecat to orchestrate a real-time handover between a command agent (Jev) that drives the UI, and voice agent (PhoneLLM) for handling generations and (not in the video) discussing ideas and session setup with the user.

Jon Taylor

26,440 views • 1 day ago

New video drop! This short walkthrough shows how Docker can be used to quickly launch a CrewAI project from the Compose for Agents repo, configured with a YAML file to set up a complete marketing team of AI agents in just minutes. Watch here:

New video drop! This short walkthrough shows how Docker can be used to quickly launch a CrewAI project from the Compose for Agents repo, configured with a YAML file to set up a complete marketing team of AI agents in just minutes. Watch here:

Docker

11,702 views • 1 year ago

They were in a dogfight with two of the worst playoff teams and are trying to downplay the 1 Seed that THEY lost Emmanwori was right this is another sweet ass team

They were in a dogfight with two of the worst playoff teams and are trying to downplay the 1 Seed that THEY lost Emmanwori was right this is another sweet ass team

ᴅᴏxx ⚡️

222,314 views • 7 months ago

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

Jake Fitzgerald

22,113 views • 4 months ago

The Rockies are trying to break a record tonight for the largest gathering of people with one name. That name is Ryan. Naturally, Ryan McMahon launches one 467 to celebrate!

The Rockies are trying to break a record tonight for the largest gathering of people with one name. That name is Ryan. Naturally, Ryan McMahon launches one 467 to celebrate!

Just Baseball

1,600,251 views • 1 year ago

Here is that video on profile I promised. Covering an error I see a ton of traders make with this tool. Short a sweet, if you want detailed instruction on how to use this, and more of the nuances, that is what the course playlist is for. Enjoy.

Horse

51,638 views • 2 months ago

DeepSeek Engineer v2 is here! 🐋 After trying the latest checkpoint for r1 and being super impressed, I decided to update the repo on GitHub, now with function calling, which is supported natively by the model. "A rotating cube with the solar system inside, all in HTML"

DeepSeek Engineer v2 is here! 🐋 After trying the latest checkpoint for r1 and being super impressed, I decided to update the repo on GitHub, now with function calling, which is supported natively by the model. "A rotating cube with the solar system inside, all in HTML"

Pietro Schirano

114,085 views • 1 year ago

Weekend project: A reusable deployment pipeline defined in C# that can take any aspire project and deploy it to a VM with ssh and docker. - Use of C# >>>> Shell scripts interleaved in YAML 🤮. - The pipeline code that runs over a model. This same code can work for any aspire app without changes. It's a dynamic pipeline template. - It can run locally or in the CI - This was built on top of upcoming APIs in Aspire 9.4 (you can build your own pipeline and deploy anywhere too!) #dotnet #aspire

Weekend project: A reusable deployment pipeline defined in C# that can take any aspire project and deploy it to a VM with ssh and docker. - Use of C# >>>> Shell scripts interleaved in YAML 🤮. - The pipeline code that runs over a model. This same code can work for any aspire app without changes. It's a dynamic pipeline template. - It can run locally or in the CI - This was built on top of upcoming APIs in Aspire 9.4 (you can build your own pipeline and deploy anywhere too!) #dotnet #aspire

David Fowler

23,700 views • 1 year ago

One game is all it takes to write your name in football history and just a few players can boast of that Here are 10 matches where players became immortal A Thread 1. Zidane with one of the greatest volleys in the game to secure Madrid’s 2002 UCL

One game is all it takes to write your name in football history and just a few players can boast of that Here are 10 matches where players became immortal A Thread 1. Zidane with one of the greatest volleys in the game to secure Madrid’s 2002 UCL

Joey👽

3,434,267 views • 1 year ago

In all seriousness, the name General Intuition comes from this Demis Hassabis talk, as the north star of the lab is a general agent, where "Play the World" applies to scientific problems that can become games too! Another example:

In all seriousness, the name General Intuition comes from this Demis Hassabis talk, as the north star of the lab is a general agent, where "Play the World" applies to scientific problems that can become games too! Another example:

Pim de Witte

12,166 views • 2 months ago

I've been playing around with FLUX 3 for the last few week - it's an incredibly impressive model. It can generate up to 20 seconds of video across multiple shots with one prompt 🤯 This model is so strong at the little details that make a clip look real - a few favorites 👇

I've been playing around with FLUX 3 for the last few week - it's an incredibly impressive model. It can generate up to 20 seconds of video across multiple shots with one prompt 🤯 This model is so strong at the little details that make a clip look real - a few favorites 👇

Justine Moore

66,759 views • 1 month ago

Introducing RAGs v3 🌟: Build a RAG bot that can also search the web 🌐, to find answers that aren’t immediately in its corpus. Do this all in natural language, not code! 💬 Get a better experience than ChatGPT + Bing ⚡️ We used our integration with Metaphor - a search engine designed for LLMs - to pull in relevant text from the internet. Additionally: - you can now view the tools that the agent is using. - note: web search is only accessible for our OpenAI agent Check out the video below, and get started with the repo today. Repo:

Introducing RAGs v3 🌟: Build a RAG bot that can also search the web 🌐, to find answers that aren’t immediately in its corpus. Do this all in natural language, not code! 💬 Get a better experience than ChatGPT + Bing ⚡️ We used our integration with Metaphor - a search engine designed for LLMs - to pull in relevant text from the internet. Additionally: - you can now view the tools that the agent is using. - note: web search is only accessible for our OpenAI agent Check out the video below, and get started with the repo today. Repo:

LlamaIndex 🦙

76,371 views • 2 years ago

"I've been in all situations. From a MF that don't play, franchise player, all that. Don't take this s--- for granted. Embrace the journey. It ain't always going to be sweet. Y'all know what I'm dealing with right now. I'm trying to get back in." Isaiah Thomas | Rico Hines

"I've been in all situations. From a MF that don't play, franchise player, all that. Don't take this s--- for granted. Embrace the journey. It ain't always going to be sweet. Y'all know what I'm dealing with right now. I'm trying to get back in." Isaiah Thomas | Rico Hines

Ballislife.com

3,369,381 views • 3 years ago