Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

This is "Whisper.cpp" a local lightweight AI model that can be run on mobile phones to convert speech to text with real time streaming

Abdelhamed M.

6,177 subscribers

143,177 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

11 Comments

Hydra1 year ago

original package: react native package:

Hydra1 year ago

I’m not the creator of the package - just using it for a project I’m working on I’ll share the project once I finish it, hopefully by tomorrow

AssemblyAI1 year ago

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

VibeCode1 year ago

need this

Hydra1 year ago

We are cooking for @OblienHQ We are coming for you

Peter Fasula1 year ago

Pretty cool @VZHydra . So it’s not actually passing any data to open AI? Any idea how much it increases your app bundle size?

Hydra1 year ago

No, unlike the official Whisper API, this is a fully local solution that runs on CPU BTW, the advanced Whisper model is open-source and can be run locally without requiring powerful hardware

Rishi1 year ago

Can anyone tell me how such models are run locally ?

Hydra1 year ago

C++ magic

sadiq🇬🇭1 year ago

Good news

Metin1 year ago

Only works well in English with a small model; others take up too much space.

Related Videos

🎙️Do you know you now have all the building blocks for full speech-to-speech? - Voxtral Realtime: High-quality, real-time speech-to-text. - Mistral Small 4: Fast, efficient, general-purpose agentic model. - Voxtral TTS: Realistic customizable text-to-speech with streaming output.

🎙️Do you know you now have all the building blocks for full speech-to-speech? - Voxtral Realtime: High-quality, real-time speech-to-text. - Mistral Small 4: Fast, efficient, general-purpose agentic model. - Voxtral TTS: Realistic customizable text-to-speech with streaming output.

Mistral AI for Developers

27,647 views • 3 months ago

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

Introducing SeamlessM4T, the first all-in-one, multilingual multimodal translation model. This single model can perform tasks across speech-to-text, speech-to-speech, text-to-text translation & speech recognition for up to 100 languages depending on the task. Details ⬇️

AI at Meta

592,704 views • 2 years ago

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required. Open-source, lightweight, and incredibly fast. 🧵👇

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required. Open-source, lightweight, and incredibly fast. 🧵👇

kyutai

236,570 views • 5 months ago

Universal-Streaming is our real-time speech-to-text model — built for voice agents with ~300ms latency, high accuracy, unlimited concurrency, and just $0.15/hour. 👉 Test it live in the playground →

Universal-Streaming is our real-time speech-to-text model — built for voice agents with ~300ms latency, high accuracy, unlimited concurrency, and just $0.15/hour. 👉 Test it live in the playground →

AssemblyAI

1,017,880 views • 1 year ago

Former Pakistani cricketer Shahid Afridi said that children and girls should not be given mobile phones. According to him, girls should get mobile phones only after marriage. Shahid Afridi even said that he gave mobile phones to his own daughters only after their marriage. He also called mobile phones a disease. At a time when the world is moving so fast, the era of AI has arrived, and everything from education to daily work is happening online through mobile phones and AI, Pakistanis are saying that girls should get phones only after marriage. Pakistan is an anti-women country. Because of this mindset, Pakistan remains so backward and poor.

Former Pakistani cricketer Shahid Afridi said that children and girls should not be given mobile phones. According to him, girls should get mobile phones only after marriage. Shahid Afridi even said that he gave mobile phones to his own daughters only after their marriage. He also called mobile phones a disease. At a time when the world is moving so fast, the era of AI has arrived, and everything from education to daily work is happening online through mobile phones and AI, Pakistanis are saying that girls should get phones only after marriage. Pakistan is an anti-women country. Because of this mindset, Pakistan remains so backward and poor.

Saffron Chargers

478,486 views • 1 month ago

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Did xAI just mass-murder the entire voice AI industry? 🤯 Grok just launched two voice APIs. Speech-to-Text and Text-to-Speech. Built on the same stack powering Tesla cars and Starlink support. And priced at 10x cheaper than ElevenLabs. Speech-to-Text: $0.10/hr batch. $0.20/hr streaming. Text-to-Speech: $4.20 per million characters. 25+ languages. Real-time streaming. Speaker diarization. Already outperforming ElevenLabs, Deepgram, and AssemblyAI on word error rate. TTS ships with expressive tags like [laugh], [sigh], , . Voices that don't sound like robots reading a script. ElevenLabs spent years building a voice AI company. xAI built voice AI for cars and satellites.

Vaibhav Sisinty

24,524,333 views • 2 months ago

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Speak with ChatGPT: You can now use voice to engage in a back-and-forth conversation with your assistant. The hyper-realistic text-to-speech model allows you to choose from five different voices. On mobile, opt-in to voice in Settings → New Features on the mobile app.

Rowan Cheung

55,777 views • 2 years ago

A brain implant can convert thoughts to speech

A brain implant can convert thoughts to speech

TIME

25,518 views • 1 year ago

The Moshi speech-to-speech model by kyutai, hosted on Modal, absolutely cranks! Near instantaneous voice chat AI, thanks to bidirectional streaming. Whole startups could be built on this stack.

The Moshi speech-to-speech model by kyutai, hosted on Modal, absolutely cranks! Near instantaneous voice chat AI, thanks to bidirectional streaming. Whole startups could be built on this stack.

Erik Dunteman

55,400 views • 1 year ago

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

AK

43,659 views • 11 months ago

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with <300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice & accent Try here -

Realtime AI Conversations are here! Introducing PlayHT 2.0 Turbo ⚡️ Our new blazing fast Conversational AI Text-to-Speech model with <300ms latency! ✅ Input text streaming from LLMs ✅ Output audio streaming ✅ Clone any voice & accent Try here -

PlayAI

392,536 views • 2 years ago

How to convert ANY web app into a Mobile App... That can be shipped to the App Store from your phone. (Zero Coding Required) This works with web apps made on Replit, Bolt, v0, Replit or Cursor. Tech Stack: 1. Claude Desktop (works with ChatGPT) 2. - Builds the Mobile app, Ships to App Store directly from the phone (Uses Claude Code) (Prompt for conversion below) TIMESTAMPS Intro The Web App We Will Convert The Mobile App that Builds Mobile Apps Mobile App Version 1 You can Publish this to the App Store Adding Additional Features (AI APIs) to your app Editing the Mobile App with the Select Feature

How to convert ANY web app into a Mobile App... That can be shipped to the App Store from your phone. (Zero Coding Required) This works with web apps made on Replit, Bolt, v0, Replit or Cursor. Tech Stack: 1. Claude Desktop (works with ChatGPT) 2. - Builds the Mobile app, Ships to App Store directly from the phone (Uses Claude Code) (Prompt for conversion below) TIMESTAMPS Intro The Web App We Will Convert The Mobile App that Builds Mobile Apps Mobile App Version 1 You can Publish this to the App Store Adding Additional Features (AI APIs) to your app Editing the Mobile App with the Select Feature

Riley Brown

59,326 views • 7 months ago

This is wild. Hume AI just dropped Octave 2. Ultra realistic text-to-speech AI model in 10+ languages with multi-speaker + voice cloning. 100% AI 8 wild examples + how to try:

This is wild. Hume AI just dropped Octave 2. Ultra realistic text-to-speech AI model in 10+ languages with multi-speaker + voice cloning. 100% AI 8 wild examples + how to try:

Min Choi

212,441 views • 8 months ago

🚨 New model alert! Dialog by vibx — a leading text-to-speech model — now runs on GroqCloud™. That means natural-sounding speech with ultra-low latency, making real-time voice applications smoother and more responsive. Learn more & build fast — links in the comments!

🚨 New model alert! Dialog by vibx — a leading text-to-speech model — now runs on GroqCloud™. That means natural-sounding speech with ultra-low latency, making real-time voice applications smoother and more responsive. Learn more & build fast — links in the comments!

Groq Inc

47,183 views • 1 year ago

Daily #Blender Tip: If you have a model that fakes detail using a Normal map, it can be interesting to convert it to ‘real’ geometry that you can continue sculpting on, or for use in 3D printing. This shows you how to do it! #B3D

Daily #Blender Tip: If you have a model that fakes detail using a Normal map, it can be interesting to convert it to ‘real’ geometry that you can continue sculpting on, or for use in 3D printing. This shows you how to do it! #B3D

Jan van den Hemel

75,105 views • 3 years ago

Some folks asked if this offline local, no internet can ever connected, real-time speech to text device requires me to speak very slowly. In the prior video, I was reading what I was saying. Here I am just speaking in my normal mumble. This is designed to build AI models based on everything I heard as speech, like videos, radio, phone (with permission), conversations to be memorized by my local AI. It should be clear that this device takes up all of my conversations for about 10 hours per day on a single battery charge. This is what I am building for you, if you want it. No, there is no monthly subscription on the to access the context of your life, and there better not be, ever. It just will take me some time.

Some folks asked if this offline local, no internet can ever connected, real-time speech to text device requires me to speak very slowly. In the prior video, I was reading what I was saying. Here I am just speaking in my normal mumble. This is designed to build AI models based on everything I heard as speech, like videos, radio, phone (with permission), conversations to be memorized by my local AI. It should be clear that this device takes up all of my conversations for about 10 hours per day on a single battery charge. This is what I am building for you, if you want it. No, there is no monthly subscription on the to access the context of your life, and there better not be, ever. It just will take me some time.

Brian Roemmele

72,280 views • 7 months ago

Generate lifelike audio in real-time without a GPU! 🚀 Check out orpheus-cpp: a llama.cpp port of orpheus 3b text-to-speech model with built-in support for sync and async streaming. 𝚙𝚒𝚙 𝚒𝚗𝚜𝚝𝚊𝚕𝚕 𝚘𝚛𝚙𝚑𝚎𝚞𝚜-𝚌𝚙𝚙 𝚙𝚢𝚝𝚑𝚘𝚗 -𝚖 𝚘𝚛𝚙𝚑𝚎𝚞𝚜_𝚌𝚙𝚙

Generate lifelike audio in real-time without a GPU! 🚀 Check out orpheus-cpp: a llama.cpp port of orpheus 3b text-to-speech model with built-in support for sync and async streaming. 𝚙𝚒𝚙 𝚒𝚗𝚜𝚝𝚊𝚕𝚕 𝚘𝚛𝚙𝚑𝚎𝚞𝚜-𝚌𝚙𝚙 𝚙𝚢𝚝𝚑𝚘𝚗 -𝚖 𝚘𝚛𝚙𝚑𝚎𝚞𝚜_𝚌𝚙𝚙

Freddy A Boulton

17,465 views • 1 year ago

It's happening... Google DeepMind is about to drop Genie 3 This AI can generate interactive 3D worlds with just text... And you can interact with objects in real time 🤯

It's happening... Google DeepMind is about to drop Genie 3 This AI can generate interactive 3D worlds with just text... And you can interact with objects in real time 🤯

Min Choi

727,361 views • 5 months ago

Claude computer use is too expensive check out ShowUI Local Run! Now, you can use a lightweight model to power a local GUI Agent. It's compatible with "GPT-4o + ShowUI" and "Qwen2-VL + ShowUI" Each task costs just a few cents—30x cheaper than Claude!

Claude computer use is too expensive check out ShowUI Local Run! Now, you can use a lightweight model to power a local GUI Agent. It's compatible with "GPT-4o + ShowUI" and "Qwen2-VL + ShowUI" Each task costs just a few cents—30x cheaper than Claude!

AK

30,822 views • 1 year ago

Google researchers just developed GameNGen, an AI that can simulate DOOM at over 20 fps It works by predicting each frame in real time with a diffusion model At scale this could mean AI will be able to create games on the fly, personalized to each player

Google researchers just developed GameNGen, an AI that can simulate DOOM at over 20 fps It works by predicting each frame in real time with a diffusion model At scale this could mean AI will be able to create games on the fly, personalized to each player

Rowan Cheung

880,322 views • 1 year ago