Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

Sophia Yang, Ph.D.

30,074 subscribers

49,801 görüntüleme • 1 yıl önce •via X (Twitter)

Sağlık & İyilik Bilim & Teknoloji Eğitim

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

How to train a reasoning model, specifically Magistral from Mistral AI? Here is a quick video walkthrough of the paper.

How to train a reasoning model, specifically Magistral from Mistral AI? Here is a quick video walkthrough of the paper.

Sophia Yang, Ph.D.

32,062 görüntüleme • 1 yıl önce

Introducing Voxtral WebGPU: Real-time speech transcription entirely in your browser. This demo runs Voxtral-Mini-4B, a powerful streaming ASR model from Mistral AI, locally on WebGPU. The model supports 13 languages and is capable of <500 ms latency. Fully private. Zero cost.

Introducing Voxtral WebGPU: Real-time speech transcription entirely in your browser. This demo runs Voxtral-Mini-4B, a powerful streaming ASR model from Mistral AI, locally on WebGPU. The model supports 13 languages and is capable of <500 ms latency. Fully private. Zero cost.

Xenova

94,356 görüntüleme • 5 ay önce

This is what happens when you plug LLMs into voice assistants, instead of a decade of handwritten rules. This video dissects Voxtral (a family of OSS speech models) and the foundational work behind it (audio tokenization, semantic/acoustic disaggregation, etc). Thank you Mistral AI for your collaboration and for your detailed technical reports in an increasingly opaque industry! 00:00 Intro 01:03 Modular vs end-to-end speech models 03:30 Speech-to-Text 06:07 Delayed Streams Modeling (DSM) 09:41 Whisper Streaming 10:33 Voxtral Realtime 13:07 Voxtral Text-to-Speech 14:28 Throwback: WaveNet 15:24 Audio tokenization 20:39 The Voxtral Codec 21:49 Back to Voxtral TTS 25:30 Outro

This is what happens when you plug LLMs into voice assistants, instead of a decade of handwritten rules. This video dissects Voxtral (a family of OSS speech models) and the foundational work behind it (audio tokenization, semantic/acoustic disaggregation, etc). Thank you Mistral AI for your collaboration and for your detailed technical reports in an increasingly opaque industry! 00:00 Intro 01:03 Modular vs end-to-end speech models 03:30 Speech-to-Text 06:07 Delayed Streams Modeling (DSM) 09:41 Whisper Streaming 10:33 Voxtral Realtime 13:07 Voxtral Text-to-Speech 14:28 Throwback: WaveNet 15:24 Audio tokenization 20:39 The Voxtral Codec 21:49 Back to Voxtral TTS 25:30 Outro

Julia Turc

297,055 görüntüleme • 1 ay önce

What a week for open audio models! 🕺 💃 I demo: 🗣️ Voxtral 4B TTS from Mistral AI 🎙️ Transcribe 2B from Cohere 🏭 and how to run a batch transcribe job in 1 line of CLI using Daniel van Strien uv script links below

What a week for open audio models! 🕺 💃 I demo: 🗣️ Voxtral 4B TTS from Mistral AI 🎙️ Transcribe 2B from Cohere 🏭 and how to run a batch transcribe job in 1 line of CLI using Daniel van Strien uv script links below

Jeff Boudier 🤗

32,940 görüntüleme • 4 ay önce

🤔What does it take to train an omni world model from scratch? 📽️Petabytes of video. 6⃣pipeline stages. And every improvement in data quality pays off twice when you're training a model that both generates and understands video. Our Reka Labs' data team on how it works → #PhysicalAI #WorldModels #AI #RekaLabs

🤔What does it take to train an omni world model from scratch? 📽️Petabytes of video. 6⃣pipeline stages. And every improvement in data quality pays off twice when you're training a model that both generates and understands video. Our Reka Labs' data team on how it works → #PhysicalAI #WorldModels #AI #RekaLabs

Reka

15,051 görüntüleme • 1 ay önce

A quick walkthrough of Terminal Mode on Even G2 to show what it actually looks like in use and how easy it is to setup. #DevTools #BuildInPublic #vibecoding

A quick walkthrough of Terminal Mode on Even G2 to show what it actually looks like in use and how easy it is to setup. #DevTools #BuildInPublic #vibecoding

Even Realities

14,666 görüntüleme • 3 ay önce

Ok this new Indy AI from Contra actually feels like a humongous deal... so I made a quick ~1 min walkthrough 👀

Ok this new Indy AI from Contra actually feels like a humongous deal... so I made a quick ~1 min walkthrough 👀

Ridd 🤿

16,692 görüntüleme • 1 yıl önce

1. Meta’s open-sourced multisensory model Meta is back (again!) with yet another exciting open-source project. Introducing ImageBind, a new AI research model that understands and combines text, audio, visual, movement, thermal, AND depth data.

1. Meta’s open-sourced multisensory model Meta is back (again!) with yet another exciting open-source project. Introducing ImageBind, a new AI research model that understands and combines text, audio, visual, movement, thermal, AND depth data.

Rowan Cheung

173,987 görüntüleme • 3 yıl önce

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)

AK

23,082 görüntüleme • 10 ay önce

LFM2-Audio just dropped! It's a 1.5B model that understands and generates both text and audio Inference 10x faster + quality on par with models 10x larger Available today on Hugging Face and our playground 🥳

LFM2-Audio just dropped! It's a 1.5B model that understands and generates both text and audio Inference 10x faster + quality on par with models 10x larger Available today on Hugging Face and our playground 🥳

Maxime Labonne

85,053 görüntüleme • 10 ay önce

More Adobe Substance Modeler +AI experiments. Using quick, gestural 3d sketching to drive generative AI with much higher levels of creative control than you could ever get from a text prompt alone. The more detailed your 3d model is, the more you can leave the ai to just finish details like skin texture and individual strands of hair.. my model is obviously just a quick sketch😅 #art #ai

More Adobe Substance Modeler +AI experiments. Using quick, gestural 3d sketching to drive generative AI with much higher levels of creative control than you could ever get from a text prompt alone. The more detailed your 3d model is, the more you can leave the ai to just finish details like skin texture and individual strands of hair.. my model is obviously just a quick sketch😅 #art #ai

Martin Nebelong

56,554 görüntüleme • 2 yıl önce

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

Awni Hannun

66,565 görüntüleme • 2 yıl önce

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

Hamza Khalid

26,707 görüntüleme • 10 ay önce

An open-weight alternative to ElevenLabs! Voxtral is a TTS model by Mistral with: - just 4B params - 70ms latency for voice agents - voice cloning from 3s of audio - 9 languages + cross-lingual transfer - 68.4% win rate over ElevenLabs Flash v2.5 Open weights on Hugging Face.

An open-weight alternative to ElevenLabs! Voxtral is a TTS model by Mistral with: - just 4B params - 70ms latency for voice agents - voice cloning from 3s of audio - 9 languages + cross-lingual transfer - 68.4% win rate over ElevenLabs Flash v2.5 Open weights on Hugging Face.

Avi Chawla

28,162 görüntüleme • 4 ay önce

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Rohan Paul

15,672 görüntüleme • 4 ay önce

AI Text-to-Music has arrived. MusicLM is a model by Google Research that generates high-fidelity music from text descriptions. Basically just enter some text and it will create the music. 🤯 Attached video shows sample text prompts and AI generated music.

AI Text-to-Music has arrived. MusicLM is a model by Google Research that generates high-fidelity music from text descriptions. Basically just enter some text and it will create the music. 🤯 Attached video shows sample text prompts and AI generated music.

Dave Lee

72,546 görüntüleme • 3 yıl önce

you might've heard about context engineering, but nobody has explained what it really is... that's why I made a full practical walkthrough on how to use it to make your AI outputs personalized and consistent in this walkthrough, I go over: – what context engineering actually is (without the jargon) – how to train AI on your own data and preferences – how to get outputs that sound like you wrote them (not generic AI) reply "context" and I'll DM it to you (must be following)

you might've heard about context engineering, but nobody has explained what it really is... that's why I made a full practical walkthrough on how to use it to make your AI outputs personalized and consistent in this walkthrough, I go over: – what context engineering actually is (without the jargon) – how to train AI on your own data and preferences – how to get outputs that sound like you wrote them (not generic AI) reply "context" and I'll DM it to you (must be following)

Tyler

20,017 görüntüleme • 11 ay önce

Most conversational AI understands words, not people. Introducing Raven-1, our audio and video perception model that gives AI the ability to understand emotion, intent, and context the way humans do.

Most conversational AI understands words, not people. Introducing Raven-1, our audio and video perception model that gives AI the ability to understand emotion, intent, and context the way humans do.

Tavus

1,672,610 görüntüleme • 6 ay önce

The Qwen model family cyber atlas: a visual walkthrough of how AI really evolved — from large language models to embodied intelligence. Discover the full evolution in the video below. Ready to explore further? • Try Model Studio: • or Discover Qwen Cloud:

The Qwen model family cyber atlas: a visual walkthrough of how AI really evolved — from large language models to embodied intelligence. Discover the full evolution in the video below. Ready to explore further? • Try Model Studio: • or Discover Qwen Cloud:

Alibaba Cloud

27,703 görüntüleme • 19 gün önce

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Meta Newsroom

204,678 görüntüleme • 8 ay önce