Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access... show more

kyutai

27,387 subscribers

47,965 görüntüleme • 1 yıl önce •via X (Twitter)

Sağlık & İyilik Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

11 Yorum

kyutai profil fotoğrafı

kyutai1 yıl önce

🧠 How it works MoshiVis builds on Moshi, our speech-to-speech LLM — now enhanced with vision. 206M lightweight parameters on top of a frozen Moshi give it the power to discuss images while still remaining real-time on consumer-grade hardware. E.g., on a MacMini M4, MoshiVis only adds ~7ms per step to the ~45ms per step of the base model, thus remaining well below the 80ms threshold for our audio codec. That means fluid, live, and multimodal conversations with Moshi on your own device!

kyutai profil fotoğrafı

kyutai1 yıl önce

🧰 Fully open-source We’re releasing a detailed preprint, along with model weights and a first of its kind benchmark dataset for spoken visual question answering: 📄 Preprint 🧠 Speech Benchmarks 🧾 Model weights 🧪 Inference code in PyTorch, MLX, and Rust

kyutai profil fotoğrafı

kyutai1 yıl önce

If you want to work on cutting-edge research, join our non-profit AI lab in Paris 🇫🇷 Thanks to Iliad Group, CMA-CGM Group, Schmidt Sciences — and the open-source community.

AssemblyAI profil fotoğrafı

AssemblyAI1 yıl önce

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

ZOHEB profil fotoğrafı

ZOHEB1 yıl önce

More lightweight! Sweet

Utopic e/λ profil fotoğrafı

Utopic e/λ1 yıl önce

share the repo with us ❤

Danish Khan profil fotoğrafı

Danish Khan1 yıl önce

Tried it out, very good performance for everyday tasks!

Frédéric H. (E/ACC) profil fotoğrafı

Frédéric H. (E/ACC)1 yıl önce

Ridiculous. Your AI is still not able to speak other language than English and doesn't even work decently. What a shame.

haareblond profil fotoğrafı

haareblond1 yıl önce

wow this is very cool!

Mogomra (e/acc) profil fotoğrafı

Mogomra (e/acc)1 yıl önce

Holey moley, it's the AI from "Start-up" in the real world!!

X_Learning969 profil fotoğrafı

X_Learning9691 yıl önce

Local not working. Despite enabling mic and trying out the web ui it does not speak. Wonder if it can even hear me.

Benzer Videolar

Today we are launching VE2 — a brand new model for Visual Electric. It can produce hyper realistic photos, accurate text, and generate images 2.5x faster. This is the biggest update ever to Visual Electric. 🔊 SOUND ON

Colin Dunn

41,443 görüntüleme • 2 yıl önce

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

AI at Meta

1,251,245 görüntüleme • 7 ay önce

π0.7 handles diverse prompts that don't just say what to do, but also how to do it, including rich language and multimodal information, such as visual subgoal images. At test time, these images can be produced by a lightweight world model.

π0.7 handles diverse prompts that don't just say what to do, but also how to do it, including rich language and multimodal information, such as visual subgoal images. At test time, these images can be produced by a lightweight world model.

Physical Intelligence

33,708 görüntüleme • 3 ay önce

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 görüntüleme • 7 ay önce

Gemma 3 understands images, text, and video - all at once. In this deep dive, learn how the model integrates multiple sources and performs a range of tasks from answering questions about documents to describing visual scenes in detail. Explore why multimodality matters.

Gemma 3 understands images, text, and video - all at once. In this deep dive, learn how the model integrates multiple sources and performs a range of tasks from answering questions about documents to describing visual scenes in detail. Explore why multimodality matters.

Google AI Developers

41,128 görüntüleme • 10 ay önce

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Google AI

68,285 görüntüleme • 10 ay önce

New open Omni model released! 👀OpenBMB MiniCPM-o 2.6 is a new 8B parameters, any-to-any multimodal model that can understand vision, speech, and language and runs on edge devices like phones and tablets. TL;DR: 🧠 8B total parameters (SigLip-400M + Whisper-300M + ChatTTS-200M + Qwen2.5-7B) 🔥 Outperforms GPT-4V on visual tasks with 70.2 average score on OpenCompass 🎙️ Best-in-class bilingual speech capabilities with real-time conversation and voice cloning 🎬 Supports multimodal streaming with support for continuous video/audio processing 📱 Runs on iPads and phones and supports 30+ languages 🖼️ Processes images up to 1.8M pixels (1344x1344) with OCR capabilities 🛠️ Easy integration with popular frameworks (llama.cpp, vLLM, Gradio) 🤗 commercial friendly (< 1 mio DAU) and available on Hugging Face

New open Omni model released! 👀OpenBMB MiniCPM-o 2.6 is a new 8B parameters, any-to-any multimodal model that can understand vision, speech, and language and runs on edge devices like phones and tablets. TL;DR: 🧠 8B total parameters (SigLip-400M + Whisper-300M + ChatTTS-200M + Qwen2.5-7B) 🔥 Outperforms GPT-4V on visual tasks with 70.2 average score on OpenCompass 🎙️ Best-in-class bilingual speech capabilities with real-time conversation and voice cloning 🎬 Supports multimodal streaming with support for continuous video/audio processing 📱 Runs on iPads and phones and supports 30+ languages 🖼️ Processes images up to 1.8M pixels (1344x1344) with OCR capabilities 🛠️ Easy integration with popular frameworks (llama.cpp, vLLM, Gradio) 🤗 commercial friendly (< 1 mio DAU) and available on Hugging Face

Philipp Schmid

19,409 görüntüleme • 1 yıl önce

Meet FLUX Kontext – the new context-aware image generation and editing model. We’ve teamed up with Black Forest Labs to drop the model on OpenArt today! You can try out now for free. And we’ve built a 50-page visual guidebook packed with detailed examples, pro tips & secret tricks you won’t find anywhere else for Flux Kontext. Find it in the first comment. Get ready for the new Kontext era. If you find the book helpful, remember to like and repost. #fluxkontext #openart #kontext

Meet FLUX Kontext – the new context-aware image generation and editing model. We’ve teamed up with Black Forest Labs to drop the model on OpenArt today! You can try out now for free. And we’ve built a 50-page visual guidebook packed with detailed examples, pro tips & secret tricks you won’t find anywhere else for Flux Kontext. Find it in the first comment. Get ready for the new Kontext era. If you find the book helpful, remember to like and repost. #fluxkontext #openart #kontext

OpenArt

12,911 görüntüleme • 1 yıl önce

We're releasing Hibiki-Zero, a new real-time and multilingual speech translation model that can translate 🇫🇷French, 🇪🇸Spanish, 🇵🇹Portuguese and 🇩🇪German to English: accurate, low-latency, high audio quality, with voice transfer. And best of all: open-source.

We're releasing Hibiki-Zero, a new real-time and multilingual speech translation model that can translate 🇫🇷French, 🇪🇸Spanish, 🇵🇹Portuguese and 🇩🇪German to English: accurate, low-latency, high audio quality, with voice transfer. And best of all: open-source.

kyutai

80,869 görüntüleme • 5 ay önce

Viggle’s Mic 2.0 is here! With our upgraded model, take precise control over how a character talks and moves, in one shot. Turn images into videos simply with an audio or text prompt for speech. Or take it further—add a motion source to drive the character’s movements—action and voice, perfectly aligned. Try it free on our web, or on our iOS/Android apps.👇

Viggle’s Mic 2.0 is here! With our upgraded model, take precise control over how a character talks and moves, in one shot. Turn images into videos simply with an audio or text prompt for speech. Or take it further—add a motion source to drive the character’s movements—action and voice, perfectly aligned. Try it free on our web, or on our iOS/Android apps.👇

ViggleAI

36,121 görüntüleme • 1 yıl önce

🚀Announcing LangGraph Studio: The first agent IDE LangGraph Studio offers a new way to develop LLM applications by providing a specialized agent IDE that enables visualization, interaction, and debugging of complex agentic applications With visual graphs and the ability to edit state, you can better understand agent workflows and iterate faster. LangGraph Studio integrates with LangSmith so you can collaborate with teammates to debug failure modes LangGraph Studio is available for free to all LangSmith users on any plan tier during its early development. Read more about it here: Watch a YouTube walkthrough: Try out LangGraph Studio for free here: Sign up for a LangSmith account:

🚀Announcing LangGraph Studio: The first agent IDE LangGraph Studio offers a new way to develop LLM applications by providing a specialized agent IDE that enables visualization, interaction, and debugging of complex agentic applications With visual graphs and the ability to edit state, you can better understand agent workflows and iterate faster. LangGraph Studio integrates with LangSmith so you can collaborate with teammates to debug failure modes LangGraph Studio is available for free to all LangSmith users on any plan tier during its early development. Read more about it here: Watch a YouTube walkthrough: Try out LangGraph Studio for free here: Sign up for a LangSmith account:

LangChain

185,873 görüntüleme • 2 yıl önce

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

🚨New Episode Drop!🚨 🧠 AI Research Lab - Explained: The Future is Multimodal You text, share photos, record videos—seamlessly switching between data types. Why can't AI? Our Salesforce AI team builds multimodal systems that understand text, images, audio, and video simultaneously—just like humans. Real applications: ➡️ Visual web interaction (clicking, filling forms) ➡️ Advanced cross-image pattern recognition ➡️ Future robots that see, hear, and communicate Featuring our XGEN-MM (BLIP) model—a breakthrough in visual language understanding. Real intelligence isn't one-dimensional. Neither should AI be. #AIResearch #MultimodalAI #Innovation

Salesforce AI Research

13,838 görüntüleme • 1 yıl önce

StarVector is out on Hugging Face StarVector is a foundation model for generating Scalable Vector Graphics (SVG) code from images and text. It utilizes a Vision-Language Modeling architecture to understand both visual and textual inputs, enabling high-quality vectorization and text-guided SVG creation.

StarVector is out on Hugging Face StarVector is a foundation model for generating Scalable Vector Graphics (SVG) code from images and text. It utilizes a Vision-Language Modeling architecture to understand both visual and textual inputs, enabling high-quality vectorization and text-guided SVG creation.

AK

254,342 görüntüleme • 1 yıl önce

🎥Meet Lucy Edit LIVE! The first REAL-TIME video editing model. It feels surreal to play with and is already changing how I think about AI video. And YOU CAN TRY it now in our platform @Decart →

🎥Meet Lucy Edit LIVE! The first REAL-TIME video editing model. It feels surreal to play with and is already changing how I think about AI video. And YOU CAN TRY it now in our platform @Decart →

Kfir Aberman

15,055 görüntüleme • 9 ay önce

Excited about the launch of Amazon Nova Sonic, our new speech-to-speech model that helps make AI voice applications feel remarkably natural. It's designed to understand not just what people say, but how they say it – working with tone, style, and conversation flow including pauses and interruptions. Nova Sonic delivers speech understanding and generation through a single, unified model, making it easier for builders to develop voice applications that maintain important context and nuance for customer service, AI agents, and other use cases across industries. It’s available in Amazon Bedrock now. Look forward to seeing what teams build with Nova Sonic!

Excited about the launch of Amazon Nova Sonic, our new speech-to-speech model that helps make AI voice applications feel remarkably natural. It's designed to understand not just what people say, but how they say it – working with tone, style, and conversation flow including pauses and interruptions. Nova Sonic delivers speech understanding and generation through a single, unified model, making it easier for builders to develop voice applications that maintain important context and nuance for customer service, AI agents, and other use cases across industries. It’s available in Amazon Bedrock now. Look forward to seeing what teams build with Nova Sonic!

Andy Jassy

155,772 görüntüleme • 1 yıl önce

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Meta Newsroom

204,592 görüntüleme • 7 ay önce

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Rohan Paul

15,672 görüntüleme • 4 ay önce

Exciting News from Open-Sora! 🚀 They've just made the ENTIRE suite of their video-generation model open source! Dive into the world of cutting-edge AI with access to model weights, comprehensive training source code, and detailed architecture insights. Start building your dream video-generation model today! Check it out 👉

Exciting News from Open-Sora! 🚀 They've just made the ENTIRE suite of their video-generation model open source! Dive into the world of cutting-edge AI with access to model weights, comprehensive training source code, and detailed architecture insights. Start building your dream video-generation model today! Check it out 👉

Yang You

245,742 görüntüleme • 2 yıl önce

Vidu S1, the World’s Most Advanced Model for Real-Time AI Interaction Vidu S1 introduces a new era of real-time interactive video. Voice-Driven Character Control: Go beyond voice-driven lip sync with real-time behavior generation. Characters understand, react, and respond naturally. Unlimited Real-Time Generation: Experience seamless real-time video generation with unlimited interaction duration. 540P at 25 FPS, built for live interaction: Generate video in real time at 540P / 25 FPS, with support for up to 42 FPS for even smoother experiences. Create Custom Characters from Images & Voices: Customize both appearance and voice. Bring humans, anime characters, pets, and more to life with personalized AI characters. Use code VIDUS1 to get started for free.

Vidu S1, the World’s Most Advanced Model for Real-Time AI Interaction Vidu S1 introduces a new era of real-time interactive video. Voice-Driven Character Control: Go beyond voice-driven lip sync with real-time behavior generation. Characters understand, react, and respond naturally. Unlimited Real-Time Generation: Experience seamless real-time video generation with unlimited interaction duration. 540P at 25 FPS, built for live interaction: Generate video in real time at 540P / 25 FPS, with support for up to 42 FPS for even smoother experiences. Create Custom Characters from Images & Voices: Customize both appearance and voice. Bring humans, anime characters, pets, and more to life with personalized AI characters. Use code VIDUS1 to get started for free.

Vidu AI

1,529,413 görüntüleme • 1 ay önce