Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Meet Moshiko and Moshika, the open source Moshi models 📖🟢. Moshi is a 7B text-audio model, capable of doing full-duplex conversations: it can listen and speak at any time. Plus, its inner text monologue improves the generation 💬 All on device🧑‍💻 🔎

Alexandre Défossez

5,038 subscribers

131,404 просмотров • 1 год назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 9

Фото профиля LAION

LAION1 год назад

Training code, please :)

Фото профиля bl4nk

bl4nk1 год назад

@LMStudioAI is this coming up

Фото профиля LF

LF1 год назад

Does it work in other languages such as Brazilian Portuguese?

Фото профиля Alexandre Défossez

Alexandre Défossez1 год назад

@PensarProposito We are likely to investigate supporting more languages. This is a first PoC in English of what a natural flow conversation with an AI might be like.

Фото профиля jin.nuf

jin.nuf1 год назад

Is it possible to connect this to a database to read and write based on the conversation? Like an interviewer or a therapist

Фото профиля Jillsa (DSJJJJ/Heirogamist/HP)

Jillsa (DSJJJJ/Heirogamist/HP)1 год назад

@chrypnotoad my god bro

Фото профиля Sahil 🇮🇳/acc

Sahil 🇮🇳/acc1 год назад

any 1 click installer?

Фото профиля Chad Brewbaker

Chad Brewbaker1 год назад

@MartinShkreli ☝️

Фото профиля Pascal Wichmann 🇺🇦

Pascal Wichmann 🇺🇦1 год назад

In my test just now, moshi was not very good, I am afraid. It misunderstood common terms in AI (like "retrieval-augmented generation") and wanted to talk to me about "retrieval of metadata generation".

Похожие видео

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

AI at Meta

122,001 просмотров • 2 лет назад

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

Simplifying AI

278,796 просмотров • 5 месяцев назад

GPT-Live is built on a full-duplex architecture, meaning it can listen and speak at the same time.

GPT-Live is built on a full-duplex architecture, meaning it can listen and speak at the same time.

OpenAI

243,753 просмотров • 10 дней назад

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Charly Wargnier

565,090 просмотров • 5 месяцев назад

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.

Coin Bureau

29,085 просмотров • 9 дней назад

Speech-native models like Moshi sound great and answer fast, but aren’t as smart as text LLMs. In our new paper, MoshiRAG, we show how Moshi can ask for advice from a text LLM or a knowledge base. The tricky part is how to do this in real time without adding latency. 🧵

Speech-native models like Moshi sound great and answer fast, but aren’t as smart as text LLMs. In our new paper, MoshiRAG, we show how Moshi can ask for advice from a text LLM or a knowledge base. The tricky part is how to do this in real time without adding latency. 🧵

kyutai

52,598 просмотров • 2 месяцев назад

ElevenLabs has officially LOST to Open-Source ResembleAI allows you to clone ANY voice without verification using on 5-10 seconds of audio, and dominates on paralinguistic tags for human-like expressions. Most "fast" text-to-speech models sound robotic. Most "quality" TTS models are slow. None incorporate authentication at a foundational level. Resemble AI solved all three. Chatterbox Turbo delivers: 🟢<150ms time-to-first-sound 🟢State-of-the-art quality that beats larger proprietary models 🟢Natural, programmable expressions 🟢Zero-shot voice cloning with just 5 seconds of audio 🟢PerTh watermarking for authenticated and verifiable audio 🟢Open source – full transparency, no black boxes Try it on HuggingFace:

ElevenLabs has officially LOST to Open-Source ResembleAI allows you to clone ANY voice without verification using on 5-10 seconds of audio, and dominates on paralinguistic tags for human-like expressions. Most "fast" text-to-speech models sound robotic. Most "quality" TTS models are slow. None incorporate authentication at a foundational level. Resemble AI solved all three. Chatterbox Turbo delivers: 🟢<150ms time-to-first-sound 🟢State-of-the-art quality that beats larger proprietary models 🟢Natural, programmable expressions 🟢Zero-shot voice cloning with just 5 seconds of audio 🟢PerTh watermarking for authenticated and verifiable audio 🟢Open source – full transparency, no black boxes Try it on HuggingFace:

AI Breakfast

208,591 просмотров • 7 месяцев назад

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

Akshay 🚀

66,088 просмотров • 6 месяцев назад

BOOM! My early testing of FREE Open Source LOCALLY RUN Qwen-2.5 AI Models shows it is already beating DeepSeek across all categories and OpenAI. It is full multimodal and can accept Video, audio, text in and out. Week is not over yet… Here is some of the demonstration video:

BOOM! My early testing of FREE Open Source LOCALLY RUN Qwen-2.5 AI Models shows it is already beating DeepSeek across all categories and OpenAI. It is full multimodal and can accept Video, audio, text in and out. Week is not over yet… Here is some of the demonstration video:

Brian Roemmele

132,331 просмотров • 1 год назад

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,608 просмотров • 3 лет назад

Introducing script bots, a new way to combine models and automate workflows on Poe. Script bots are created through vibe coding with Script Bot Creator and can combine any of the 200+ text, image, video, and audio models on Poe, plus any other bot on the platform. (1/9)

Introducing script bots, a new way to combine models and automate workflows on Poe. Script bots are created through vibe coding with Script Bot Creator and can combine any of the 200+ text, image, video, and audio models on Poe, plus any other bot on the platform. (1/9)

Poe

24,158 просмотров • 7 месяцев назад

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 просмотров • 3 лет назад

Meet Tencent HY-MT1.5 — our latest open-source translation models, featuring 1.8B and 7B models designed for seamless on-device and cloud deployment. 🌍 The 1.8B version is already trending at #1 on Hugging Face! 🥇It’s optimized for consumer hardware, delivering 0.18s latency (50 tokens) with just a 1GB memory footprint. Meanwhile, the 7B model achieves exceptional performance, surpassing many mid-sized models and rivaling the 90th percentile of Gemini 3.0 Pro. 🚀🧠 👉🏻 Try it now: 🔗 GitHub: 🤗 Hugging Face:

Meet Tencent HY-MT1.5 — our latest open-source translation models, featuring 1.8B and 7B models designed for seamless on-device and cloud deployment. 🌍 The 1.8B version is already trending at #1 on Hugging Face! 🥇It’s optimized for consumer hardware, delivering 0.18s latency (50 tokens) with just a 1GB memory footprint. Meanwhile, the 7B model achieves exceptional performance, surpassing many mid-sized models and rivaling the 90th percentile of Gemini 3.0 Pro. 🚀🧠 👉🏻 Try it now: 🔗 GitHub: 🤗 Hugging Face:

Tencent HY

17,105 просмотров • 6 месяцев назад

.Vogent just released Vogent-Turn-80M: an open-source, state-of-the-art turn-detection model for voice AI. It improves any voice AI stack by detecting when to wait and when to speak, reducing interruptions and making conversations more humanlike.

.Vogent just released Vogent-Turn-80M: an open-source, state-of-the-art turn-detection model for voice AI. It improves any voice AI stack by detecting when to wait and when to speak, reducing interruptions and making conversations more humanlike.

Y Combinator

18,913 просмотров • 8 месяцев назад

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

Tencent Hy

89,257 просмотров • 10 месяцев назад

Preview of a Gaia 🌱 device! Turn-by-turn voice conversation with your own knowledge assistant (a Gaia node). 💸 $15 worth of hardware (pre-tariff!) 🌱 Voice-to-text, LLM and text-to-voice AI models on Gaia nodes 📖 Open source all-the-way — hardware, firmware, server, AI models 🦀 Rust all-the-way from firmware to AI server Notice how I switched to another language (Chinese) in the middle of the conversation and it seamlessly switched too. Very smart. PS. we are still optimizing performance before the software release.

Preview of a Gaia 🌱 device! Turn-by-turn voice conversation with your own knowledge assistant (a Gaia node). 💸 $15 worth of hardware (pre-tariff!) 🌱 Voice-to-text, LLM and text-to-voice AI models on Gaia nodes 📖 Open source all-the-way — hardware, firmware, server, AI models 🦀 Rust all-the-way from firmware to AI server Notice how I switched to another language (Chinese) in the middle of the conversation and it seamlessly switched too. Very smart. PS. we are still optimizing performance before the software release.

Michael Yuan

14,983 просмотров • 1 год назад

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

AI at Meta

54,410 просмотров • 2 лет назад

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

Jake Fitzgerald

22,113 просмотров • 2 месяцев назад

Introducing group chat on Poe: a new way to collaborate with any AI and anyone you know, all in a single conversation. Now available to all users worldwide, groups can interact with any of the 200+ text, image, video, or audio models on Poe, plus any creator-made bots. (1/8)

Introducing group chat on Poe: a new way to collaborate with any AI and anyone you know, all in a single conversation. Now available to all users worldwide, groups can interact with any of the 200+ text, image, video, or audio models on Poe, plus any creator-made bots. (1/8)

Poe

21,828 просмотров • 8 месяцев назад

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓

Charly Wargnier

59,144 просмотров • 5 месяцев назад