Andi Marafioti
@andimarafioti • 9,040 subscribers
leading multimodal research @huggingface (prev @unity)
Shorts
Videos
Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!
Andi Marafioti78,540 次观看 • 26 天前
Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:
Andi Marafioti77,716 次观看 • 4 个月前
Reachy Mini just got a new Brain! We released a fully open-source backend for talking to Reachy Mini. In the last 48 hours, 3,000+ robots have already hit the version deployed on Hugging Face's infra. Until today, if you wanted a real-time voice agent in your robot, you were looking at $20+ a day. Realtime APIs are expensive because every second the connection is up, you are being charged! But with our approach, you can run the audio models locally and use your OpenAI or Claude subscription for the LLM, at no extra costs. The audio pipeline is very efficient; in the video, it's running on the cheapest Mac mini, 16GB of RAM (the one that sold out a couple of months back when people were buying them up for openclaws setups). Tutorials coming in the next few weeks: how to run your robot fully locally, or with your favorite agent. Stay tuned!
Andi Marafioti56,158 次观看 • 5 个月前
🚀 Today, we are introducing SmolTools! 🚀 Last week, at Hugging Face we made a significant leap forward with the release of SmolLM2, a compact 1.7B language model that sets a new benchmark for performance among models of its size. But beyond the impressive stats, SmolLM2 truly shines in practical, real-world applications, unlocking new possibilities for on-device inference. To demonstrate its potential, we're thrilled to introduce Smol-Tools, a suite of simple yet powerful applications that showcase the capabilities of SmolLM2. 🌟 Today, we present you two key tools: Summarize and Rewrite. 🔹 Summarize: Feed SmolLM2 a text up to 20 pages long, and it will provide you with a concise summary. Need to dig deeper? Just ask follow-up questions to clarify any details – it's that simple. 🔹 Rewrite: Draft a response to a message with your main points, and SmolLM2 will transform it into a clear, polished version that's easy for your recipient to read and understand. These tools are designed to make your workflow smoother and more efficient, leveraging the power of SmolLM2 in practical ways. 💡We hope you give these tools a try and see their potential for yourself! Feel free to build your own SmolTools and contribute them to the project. We'd love to see what you build! Check out the code here:
Andi Marafioti100,813 次观看 • 1 年前
🚨 New paper out! “FineVision: Open Data Is All You Need” 🥳 We unified 200+ data sources into 24M samples. That’s 17.3M images and 9.5B answer tokens, the largest open VLM dataset ever released. All fully documented, reproducible, and available for everyone. And there's more! 🎢
Andi Marafioti46,507 次观看 • 1 年前
没有更多内容可加载