Andi Marafioti's banner

Andi Marafioti

@andimarafioti • 7,618 subscribers

leading multimodal research @huggingface (prev @unity)

Shorts

Can a VLM see without a vision encoder? We trained one for $100, inspired by Gemma 4 12B. Latency on an M3 Pro MacBook: 112 ms -> 1.1 ms for the image path 30% lower end-to-end image+LLM The architecture is just: patchify the image -> linear projection with pos embeddings -> LLM Writeup:

Can a VLM see without a vision encoder? We trained one for $100, inspired by Gemma 4 12B. Latency on an M3 Pro MacBook: 112 ms -> 1.1 ms for the image path 30% lower end-to-end image+LLM The architecture is just: patchify the image -> linear projection with pos embeddings -> LLM Writeup:

60,106 Aufrufe

ok who put Hugging Face in Rick and Morty

ok who put Hugging Face in Rick and Morty

47,931 Aufrufe

Real-time SmolVLM in a web-browser with transformers.js. All running locally with no installs. Just open the website.

Real-time SmolVLM in a web-browser with transformers.js. All running locally with no installs. Just open the website.

86,299 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Talk to Gemma 4 31B with our voice app! It sees and searches the web faster than you blink. Thanks to Cerebras’ ultra fast inference, the LLM is almost instantaneous. The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. Demo:

Talk to Gemma 4 31B with our voice app! It sees and searches the web faster than you blink. Thanks to Cerebras’ ultra fast inference, the LLM is almost instantaneous. The whole stack is fully open-source, and is a drop-in replacement for OpenAI's realtime API. Demo:

88,523 Aufrufe • vor 19 Tagen

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

Introducing local Reachy Mini conversations: free chats forever! So fast that we had to hardcode delays to stop it from interrupting you mid-sentence. We built an open-source Realtime API powered by llama.cpp: Parakeet -> Gemma 4 E4B -> Qwen3TTS Run it anywhere you run local LMs. Video shows DGX Spark and a 36GB M3 Pro MacBook. Blog:

76,150 Aufrufe • vor 1 Monat

Reachy Mini just got a new Brain! We released a fully open-source backend for talking to Reachy Mini. In the last 48 hours, 3,000+ robots have already hit the version deployed on Hugging Face's infra. Until today, if you wanted a real-time voice agent in your robot, you were looking at $20+ a day. Realtime APIs are expensive because every second the connection is up, you are being charged! But with our approach, you can run the audio models locally and use your OpenAI or Claude subscription for the LLM, at no extra costs. The audio pipeline is very efficient; in the video, it's running on the cheapest Mac mini, 16GB of RAM (the one that sold out a couple of months back when people were buying them up for openclaws setups). Tutorials coming in the next few weeks: how to run your robot fully locally, or with your favorite agent. Stay tuned!

Reachy Mini just got a new Brain! We released a fully open-source backend for talking to Reachy Mini. In the last 48 hours, 3,000+ robots have already hit the version deployed on Hugging Face's infra. Until today, if you wanted a real-time voice agent in your robot, you were looking at $20+ a day. Realtime APIs are expensive because every second the connection is up, you are being charged! But with our approach, you can run the audio models locally and use your OpenAI or Claude subscription for the LLM, at no extra costs. The audio pipeline is very efficient; in the video, it's running on the cheapest Mac mini, 16GB of RAM (the one that sold out a couple of months back when people were buying them up for openclaws setups). Tutorials coming in the next few weeks: how to run your robot fully locally, or with your favorite agent. Stay tuned!

55,975 Aufrufe • vor 2 Monaten

30x real-time speech-to-text in your browser. No installs. No servers. Just open the website.

30x real-time speech-to-text in your browser. No installs. No servers. Just open the website.

105,122 Aufrufe • vor 5 Monaten

Introducing Faster Qwen3TTS! Realistic voice generation at 4x real time: - Same amazing voice quality from Qwen's model - Streaming support with <200 ms to first audio - 5x faster than the official implementation Just pip install faster-qwen3-tts Try the demo!

Introducing Faster Qwen3TTS! Realistic voice generation at 4x real time: - Same amazing voice quality from Qwen's model - Streaming support with <200 ms to first audio - 5x faster than the official implementation Just pip install faster-qwen3-tts Try the demo!

92,135 Aufrufe • vor 4 Monaten

Mixing vision and robotics is incredibly hard, but when it finally works it feels like magic.

Mixing vision and robotics is incredibly hard, but when it finally works it feels like magic.

92,881 Aufrufe • vor 8 Monaten

I got NVIDIA’s PersonaPlex running on Reachy Mini and it’s uncanny. Ultra-low latency changes everything for voice agents. BTW, NVIDIA is offering a ~$7k prize (including a DGX Spark!) for the best Reachy Mini community app in the next 15 days.

I got NVIDIA’s PersonaPlex running on Reachy Mini and it’s uncanny. Ultra-low latency changes everything for voice agents. BTW, NVIDIA is offering a ~$7k prize (including a DGX Spark!) for the best Reachy Mini community app in the next 15 days.

54,356 Aufrufe • vor 5 Monaten

Sneak peek of Reachy Mini's conversation capabilities! He can chat fluidly and change languages naturally. You can interrupt him and he can see and react to his environment. They start shipping soon, I'm excited to see what people start building with him!

Sneak peek of Reachy Mini's conversation capabilities! He can chat fluidly and change languages naturally. You can interrupt him and he can see and react to his environment. They start shipping soon, I'm excited to see what people start building with him!

54,430 Aufrufe • vor 8 Monaten

🚀 Today, we are introducing SmolTools! 🚀 Last week, at Hugging Face we made a significant leap forward with the release of SmolLM2, a compact 1.7B language model that sets a new benchmark for performance among models of its size. But beyond the impressive stats, SmolLM2 truly shines in practical, real-world applications, unlocking new possibilities for on-device inference. To demonstrate its potential, we're thrilled to introduce Smol-Tools, a suite of simple yet powerful applications that showcase the capabilities of SmolLM2. 🌟 Today, we present you two key tools: Summarize and Rewrite. 🔹 Summarize: Feed SmolLM2 a text up to 20 pages long, and it will provide you with a concise summary. Need to dig deeper? Just ask follow-up questions to clarify any details – it's that simple. 🔹 Rewrite: Draft a response to a message with your main points, and SmolLM2 will transform it into a clear, polished version that's easy for your recipient to read and understand. These tools are designed to make your workflow smoother and more efficient, leveraging the power of SmolLM2 in practical ways. 💡We hope you give these tools a try and see their potential for yourself! Feel free to build your own SmolTools and contribute them to the project. We'd love to see what you build! Check out the code here:

🚀 Today, we are introducing SmolTools! 🚀 Last week, at Hugging Face we made a significant leap forward with the release of SmolLM2, a compact 1.7B language model that sets a new benchmark for performance among models of its size. But beyond the impressive stats, SmolLM2 truly shines in practical, real-world applications, unlocking new possibilities for on-device inference. To demonstrate its potential, we're thrilled to introduce Smol-Tools, a suite of simple yet powerful applications that showcase the capabilities of SmolLM2. 🌟 Today, we present you two key tools: Summarize and Rewrite. 🔹 Summarize: Feed SmolLM2 a text up to 20 pages long, and it will provide you with a concise summary. Need to dig deeper? Just ask follow-up questions to clarify any details – it's that simple. 🔹 Rewrite: Draft a response to a message with your main points, and SmolLM2 will transform it into a clear, polished version that's easy for your recipient to read and understand. These tools are designed to make your workflow smoother and more efficient, leveraging the power of SmolLM2 in practical ways. 💡We hope you give these tools a try and see their potential for yourself! Feel free to build your own SmolTools and contribute them to the project. We'd love to see what you build! Check out the code here:

100,813 Aufrufe • vor 1 Jahr

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

49,888 Aufrufe • vor 8 Monaten

🚨 New paper out! “FineVision: Open Data Is All You Need” 🥳 We unified 200+ data sources into 24M samples. That’s 17.3M images and 9.5B answer tokens, the largest open VLM dataset ever released. All fully documented, reproducible, and available for everyone. And there's more! 🎢

🚨 New paper out! “FineVision: Open Data Is All You Need” 🥳 We unified 200+ data sources into 24M samples. That’s 17.3M images and 9.5B answer tokens, the largest open VLM dataset ever released. All fully documented, reproducible, and available for everyone. And there's more! 🎢

46,481 Aufrufe • vor 9 Monaten

The Reachy mini desktop app is so aesthetically pleasing. Huge fan.

The Reachy mini desktop app is so aesthetically pleasing. Huge fan.

34,449 Aufrufe • vor 7 Monaten

This is getting crazy good. Running locally on my laptop with streaming transcriptions without compromising quality.

This is getting crazy good. Running locally on my laptop with streaming transcriptions without compromising quality.

24,568 Aufrufe • vor 5 Monaten

Live demos are risky, but so worth it! With Gradium, we plugged their conversational demo into Reachy Mini. The result? ✅ Personality switching (Gym Bro mode!🏋️‍♂️) ✅ Multilingual (Québécois accent 🇨🇦) ✅ Dancing/emotions on command 💃 Reachy comes alive. Unscripted demo👇

Live demos are risky, but so worth it! With Gradium, we plugged their conversational demo into Reachy Mini. The result? ✅ Personality switching (Gym Bro mode!🏋️‍♂️) ✅ Multilingual (Québécois accent 🇨🇦) ✅ Dancing/emotions on command 💃 Reachy comes alive. Unscripted demo👇

15,165 Aufrufe • vor 7 Monaten

Keine weiteren Inhalte verfügbar