Loading video...
Video Failed to Load
Check out mistral.rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as... show more
10 Comments
@huggingface This is really rly cool! I thought it was only limited to Mistral related LLMs though maybe it's not too late to change the name to something generic to not confuse people?
@huggingface Yes, the tool with with many architectures, including Llama-3, Phi-3 (up to 128k context), Mistral/Mixtral, Gemma, X-LoRA mixture-of-expert (MoE) models, & many others. Also features in-situ quantization to any level.
@huggingface Awesome! More apps like this written in Rust 😍
@huggingface This looks really cool. I’ve been trying to figure out optimal branching in rust candle. Nice job.
@huggingface wow!
@huggingface 🦀
@huggingface Does it support GPU hardware acceleration ?
@huggingface Impressive work !
@huggingface Thank you!
@_philschmid @huggingface Is this affiliated with Mistral AI? The name certainly implies it.
![Hugging Face is NUTS! A high-performance robotic arm for $115 😱 😱😱 [ DIY at home ⬇ ] Meet the game-changer: SO-100 At 1/3 the cost and 2x the capabilities of their previous arms! Fully integrated with LeRobot: ✅ Go-to for tutorials ✅ Datasets, and ✅ Pre-trained models — all in Python Github: Discord: Now is the best time to get into AI for robotics! Order your parts now! Crafted by Hugging Face & Rob Knight by Rob Knight! And kudos to Remi Cadene and his team for bringing robotics to the people! • • • • • • • --------------------------------------------------- 🚀 Scaling Deep Tech Startups:](https://image.24vids.com/tw-1841552810996887563/ext_tw_video_thumb/1841552765878685696/pu/img/Q3N1yVcYVdmvojaP.jpg)
![💥 A 450M model just beat bigger VLAs on real robot tasks, and it’s 100% open source [📍 bookmark for later] Came across SmolVLA, a new vision-language-action model for robotics that’s compact, fast, and trained entirely on open community datasets from LeRobot via Hugging Face. What stood out to me is how it matches or outperforms much larger models like ACT using noisy, real-world community data instead of giant private datasets. Why it’s worth a look ✅ 26% performance boost from pretraining on open-source data ✅ Runs on consumer hardware, even a MacBook ✅ 30% faster responses with async inference and smart architecture tweaks ✅ Strong results across Meta-World, LIBERO, SO100, and SO101 ✅ Fully open source: weights, code, training pipeline, eval stack They also introduced smart efficiency tricks like using fewer visual tokens, pulling outputs from mid-layer, and separating perception from action to make it all run fast. SmolVLA is a strong case for what can happen when the robotics community shares data and builds in the open. Definitely worth keeping an eye on.](https://image.24vids.com/tw-1961486533535412392/ext_tw_video_thumb/1961486511477526528/pu/img/jFqHorQt-jLmMYqg.jpg)
