Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Check out mistral.​rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as...

ProfBuehlerMIT's profile picture

Markus J. Buehler

20,454 subscribers

73,575 Aufrufe • vor 2 Jahren •via X (Twitter)

BildungWissenschaft & Technologie#Rust#LLM#Candle#Rust
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von nicoarq
nicoarqvor 2 Jahren

@huggingface This is really rly cool! I thought it was only limited to Mistral related LLMs though maybe it's not too late to change the name to something generic to not confuse people?

Profilbild von Markus J. Buehler
Markus J. Buehlervor 2 Jahren

@huggingface Yes, the tool with with many architectures, including Llama-3, Phi-3 (up to 128k context), Mistral/Mixtral, Gemma, X-LoRA mixture-of-expert (MoE) models, & many others. Also features in-situ quantization to any level.

Profilbild von Ingo Villnow (DM5DK) 🇺🇦🌻
Ingo Villnow (DM5DK) 🇺🇦🌻vor 2 Jahren

@huggingface Awesome! More apps like this written in Rust 😍

Profilbild von Jonathan Eisenzopf
Jonathan Eisenzopfvor 2 Jahren

@huggingface This looks really cool. I’ve been trying to figure out optimal branching in rust candle. Nice job.

Profilbild von Thomas Wolf
Thomas Wolfvor 2 Jahren

@huggingface wow!

Profilbild von PΔBLØ ᄃΞ
PΔBLØ ᄃΞvor 2 Jahren

@huggingface 🦀

Profilbild von Alexander Ocsa
Alexander Ocsavor 2 Jahren

@huggingface Does it support GPU hardware acceleration ?

Profilbild von Nicolas Patry
Nicolas Patryvor 2 Jahren

@huggingface Impressive work !

Profilbild von Markus J. Buehler
Markus J. Buehlervor 2 Jahren

@huggingface Thank you!

Profilbild von iandanforth 🦋 @iandanforth.bsky.social
iandanforth 🦋 @iandanforth.bsky.socialvor 2 Jahren

@_philschmid @huggingface Is this affiliated with Mistral AI? The name certainly implies it.

Ähnliche Videos

I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Wrote the full transformer architecture, and BPE tokenizer from scratch. The framework features: - Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x increased throughput - Automatic WebGPU fallback for non-NVIDIA devices - TypeScript API with Rust compute backend - One npm install to get started, prebuilt binaries for every platform Try out the model for yourself: Built with Reese Chong. Check out the repos and blog if you want to learn more. Shoutout to Modal for the compute credits allowing me to train on 2 A100 GPUs without going broke cc sunny madra Gavin
2:12

I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Wrote the full transformer architecture, and BPE tokenizer from scratch. The framework features: - Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x increased throughput - Automatic WebGPU fallback for non-NVIDIA devices - TypeScript API with Rust compute backend - One npm install to get started, prebuilt binaries for every platform Try out the model for yourself: Built with Reese Chong. Check out the repos and blog if you want to learn more. Shoutout to Modal for the compute credits allowing me to train on 2 A100 GPUs without going broke cc sunny madra Gavin

Aadi Kulshrestha

808,841 Aufrufe • vor 1 Monat

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:
1:55

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

Andrew Ng

386,934 Aufrufe • vor 2 Jahren

Download the latest ChatRTX update for new features: ✅ More LLM Models ✅ Image Search ✅ Voice Support Learn more in this week's #AIDecoded blog 👉
0:59

Download the latest ChatRTX update for new features: ✅ More LLM Models ✅ Image Search ✅ Voice Support Learn more in this week's #AIDecoded blog 👉

NVIDIA GeForce

98,671 Aufrufe • vor 2 Jahren

3. Agent factory: Foundry is the complete app platform for building apps and agents. We are adding support for more models from Grok, Hugging Face, Meta, Mistral, and more. Plus: Agentic retrieval in Azure AI Search, Foundry Agent Service, integration with Copilot Studio, and more. And we are ensuring the tools you already use for identity, management, and security will now all extend to agents too. The NFL has already used Foundry, along with our data stack, to run its recent Combine.
1:41

3. Agent factory: Foundry is the complete app platform for building apps and agents. We are adding support for more models from Grok, Hugging Face, Meta, Mistral, and more. Plus: Agentic retrieval in Azure AI Search, Foundry Agent Service, integration with Copilot Studio, and more. And we are ensuring the tools you already use for identity, management, and security will now all extend to agents too. The NFL has already used Foundry, along with our data stack, to run its recent Combine.

Satya Nadella

180,396 Aufrufe • vor 1 Jahr

This Friday, please get out and vote and please vote #YesYes. ✅ To make the definition of family more inclusive. ✅ To remove archaic language on a woman's duties in the home. ✅ To recognise care as a value in our constitution for the first time.
0:56

This Friday, please get out and vote and please vote #YesYes. ✅ To make the definition of family more inclusive. ✅ To remove archaic language on a woman's duties in the home. ✅ To recognise care as a value in our constitution for the first time.

Simon Harris TD

34,650 Aufrufe • vor 2 Jahren

Enhance and accelerate your video editing workflow with GeForce RTX 50 Series GPUs. 🎥 ✅ Faster encoding & decoding ✅ Support for 4:2:2 video ✅ Accelerated AI effects & generative AI tools ✅ More Check it out. 🎬
2:08

Enhance and accelerate your video editing workflow with GeForce RTX 50 Series GPUs. 🎥 ✅ Faster encoding & decoding ✅ Support for 4:2:2 video ✅ Accelerated AI effects & generative AI tools ✅ More Check it out. 🎬

NVIDIA Studio

142,073 Aufrufe • vor 1 Jahr

#Developers : Unlock seamless exploration with our #API Playground—where you can test and experiment with features without altering your codebase. ✅ Real-time testing ✅ No commitment ✅ Multi-language support Drop a comment below to explore and test it today. #onlinepayment
0:42

#Developers : Unlock seamless exploration with our #API Playground—where you can test and experiment with features without altering your codebase. ✅ Real-time testing ✅ No commitment ✅ Multi-language support Drop a comment below to explore and test it today. #onlinepayment

Pine Labs Online

1,083,518 Aufrufe • vor 1 Jahr

It's time for an upgrade... welcome to the new ICP dashboard ✅ Greater ecosystem visibility ✅ Dive deeper into the data ✅ Easily compare subnets ✅ Better UI and there's more on the way.
0:29

It's time for an upgrade... welcome to the new ICP dashboard ✅ Greater ecosystem visibility ✅ Dive deeper into the data ✅ Easily compare subnets ✅ Better UI and there's more on the way.

DFINITY Foundation

23,489 Aufrufe • vor 1 Jahr

Introducing Gamma API: a connector to automate your workflows. It's the best way to seamlessly integrate Gamma with your existing tools. Our new API plugs into Zapier, Make, or any data source to generate decks, reports, and carousels straight from your workflows. With our Gamma API, you can go from: ✅ CRM to sales deck ✅ Blog to LinkedIn carousel ✅ Research to summary deck ✅ Job link to onboarding deck And many more. Gamma API is now live.
1:10

Introducing Gamma API: a connector to automate your workflows. It's the best way to seamlessly integrate Gamma with your existing tools. Our new API plugs into Zapier, Make, or any data source to generate decks, reports, and carousels straight from your workflows. With our Gamma API, you can go from: ✅ CRM to sales deck ✅ Blog to LinkedIn carousel ✅ Research to summary deck ✅ Job link to onboarding deck And many more. Gamma API is now live.

Gamma

12,294 Aufrufe • vor 8 Monaten

Hugging Face is NUTS! A high-performance robotic arm for $115 😱 😱😱 [ DIY at home ⬇ ] Meet the game-changer: SO-100 At 1/3 the cost and 2x the capabilities of their previous arms! Fully integrated with LeRobot: ✅ Go-to for tutorials ✅ Datasets, and ✅ Pre-trained models — all in Python Github: Discord: Now is the best time to get into AI for robotics! Order your parts now! Crafted by Hugging Face & Rob Knight by Rob Knight! And kudos to Remi Cadene and his team for bringing robotics to the people! • • • • • • • --------------------------------------------------- 🚀 Scaling Deep Tech Startups:
0:29

Hugging Face is NUTS! A high-performance robotic arm for $115 😱 😱😱 [ DIY at home ⬇ ] Meet the game-changer: SO-100 At 1/3 the cost and 2x the capabilities of their previous arms! Fully integrated with LeRobot: ✅ Go-to for tutorials ✅ Datasets, and ✅ Pre-trained models — all in Python Github: Discord: Now is the best time to get into AI for robotics! Order your parts now! Crafted by Hugging Face & Rob Knight by Rob Knight! And kudos to Remi Cadene and his team for bringing robotics to the people! • • • • • • • --------------------------------------------------- 🚀 Scaling Deep Tech Startups:

Ilir Aliu

27,503 Aufrufe • vor 1 Jahr

What a year! ✅ A record launch year for Electron and HASTE with 21 launches and 100% mission success. ✅Opened Launch Complex 3. ✅Qualified Neutron's Hungry Hippo Fairing and second stage for flight. ✅ ESCAPADE is on the way to Mars. ✅Welcomed Geost to the Rocket Lab family. ✅Awarded $816M contract by SDA to produce more satellites for the PWSA, our largest contract ever. ✅ + so much more. We’re ready to make 2026 even bigger.
2:08

What a year! ✅ A record launch year for Electron and HASTE with 21 launches and 100% mission success. ✅Opened Launch Complex 3. ✅Qualified Neutron's Hungry Hippo Fairing and second stage for flight. ✅ ESCAPADE is on the way to Mars. ✅Welcomed Geost to the Rocket Lab family. ✅Awarded $816M contract by SDA to produce more satellites for the PWSA, our largest contract ever. ✅ + so much more. We’re ready to make 2026 even bigger.

Rocket Lab

176,272 Aufrufe • vor 5 Monaten

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:
5:07

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:

LlamaIndex 🦙

143,087 Aufrufe • vor 2 Jahren

It's been a good H1 at M&S, with growth in: ✅Customers ✅Sales ✅Market share ✅Profit ✅Investment returns However, there's still so much more to do and so much opportunity for future growth - and this energises us! A big thank you to our customers for shopping with us🙏
2:06

It's been a good H1 at M&S, with growth in: ✅Customers ✅Sales ✅Market share ✅Profit ✅Investment returns However, there's still so much more to do and so much opportunity for future growth - and this energises us! A big thank you to our customers for shopping with us🙏

Stuart Machin

10,914 Aufrufe • vor 1 Jahr

The Bpay Airdrop System is Now Stronger with $USDT Support! Thanks to our partnership with Gate.io You can now easily deposit USDT into Bpay! ✅ Zero transaction fees ✅ Fast withdrawals and deposits ✅ A smoother airdrop experience Choose Bpay to maximize earning opportunities in the #Web3 world! Start now and explore the benefits! You can register here ;
1:06

The Bpay Airdrop System is Now Stronger with $USDT Support! Thanks to our partnership with Gate.io You can now easily deposit USDT into Bpay! ✅ Zero transaction fees ✅ Fast withdrawals and deposits ✅ A smoother airdrop experience Choose Bpay to maximize earning opportunities in the #Web3 world! Start now and explore the benefits! You can register here ;

Bpay ｜Bountypay Official

44,313 Aufrufe • vor 1 Jahr

Are you ready for the Revolution? DeSo's new Proof of Stake consensus mechanism is coming this month. Here's what you can expect: ✅ Fast Finality ✅ Maximum Deflation ✅ Up to 20% APY Staking Rewards Check out this video to learn more about Revolution PoS and for information on an upcoming airdrop related to our new app 🪂
4:00

Are you ready for the Revolution? DeSo's new Proof of Stake consensus mechanism is coming this month. Here's what you can expect: ✅ Fast Finality ✅ Maximum Deflation ✅ Up to 20% APY Staking Rewards Check out this video to learn more about Revolution PoS and for information on an upcoming airdrop related to our new app 🪂

DeSo

19,467 Aufrufe • vor 2 Jahren

💥 A 450M model just beat bigger VLAs on real robot tasks, and it’s 100% open source [📍 bookmark for later] Came across SmolVLA, a new vision-language-action model for robotics that’s compact, fast, and trained entirely on open community datasets from LeRobot via Hugging Face. What stood out to me is how it matches or outperforms much larger models like ACT using noisy, real-world community data instead of giant private datasets. Why it’s worth a look ✅ 26% performance boost from pretraining on open-source data ✅ Runs on consumer hardware, even a MacBook ✅ 30% faster responses with async inference and smart architecture tweaks ✅ Strong results across Meta-World, LIBERO, SO100, and SO101 ✅ Fully open source: weights, code, training pipeline, eval stack They also introduced smart efficiency tricks like using fewer visual tokens, pulling outputs from mid-layer, and separating perception from action to make it all run fast. SmolVLA is a strong case for what can happen when the robotics community shares data and builds in the open. Definitely worth keeping an eye on.
0:21

💥 A 450M model just beat bigger VLAs on real robot tasks, and it’s 100% open source [📍 bookmark for later] Came across SmolVLA, a new vision-language-action model for robotics that’s compact, fast, and trained entirely on open community datasets from LeRobot via Hugging Face. What stood out to me is how it matches or outperforms much larger models like ACT using noisy, real-world community data instead of giant private datasets. Why it’s worth a look ✅ 26% performance boost from pretraining on open-source data ✅ Runs on consumer hardware, even a MacBook ✅ 30% faster responses with async inference and smart architecture tweaks ✅ Strong results across Meta-World, LIBERO, SO100, and SO101 ✅ Fully open source: weights, code, training pipeline, eval stack They also introduced smart efficiency tricks like using fewer visual tokens, pulling outputs from mid-layer, and separating perception from action to make it all run fast. SmolVLA is a strong case for what can happen when the robotics community shares data and builds in the open. Definitely worth keeping an eye on.

Ilir Aliu - eu/acc

17,353 Aufrufe • vor 9 Monaten

As part of our new Made in America Manufacturing Initiative, I just announced that the SBA will: ✅Cut $100B in red tape ✅Increase access to capital ✅Promote workforce development ✅Support President Trump’s economic agenda, including fair trade and tax cuts for manufacturers
0:31

As part of our new Made in America Manufacturing Initiative, I just announced that the SBA will: ✅Cut $100B in red tape ✅Increase access to capital ✅Promote workforce development ✅Support President Trump’s economic agenda, including fair trade and tax cuts for manufacturers

Kelly Loeffler

12,820 Aufrufe • vor 1 Jahr

📣Introducing the portable #exoskeleton controller #myControllerS570! 🤖Perfect for motion data collection and remote #teleoperation, it’s a flexible and cost-effective solution for #EmbodiedIntelligence technology. Key Features: ✅ 2-inch display for real-time feedback. ✅ Connect via USB, Wireless, or Bluetooth. ✅ Compatible with #Python and #ROS. ✅ 12 DOF for precise human arm motion simulation. Applications: ✅ #Research and Development. ✅ #Education and Training. 🎓 ✅ #Robot Remote Teleoperation. 📡 ✨ Special Offer: The first 20 controllers will get a 30% discount! 🚀Get yours👉 #AI #Robotics #TechNews #Christmas
1:33

📣Introducing the portable #exoskeleton controller #myControllerS570! 🤖Perfect for motion data collection and remote #teleoperation, it’s a flexible and cost-effective solution for #EmbodiedIntelligence technology. Key Features: ✅ 2-inch display for real-time feedback. ✅ Connect via USB, Wireless, or Bluetooth. ✅ Compatible with #Python and #ROS. ✅ 12 DOF for precise human arm motion simulation. Applications: ✅ #Research and Development. ✅ #Education and Training. 🎓 ✅ #Robot Remote Teleoperation. 📡 ✨ Special Offer: The first 20 controllers will get a 30% discount! 🚀Get yours👉 #AI #Robotics #TechNews #Christmas

Elephant Robotics - Robotic Arms

20,383 Aufrufe • vor 1 Jahr

✨ Meet our new open family of models: NVIDIA Nemotron 3 Open in weights, data, tools, and training, Nemotron 3 is built for multi-agent apps and features: • An efficient hybrid Mamba‑Transformer MoE architecture • 1M token context for long-term memory and improved reasoning • Multi‑environment reinforcement learning via NeMo Gym for advanced skill adaptation Plus NVFP4 pre-training, latent MoE, 1T tokens of data, and more. 📗Read the details in our tech blog: 🤗 Try the model on Hugging Face:
1:35

✨ Meet our new open family of models: NVIDIA Nemotron 3 Open in weights, data, tools, and training, Nemotron 3 is built for multi-agent apps and features: • An efficient hybrid Mamba‑Transformer MoE architecture • 1M token context for long-term memory and improved reasoning • Multi‑environment reinforcement learning via NeMo Gym for advanced skill adaptation Plus NVFP4 pre-training, latent MoE, 1T tokens of data, and more. 📗Read the details in our tech blog: 🤗 Try the model on Hugging Face:

NVIDIA AI Developer

68,500 Aufrufe • vor 6 Monaten

Introducing QuickPush 2.0 🚀 A lightweight macOS menu bar utility for testing and debugging push notifications and Live Activities ✅ Expo Notifications ✅ Live Activities support (start, update, end) ✅ Native APNs support ✅ FCM support ✅ Save & reuse tokens ✅ Copy cURL commands ✅ View response JSON - APNs/FCM/Expo responses ✅ New keyboard shortcuts ✅ Check receipts ✅ Open Source
2:52

Introducing QuickPush 2.0 🚀 A lightweight macOS menu bar utility for testing and debugging push notifications and Live Activities ✅ Expo Notifications ✅ Live Activities support (start, update, end) ✅ Native APNs support ✅ FCM support ✅ Save & reuse tokens ✅ Copy cURL commands ✅ View response JSON - APNs/FCM/Expo responses ✅ New keyboard shortcuts ✅ Check receipts ✅ Open Source

Beto

36,291 Aufrufe • vor 3 Monaten