Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

PSA 📣: MLX can now pull Mistral/ Llama/ TinyLlama safetensors directly from the Hub! 🔥 pip install -U mlx is all you need! All mistral/ llama fine-tunes supported too! 20,000+ checkpoints overall! P.S. We also provide a script to convert and quantise checkpoints and directly ship them to the... show more

Vaibhav (VB) Srivastav

48,825 subscribers

60,608 просмотров • 2 лет назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

Here's how you can do it too: 1. git clone 2. cd llms/hf_llms 3. pip install -r requirements.txt 4. python generate. py --model mistralai/Mistral-7B-Instruct-v0.2 --prompt "What the dog doin?" That's it! 🤗

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

You can also quantise the models and directly upload them to the Hub ⚡ python convert. py --hf-path "codellama/CodeLlama-7b-Python-hf" -q --upload-name CodeLlama-7b-Python-hf-4bit-mlx

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

Here's how the converted checkpoint would look like:

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

Always a pleasure working w/ @awnihannun & @pcuenq ❤️

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

Help us quantise more models! ❤️ python convert. py --hf-path "codellama/CodeLlama-13b-Python-hf" -q --upload-name CodeLlama-13b-Python-hf-4bit-mlx

Фото профиля Krum Arnaudov

Krum Arnaudov2 лет назад

What would be the min HW requirements to run a 7B 4bit model? Wondering if the 16GB M1 would be enough to get anything running.

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

Should be more than enough, IMO. Here's how you can test it: python generate. py --model mlx-community/Mistral-7B-Instruct-v0.2-4bit-mlx --prompt "Write a rap song about on-device inference via a mac, keep it classy."

Фото профиля Evil Genius

Evil Genius2 лет назад

Niceee

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

There's more coming! 🔥

Фото профиля Russ Green

Russ Green2 лет назад

Ive been meaning to try my luck running the 8x7b i only have a 4070 tho

Фото профиля Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastav2 лет назад

This might be of interest to you then:

Похожие видео

I needed an app to export markdown from images and PDFs, so I built one using the best model on the market: Mistral OCR. It can convert images, PDFs, and even extract tables and images, keeping them contextually within the markdown. All you need is a Mistral AI API.

I needed an app to export markdown from images and PDFs, so I built one using the best model on the market: Mistral OCR. It can convert images, PDFs, and even extract tables and images, keeping them contextually within the markdown. All you need is a Mistral AI API.

Pietro Schirano

48,786 просмотров • 1 год назад

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,761 просмотров • 1 год назад

$Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡$

Fuck it! You can now run any GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 просмотров • 1 год назад

With Bitrig Remote, you can now install apps directly onto your Home Screen and publish builds to TestFlight, all from the palm of your hand.

With Bitrig Remote, you can now install apps directly onto your Home Screen and publish builds to TestFlight, all from the palm of your hand.

Bitrig

38,392 просмотров • 2 месяцев назад

GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.

GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.

LeRobot

19,572 просмотров • 1 месяц назад

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

Santiago

55,609 просмотров • 2 лет назад

This is INSANE. With the new Telegram features, you can now run a Hermes Agent directly inside Telegram. All you need is some TON. And yes, it comes with AI credits too:

This is INSANE. With the new Telegram features, you can now run a Hermes Agent directly inside Telegram. All you need is some TON. And yes, it comes with AI credits too:

Antoine Rousseaux

276,046 просмотров • 3 месяцев назад

SAM-2 is apache license - you can indeed just build crazy apps with them without fear. This weekend I ported SAM-2 models to run optimally on Apple Silicon. I have also quantized all the SAM models to 16bit, 8bit, and 4bit variants (collection below) Repo is open source and pip installable. Comes with all you'll need for interactive video segmentation - - bidirectional propagation, - positive and negative clicks, - bounding boxes, - streaming masks back to user, - lightweight fast-api server that you can extend I basically asked Codex to port over the original Facebook sam2 codebase to mlx and optimize it for mlx. And it just worked. This demo for example is a 4bit base-plus model (~100MB download) To speed up, you can also do spatial or temporal downsampling. That speeds inference on my old macbook by ~3x and the hit in parity is negligible.

SAM-2 is apache license - you can indeed just build crazy apps with them without fear. This weekend I ported SAM-2 models to run optimally on Apple Silicon. I have also quantized all the SAM models to 16bit, 8bit, and 4bit variants (collection below) Repo is open source and pip installable. Comes with all you'll need for interactive video segmentation - - bidirectional propagation, - positive and negative clicks, - bounding boxes, - streaming masks back to user, - lightweight fast-api server that you can extend I basically asked Codex to port over the original Facebook sam2 codebase to mlx and optimize it for mlx. And it just worked. This demo for example is a 4bit base-plus model (~100MB download) To speed up, you can also do spatial or temporal downsampling. That speeds inference on my old macbook by ~3x and the hit in parity is negligible.

AVB

17,396 просмотров • 3 месяцев назад

Here's an AI assistant I put together with Apple Shortcuts, Cloudflare Workers, and llama 3. Shortcuts provide ASR, TTS, and HTTP requests, which is basically all we need locally. You can also tie a shortcut to the iPhone's action button and it'll be available from the lock screen. From there, I wrote a trivial Cloudflare Worker that takes the text from the shortcut, sends it to llama 3, handles function calls, and sends the response back to the shortcut. Of course, this is a very simple example and could be expanded to include more complex function calls, memory, etc. It's also faster than all the AI wearables and doesn't require any additional hardware or monthly fees. ;) Here's the code:

Here's an AI assistant I put together with Apple Shortcuts, Cloudflare Workers, and llama 3. Shortcuts provide ASR, TTS, and HTTP requests, which is basically all we need locally. You can also tie a shortcut to the iPhone's action button and it'll be available from the lock screen. From there, I wrote a trivial Cloudflare Worker that takes the text from the shortcut, sends it to llama 3, handles function calls, and sends the response back to the shortcut. Of course, this is a very simple example and could be expanded to include more complex function calls, memory, etc. It's also faster than all the AI wearables and doesn't require any additional hardware or monthly fees. ;) Here's the code:

Shayan

794,085 просмотров • 2 лет назад

Introducing Fullstack Arena! We connected Supabase backend and auth to Design Arena! You can now build projects like Notion and Google Sheets directly in Design Arena. The best models from Anthropic, Google, xAI, OpenAI, Mistral AI to start. Deterministic tests to verify correctness. LLM-as-judge to judge implementation. Leaderboard coming soon! Try it yourself at

Introducing Fullstack Arena! We connected Supabase backend and auth to Design Arena! You can now build projects like Notion and Google Sheets directly in Design Arena. The best models from Anthropic, Google, xAI, OpenAI, Mistral AI to start. Deterministic tests to verify correctness. LLM-as-judge to judge implementation. Leaderboard coming soon! Try it yourself at

Design Arena

16,411 просмотров • 6 месяцев назад

The Policy Checker Page is LIVE! Remember we talked about showing our intermediate model and success heatmap? You can now check them live on our hub! For each intermediate policy, you can: - View its summary — success rate, task type, checkpoints, and more. - Run it directly in your browser to observe its real-time inference. (If it looks a bit choppy, that's expected — the model is inferring live!) - Explore its success heatmap — showing the initial states under which the policy successfully completes the task. This gives the community a lens into our backend policy training, and lays the groundwork for better demonstrating our recover from failure training loop going forward. How to enter ⬇️

The Policy Checker Page is LIVE! Remember we talked about showing our intermediate model and success heatmap? You can now check them live on our hub! For each intermediate policy, you can: - View its summary — success rate, task type, checkpoints, and more. - Run it directly in your browser to observe its real-time inference. (If it looks a bit choppy, that's expected — the model is inferring live!) - Explore its success heatmap — showing the initial states under which the policy successfully completes the task. This gives the community a lens into our backend policy training, and lays the groundwork for better demonstrating our recover from failure training loop going forward. How to enter ⬇️

Axis Robotics

13,134 просмотров • 2 месяцев назад

Congratulations to the Cohere team on releasing North Mini Code 1.0 🔥 We provide day-0 support on MLX thanks to our close collaboration with the Cohere team. North Mini Code 1.0 is a 30B param MoE model with 3B active. It runs at ~66 tok/s in BF16, truly impressive speeds before any compression. Here is a quick demo I built using Pi by Mario Zechner & Armin Ronacher ⇌ as my agent hardness. Get started today: 1. Install mlx-vlm from source: > uv pip install git+ 2. Start the server and power your favourite agents (pi, opencode, hermes and etc) > uv run mlx_vlm.server --model CohereLabs/North-Mini-Code-1.0 New release tomorrow

Congratulations to the Cohere team on releasing North Mini Code 1.0 🔥 We provide day-0 support on MLX thanks to our close collaboration with the Cohere team. North Mini Code 1.0 is a 30B param MoE model with 3B active. It runs at ~66 tok/s in BF16, truly impressive speeds before any compression. Here is a quick demo I built using Pi by Mario Zechner & Armin Ronacher ⇌ as my agent hardness. Get started today: 1. Install mlx-vlm from source: > uv pip install git+ 2. Start the server and power your favourite agents (pi, opencode, hermes and etc) > uv run mlx_vlm.server --model CohereLabs/North-Mini-Code-1.0 New release tomorrow

Prince Canuma

35,367 просмотров • 2 месяцев назад

You can run Google new Gemma 4 on mobile easily. I am using Gemma 4 version E2B on my Pixel 10 Pro. Here is all you need to do: - Go to the App Store and install Google AI Edge Gallery. If you already have it, just update it. - From there, you can install the model directly and start using it. Still exploring what this small but powerful model can do :)

You can run Google new Gemma 4 on mobile easily. I am using Gemma 4 version E2B on my Pixel 10 Pro. Here is all you need to do: - Go to the App Store and install Google AI Edge Gallery. If you already have it, just update it. - From there, you can install the model directly and start using it. Still exploring what this small but powerful model can do :)

AshutoshShrivastava

226,599 просмотров • 4 месяцев назад

Today, we’re introducing the Sarvam MCP Server. It gives coding agents a standard way to discover and build with Sarvam. All you need to is a add a single configuration, and Sarvam becomes available directly in your coding agent. It already knows the right models and tools, so you can build with them and ship faster. Create a key, copy the config, start building -

Today, we’re introducing the Sarvam MCP Server. It gives coding agents a standard way to discover and build with Sarvam. All you need to is a add a single configuration, and Sarvam becomes available directly in your coding agent. It already knows the right models and tools, so you can build with them and ship faster. Create a key, copy the config, start building -

Sarvam

76,844 просмотров • 1 месяц назад

We redesigned Reve. Editing is easier than ever. Now all your tools are in one panel on the right. And you can make changes to anything just by clicking directly and prompting.

We redesigned Reve. Editing is easier than ever. Now all your tools are in one panel on the right. And you can make changes to anything just by clicking directly and prompting.

Reve

374,040 просмотров • 6 месяцев назад

Gemini now supports YT videos directly. - Transcribe and translate. - Summarize and rewrite for a tweet. - Extract the hooks. - Describe the setting of the video. All you need to do is go to Gemini AI Studio and put the youtube link over there.

Gemini now supports YT videos directly. - Transcribe and translate. - Summarize and rewrite for a tweet. - Extract the hooks. - Describe the setting of the video. All you need to do is go to Gemini AI Studio and put the youtube link over there.

Jean Gatt

135,671 просмотров • 1 год назад

Islam is forever changing. We now have Muslims saying Allah is a Father, a Mother, and a chair "in the correct sense", that he begot Jesus, and that you can pray to Jesus 😂😂😂most (not all) of this is derived directly from their Dawah heroes.

Islam is forever changing. We now have Muslims saying Allah is a Father, a Mother, and a chair "in the correct sense", that he begot Jesus, and that you can pray to Jesus 😂😂😂most (not all) of this is derived directly from their Dawah heroes.

GodLogic_GL

22,589 просмотров • 6 месяцев назад

Exciting update for AI developers! The Hugging Face Hub is now more natively integrated into Google Cloud Vertex AI Model Garden. Search through thousands of open Generative AI models from Hugging Face models & deploy them with one click to Vertex AI or GKE. 🤯 What's new: 🔎 Browse and search thousands of Hugging Face models directly within the Vertex AI Model Garden and filter based on what is currently trending in the community. 🚀 Accelerate your AI projects by leveraging readily available one-click deploy your model to Vertex AI or Google Kubernetes Engine (GKE). ⭐️ Featuring popular open models from BlackForestLabsAI - Unofficial FLUX.1, AI at Meta Llama 3.1, Mistral AI, Google DeepMind Gemma, and countless others. Get Started:

Exciting update for AI developers! The Hugging Face Hub is now more natively integrated into Google Cloud Vertex AI Model Garden. Search through thousands of open Generative AI models from Hugging Face models & deploy them with one click to Vertex AI or GKE. 🤯 What's new: 🔎 Browse and search thousands of Hugging Face models directly within the Vertex AI Model Garden and filter based on what is currently trending in the community. 🚀 Accelerate your AI projects by leveraging readily available one-click deploy your model to Vertex AI or Google Kubernetes Engine (GKE). ⭐️ Featuring popular open models from BlackForestLabsAI - Unofficial FLUX.1, AI at Meta Llama 3.1, Mistral AI, Google DeepMind Gemma, and countless others. Get Started:

Philipp Schmid

34,757 просмотров • 1 год назад