Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Fine-tuning Mistral 7B with LoRA on a 32 GB M1 (laptop!) in MLX Updated example uses less RAM + support for custom datasets 🚀

Awni Hannun

44,908 subscribers

148,149 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Comments

murat 🍥2 years ago

i was just about to ask if anyone's done LoRA/QLoRA on mlx yet. exciting!

Delip Rao e/σ2 years ago

can you talk to the gods that are and make MLX work on my M2 w/ 14.2 (23C63)?

Awni Hannun2 years ago

😬 It should run no prob for you! Send me your issue, we'll definitely debug

Eric Hartford2 years ago

@winglian you think axolotl could work on mlx?

Peter J. Liu2 years ago

Some performance comparisons with GPUs, e.g. 4090 would be useful.

Rahul Dave2 years ago

Guessing this needs 32! Darn it I got a 16…

Jay2 years ago

Can’t wait to couple this with @textcortex desktop app 🔩

ae2 years ago

any chance that it runs on 8G M2 macbook?

thyself Know2 years ago

I only have 16 GB RAM 😭

John Gorman2 years ago

@awnihannun @abacaj Yup - I'm be priming my Mixtral Q6 on my 32GB M1 to check out MLX ;)

Related Videos

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Awni Hannun

581,723 views • 2 years ago

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

Awni Hannun

66,565 views • 2 years ago

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,745 views • 2 years ago

QLoRA fine-tuning 4-bit Gemma 2B on iPhone 15 Pro with MLX Swift. A nice size for fine-tuning on device, getting 70-100 toks/sec depending on the batch. Guide here:

QLoRA fine-tuning 4-bit Gemma 2B on iPhone 15 Pro with MLX Swift. A nice size for fine-tuning on device, getting 70-100 toks/sec depending on the batch. Guide here:

Awni Hannun

19,530 views • 2 years ago

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,662 views • 1 year ago

🚀 Tensorplex Dojo (Subnet 52) in action! Meet DOJO-INTERFACE-CODER-7B: Qwen2.5-Coder-7B-Instruct, fine-tuned with Dojo datasets to craft stunning front-end UIs! ✨ Generates beautiful, interactive interfaces ✨ Trained on synthetic data with distributed human feedback ✨ Powered by (Subnet 52) on Bittensor 👇

🚀 Tensorplex Dojo (Subnet 52) in action! Meet DOJO-INTERFACE-CODER-7B: Qwen2.5-Coder-7B-Instruct, fine-tuned with Dojo datasets to craft stunning front-end UIs! ✨ Generates beautiful, interactive interfaces ✨ Trained on synthetic data with distributed human feedback ✨ Powered by (Subnet 52) on Bittensor 👇

Tensorplex Labs

18,281 views • 1 year ago

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

Standard Intelligence

71,591 views • 5 months ago

$still experimenting with LoRA based on the Thinking Machines configuration and just implemented it in colab. In this notebook I set up a fine tune of Qwen/Qwen3-0.6B on the OpenR1-Math dataset with lora rank of 1. with this setup you can get the same reward accuracy as full fine-tuning, at a fraction of vram usage.$

still experimenting with LoRA based on the Thinking Machines configuration and just implemented it in colab. In this notebook I set up a fine tune of Qwen/Qwen3-0.6B on the OpenR1-Math dataset with lora rank of 1. with this setup you can get the same reward accuracy as full fine-tuning, at a fraction of vram usage.

Ben Burtenshaw

25,624 views • 9 months ago

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

merve

15,625 views • 1 year ago

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Pau Labarta Bajo

80,190 views • 5 months ago

Here is a real-life run, end-to-end from training to using the trained LLM in LM Studio by LM Studio MLX-LoRA-Studio gives you a user friendly, extremely customisable training interface for fine-tuning and training LLMs. Its a MacOS native app, front- and backend. Will be releasing the App in the coming days. GH Repo:

Here is a real-life run, end-to-end from training to using the trained LLM in LM Studio by LM Studio MLX-LoRA-Studio gives you a user friendly, extremely customisable training interface for fine-tuning and training LLMs. Its a MacOS native app, front- and backend. Will be releasing the App in the coming days. GH Repo:

Gökdeniz Gülmez

11,906 views • 1 month ago

We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data. For the first time, one fine-tuning example can teach a new behavior that generalizes. Below: 4 folding strategies, learned from one example each and tested on held-out setups.

We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data. For the first time, one fine-tuning example can teach a new behavior that generalizes. Below: 4 folding strategies, learned from one example each and tested on held-out setups.

Tony Zhao

166,951 views • 12 days ago

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,480 views • 1 year ago

$QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.$

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.

Paolo Ardoino 🤖

42,413 views • 3 months ago

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.

Paolo Ardoino 🤖

142,533 views • 8 months ago

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

h100envy

13,856 views • 13 days ago

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

QVAC

108,569 views • 4 months ago

Stop paying for every API call. Download once and run forever. My 1-post “thread” on Gemma. → Gemma is Google’s open weights model family. → Gemma runs locally on laptop and phone. → Gemma 3N is optimized for low RAM devices. → Variants exist for medical and safety and embeddings. → Fine tuning lets you build custom models for your niche. Save this video, you’ll build your first local AI tool. Want the SOP? DM me. 💬

Stop paying for every API call. Download once and run forever. My 1-post “thread” on Gemma. → Gemma is Google’s open weights model family. → Gemma runs locally on laptop and phone. → Gemma 3N is optimized for low RAM devices. → Variants exist for medical and safety and embeddings. → Fine tuning lets you build custom models for your niche. Save this video, you’ll build your first local AI tool. Want the SOP? DM me. 💬

Julian Goldie SEO

14,403 views • 7 months ago

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Vaibhav (VB) Srivastav

80,474 views • 1 year ago