正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Fine-tuning Mistral 7B with LoRA on a 32 GB M1 (laptop!) in MLX Updated example uses less RAM + support for custom datasets 🚀

Awni Hannun

44,992 subscribers

148,211 次观看 • 2 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 条评论

murat 🍥 的头像

murat 🍥2 年前

i was just about to ask if anyone's done LoRA/QLoRA on mlx yet. exciting!

Delip Rao e/σ 的头像

Delip Rao e/σ2 年前

can you talk to the gods that are and make MLX work on my M2 w/ 14.2 (23C63)?

Awni Hannun 的头像

Awni Hannun2 年前

😬 It should run no prob for you! Send me your issue, we'll definitely debug

Eric Hartford 的头像

Eric Hartford2 年前

@winglian you think axolotl could work on mlx?

Peter J. Liu 的头像

Peter J. Liu2 年前

Some performance comparisons with GPUs, e.g. 4090 would be useful.

Rahul Dave 的头像

Rahul Dave2 年前

Guessing this needs 32! Darn it I got a 16…

Jay 的头像

Jay2 年前

Can’t wait to couple this with @textcortex desktop app 🔩

ae 的头像

ae2 年前

any chance that it runs on 8G M2 macbook?

thyself Know 的头像

thyself Know2 年前

I only have 16 GB RAM 😭

John Gorman 的头像

John Gorman2 年前

@awnihannun @abacaj Yup - I'm be priming my Mixtral Q6 on my 32GB M1 to check out MLX ;)

相关视频

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Awni Hannun

581,777 次观看 • 2 年前

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

Awni Hannun

66,585 次观看 • 2 年前

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,781 次观看 • 2 年前

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,697 次观看 • 1 年前

🚀 Tensorplex Dojo (Subnet 52) in action! Meet DOJO-INTERFACE-CODER-7B: Qwen2.5-Coder-7B-Instruct, fine-tuned with Dojo datasets to craft stunning front-end UIs! ✨ Generates beautiful, interactive interfaces ✨ Trained on synthetic data with distributed human feedback ✨ Powered by (Subnet 52) on Bittensor 👇

🚀 Tensorplex Dojo (Subnet 52) in action! Meet DOJO-INTERFACE-CODER-7B: Qwen2.5-Coder-7B-Instruct, fine-tuned with Dojo datasets to craft stunning front-end UIs! ✨ Generates beautiful, interactive interfaces ✨ Trained on synthetic data with distributed human feedback ✨ Powered by (Subnet 52) on Bittensor 👇

Tensorplex Labs

18,281 次观看 • 1 年前

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

Standard Intelligence

72,463 次观看 • 7 个月前

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

merve

16,733 次观看 • 1 年前

Here is a real-life run, end-to-end from training to using the trained LLM in LM Studio by LM Studio MLX-LoRA-Studio gives you a user friendly, extremely customisable training interface for fine-tuning and training LLMs. Its a MacOS native app, front- and backend. Will be releasing the App in the coming days. GH Repo:

Here is a real-life run, end-to-end from training to using the trained LLM in LM Studio by LM Studio MLX-LoRA-Studio gives you a user friendly, extremely customisable training interface for fine-tuning and training LLMs. Its a MacOS native app, front- and backend. Will be releasing the App in the coming days. GH Repo:

Gökdeniz Gülmez

11,952 次观看 • 3 个月前

We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data. For the first time, one fine-tuning example can teach a new behavior that generalizes. Below: 4 folding strategies, learned from one example each and tested on held-out setups.

We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data. For the first time, one fine-tuning example can teach a new behavior that generalizes. Below: 4 folding strategies, learned from one example each and tested on held-out setups.

Tony Zhao

170,975 次观看 • 2 个月前

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Advice for AI engineers 💡 A small Visual Language Model fine-tuned on your custom dataset is as accurate as GPT-5... ... and costs 50 times less. For example, LFM2.5-VL-1.6B by Liquid AI running locally at full speed with llama.cpp ↓

Pau Labarta Bajo

80,258 次观看 • 7 个月前

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,553 次观看 • 1 年前

$QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.$

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.

Paolo Ardoino 🤖

42,413 次观看 • 5 个月前

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.

Paolo Ardoino 🤖

142,888 次观看 • 10 个月前

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

QVAC

108,875 次观看 • 6 个月前

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

h100envy

13,856 次观看 • 2 个月前

Stop paying for every API call. Download once and run forever. My 1-post “thread” on Gemma. → Gemma is Google’s open weights model family. → Gemma runs locally on laptop and phone. → Gemma 3N is optimized for low RAM devices. → Variants exist for medical and safety and embeddings. → Fine tuning lets you build custom models for your niche. Save this video, you’ll build your first local AI tool. Want the SOP? DM me. 💬

Stop paying for every API call. Download once and run forever. My 1-post “thread” on Gemma. → Gemma is Google’s open weights model family. → Gemma runs locally on laptop and phone. → Gemma 3N is optimized for low RAM devices. → Variants exist for medical and safety and embeddings. → Fine tuning lets you build custom models for your niche. Save this video, you’ll build your first local AI tool. Want the SOP? DM me. 💬

Julian Goldie SEO

14,403 次观看 • 9 个月前

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Vaibhav (VB) Srivastav

80,474 次观看 • 2 年前