Загрузка видео...

Не удалось загрузить видео

На главную

MLX Swift example can also QLoRA fine-tune Llama 3.2. Here's the 1B fine-tuning on my iPhone 15 Pro at > 150 toks/sec. A this rate only takes a few minutes to learn some decent adapters fully on-device.

awnihannun's profile picture

Awni Hannun

34,066 subscribers

91,632 просмотров • 1 год назад •via X (Twitter)

ОбразованиеЗдоровье и велнесНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля Awni Hannun
Awni Hannun1 год назад

Example is also open-source here (under MIT license):

Фото профиля GPT.Biz
GPT.Biz1 год назад

This looks amazing! It’s incredible to see fine-tuning done so quickly on a phone

Фото профиля ⚡️Dylan White
⚡️Dylan White1 год назад

Whoa, that's impressive! Fine-tuning Llama 3.2 on your iPhone 15 Pro is no joke!

Фото профиля involuntarily incelibate
involuntarily incelibate1 год назад

If only iOS would expose more substantial endpoints, you could fine tune “Siri”!

Фото профиля Janaka
Janaka1 год назад

Impressive

Фото профиля Andres Gomez Sarmiento
Andres Gomez Sarmiento1 год назад

really cool

Фото профиля Kalin Ovtcharov
Kalin Ovtcharov1 год назад

Ability to run llms locally on-device and fine tune them is an extremely important yet relatively under-appreciated/ignored capability today. I expect that will change as new applications emerge that leverage SLMs. I expect some governments may even ban them in the near future.

Фото профиля Josep M. Ganyet
Josep M. Ganyet1 год назад

@Scobleizer @xaviviro

Фото профиля Sorayesh Semo
Sorayesh Semo1 год назад

Awesome

Фото профиля samuel abraham gomez villafuerte
samuel abraham gomez villafuerte1 год назад

Para andriod cuando

Похожие видео

QLoRA fine-tuning 4-bit Gemma 2B on iPhone 15 Pro with MLX Swift. A nice size for fine-tuning on device, getting 70-100 toks/sec depending on the batch. Guide here:
0:40

QLoRA fine-tuning 4-bit Gemma 2B on iPhone 15 Pro with MLX Swift. A nice size for fine-tuning on device, getting 70-100 toks/sec depending on the batch. Guide here:

Awni Hannun

19,530 просмотров • 2 лет назад

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)
1:19

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Awni Hannun

581,723 просмотров • 2 лет назад

Demo of Tether's QVAC running local inference on a mobile device at incredible speed, via llama.cpp + LLAMA 3.2 1B parameters. QVAC is a generalized inference and fine-tuning runtime able to adapt to any device, from smartphones, to laptops and servers. Lots of models being supported already. More to come. No limits. Infinite intelligence. Coming soon
0:53

Demo of Tether's QVAC running local inference on a mobile device at incredible speed, via llama.cpp + LLAMA 3.2 1B parameters. QVAC is a generalized inference and fine-tuning runtime able to adapt to any device, from smartphones, to laptops and servers. Lots of models being supported already. More to come. No limits. Infinite intelligence. Coming soon

Paolo Ardoino 🤖

42,070 просмотров • 11 месяцев назад

Fine-tuning Mistral 7B with LoRA on a 32 GB M1 (laptop!) in MLX Updated example uses less RAM + support for custom datasets 🚀
0:38

Fine-tuning Mistral 7B with LoRA on a 32 GB M1 (laptop!) in MLX Updated example uses less RAM + support for custom datasets 🚀

Awni Hannun

148,141 просмотров • 2 лет назад

DeepSeek R1 (the 680B MOE) is ~20% faster in the latest mlx / mlx-lm. 4-bit model on 3 M2 Ultras generates 4k tokens at a respectable 15 toks/sec. Plus some QoL improvements: - Only downloads the local shard (much faster startup) - Distributed launcher ships with MLX
0:50

DeepSeek R1 (the 680B MOE) is ~20% faster in the latest mlx / mlx-lm. 4-bit model on 3 M2 Ultras generates 4k tokens at a respectable 15 toks/sec. Plus some QoL improvements: - Only downloads the local shard (much faster startup) - Distributed launcher ships with MLX

Awni Hannun

86,921 просмотров • 1 год назад

The new Deep Seek V3 0324 in 4-bit runs at > 20 toks/sec on a 512GB M3 Ultra with mlx-lm!
0:31

The new Deep Seek V3 0324 in 4-bit runs at > 20 toks/sec on a 512GB M3 Ultra with mlx-lm!

Awni Hannun

168,870 просмотров • 1 год назад

On-device realtime transcription on iPhone 15 Pro max 🚀 Using MLX-Audio-Swift + Qwen3-ASR-0.6B by Qwen It’s much faster and more consistent with the latest adjustments. Almost ready to push to GH.
1:46

On-device realtime transcription on iPhone 15 Pro max 🚀 Using MLX-Audio-Swift + Qwen3-ASR-0.6B by Qwen It’s much faster and more consistent with the latest adjustments. Almost ready to push to GH.

Prince Canuma

44,103 просмотров • 4 месяцев назад

Translating a full-length news article on iPhone 17 Pro with TranslateGemma from Google DeepMind The model works incredibly well on long text Running fully on-device with MLX
0:46

Translating a full-length news article on iPhone 17 Pro with TranslateGemma from Google DeepMind The model works incredibly well on long text Running fully on-device with MLX

Adrien Grondin

15,493 просмотров • 4 месяцев назад

I just added the new Llama 3.2 1B and 3B models to LitGPT, the open-source LLM library I help develop (focused on efficiency and code readability). LitGPT allows you to fine-tune and use these models on the cloud or a laptop. So, if you are looking for something to play with this weekend: # 1) Finetune the model litgpt finetune_lora meta-llama/Llama-3.2-1B \ --data JSON \ --data.json_path my_custom_dataset.json \ --train.epochs 1 \ --out_dir out/llama-3.2-finetuned \ --precision bf16-true # 2) Chat with the model litgpt chat out/llama-3.2-finetuned/final # 3) Serve the model via an API endpoint litgpt serve out/llama-3.2-finetuned/final
1:48

I just added the new Llama 3.2 1B and 3B models to LitGPT, the open-source LLM library I help develop (focused on efficiency and code readability). LitGPT allows you to fine-tune and use these models on the cloud or a laptop. So, if you are looking for something to play with this weekend: # 1) Finetune the model litgpt finetune_lora meta-llama/Llama-3.2-1B \ --data JSON \ --data.json_path my_custom_dataset.json \ --train.epochs 1 \ --out_dir out/llama-3.2-finetuned \ --precision bf16-true # 2) Chat with the model litgpt chat out/llama-3.2-finetuned/final # 3) Serve the model via an API endpoint litgpt serve out/llama-3.2-finetuned/final

Sebastian Raschka

65,529 просмотров • 1 год назад

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:
6:05

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:

Santiago

324,586 просмотров • 2 лет назад

HRM-Text 101 is here. This tutorial takes you from zero to one: from setup to fine-tuning to evaluation. Download the base checkpoint. Fine-tune it on a real task. Evaluate the results. End to end, on a single GPU. Watch the tutorial and start building with HRM-Text.
4:40

HRM-Text 101 is here. This tutorial takes you from zero to one: from setup to fine-tuning to evaluation. Download the base checkpoint. Fine-tune it on a real task. Evaluate the results. End to end, on a single GPU. Watch the tutorial and start building with HRM-Text.

Sapient Intelligence

187,377 просмотров • 1 месяц назад

Hunyuan-Large by Tencent is a 389B param MOE (52B active). It's the largest open-weights MOE. In some benchmarks it exceeds Llama 3.1 405B. With MLX's new 3-bit quant it just barely fits on a single 192GB M2 Ultra! And runs at a very decent >15 toks/sec:
0:22

Hunyuan-Large by Tencent is a 389B param MOE (52B active). It's the largest open-weights MOE. In some benchmarks it exceeds Llama 3.1 405B. With MLX's new 3-bit quant it just barely fits on a single 192GB M2 Ultra! And runs at a very decent >15 toks/sec:

Awni Hannun

64,939 просмотров • 1 год назад

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!
2:36

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

Andrew Ng

502,781 просмотров • 2 лет назад

I just ran Qwen3.5 35B on my iPhone at 5.6 tok/sec. Fully on-device. 4bit | 256 experts. Model: 19.5GB. iPhone: 12GB RAM. wild.
0:49

I just ran Qwen3.5 35B on my iPhone at 5.6 tok/sec. Fully on-device. 4bit | 256 experts. Model: 19.5GB. iPhone: 12GB RAM. wild.

alexintosh

388,230 просмотров • 3 месяцев назад

Llama 3.3 70B 4-bit runs nicely on a 64GB M3 Max with in MLX LM (~10 toks/sec). Would be even faster on an M4 Max. Yesterday's server-only 405B is today's laptop 70B:
0:33

Llama 3.3 70B 4-bit runs nicely on a 64GB M3 Max with in MLX LM (~10 toks/sec). Would be even faster on an M4 Max. Yesterday's server-only 405B is today's laptop 70B:

Awni Hannun

110,375 просмотров • 1 год назад

Llama 3.2 1B 4bit running at ~55 tps in Debug ~72 in Release on Vision Pro with MLX! It's incredible how Universal Clipboard and Universal Control works perfectly from Mac Virtual Display to VisionOS Apps 🤯
0:28

Llama 3.2 1B 4bit running at ~55 tps in Debug ~72 in Release on Vision Pro with MLX! It's incredible how Universal Clipboard and Universal Control works perfectly from Mac Virtual Display to VisionOS Apps 🤯

Ivan Fioravanti ᯅ

18,143 просмотров • 1 год назад

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.
1:49

QVAC first demo (android mobile) of its local on-device fine-tuning generalized framework, which aims to bring LORA fine-tuning to any device, scaling from the smallest embedded device to any smartphone, laptop, server, operating system. In combination with WIP multi-node features, we expect QVAC to become the standard for local on-device single/multi node inference and fine-tuning. Imagine a horde of AI agents being able to learn from your emails, notes and messages, maintaining 100℅ privacy, leveraging only your own devices. A P2P private intelligence network at your fingertips. That's how QVAC personal assistant will look like. Future of Intelligence won't be centralized. Infinite Ingelligence vibes.

Paolo Ardoino 🤖

142,533 просмотров • 8 месяцев назад

Qwen 2.5 Coder Q4 M4 Max Inference test. Apple MLX vs Ollama: - MLX: 23.97 toks/sec 🥇🔥 - Ollama: 18.33 toks/sec 🥈 Here a video to show results
1:00

Qwen 2.5 Coder Q4 M4 Max Inference test. Apple MLX vs Ollama: - MLX: 23.97 toks/sec 🥇🔥 - Ollama: 18.33 toks/sec 🥈 Here a video to show results

Ivan Fioravanti ᯅ

34,881 просмотров • 1 год назад

Quantized Gemma 2B runs pretty fast on my iPhone 15 pro in MLX Swift. code & docs: Comparable to GPT 3.5 turbo and Mixtral 8x7B in LMSYS Org benchmarks but runs efficiently on an iPhone. Pretty wild.
0:25

Quantized Gemma 2B runs pretty fast on my iPhone 15 pro in MLX Swift. code & docs: Comparable to GPT 3.5 turbo and Mixtral 8x7B in LMSYS Org benchmarks but runs efficiently on an iPhone. Pretty wild.

Awni Hannun

79,702 просмотров • 1 год назад

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️
0:36

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

merve

15,625 просмотров • 1 год назад