正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Full Fine-Tuning vs LoRA by hand ✍️ interactive diagram. Open Here's some T/F questions to test your knowledge: [ ] Growing the batch size grows the number of trainable parameters [ ] Both rows produce an output of the same shape [ ] LoRA's ΔW is a smaller matrix... show more

Tom Yeh

65,544 subscribers

10,720 次观看 • 2 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

5 条评论

Michelle Walker 的头像

Michelle Walker2 个月前

Agreed🙌

Forlais 的头像

Forlais2 个月前

That's interesting, rank is the dial people underrate. We keep leaning on r=2 in places and it holds up far better than the maths suggests it should.

Kyssta 的头像

Kyssta2 个月前

Answers, in order: false, true. Batch size changes gradients per step, not the count of trainable parameters. And LoRA's delta W is BA, rank r, so it lands in the same shape as W while training a fraction of the weights.

EB1A Experts 的头像

EB1A Experts2 个月前

Love the interactive approach. LoRA becomes much easier to understand once you can visually see how rank changes the parameterization and training cost.

Mouad Elbaz 的头像

Mouad Elbaz2 个月前

And thire is optimize version of Lora called QLoRA that used 4 bit quantized rather than 16 bits in LoRa

相关视频

$QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.$

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.

Paolo Ardoino 🤖

43,075 次观看 • 6 个月前

This is the f-f-f-f-f-founder of t-t-t-t-t-t-t-transgender d-d-d-d-day of v-v-v-v-visibility

Sensitive content

This is the f-f-f-f-f-founder of t-t-t-t-t-t-t-transgender d-d-d-d-day of v-v-v-v-visibility

Blonde Bigot

18,215 次观看 • 7 个月前

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

Ex-JPMorgan engineer who wrote the LLM Course explained everything about fine-tuning and merging in 18 minutes - better than $2500 fine-tuning bootcamps. pick the base -> LoRA or QLoRA -> then DPO or KTO for alignment -> merge two fine-tunes into one stronger model -> ship a model that beats the base on your task. That loop is why Labonne's merges sit at the top of the Hugging Face leaderboard. LoRA + QLoRA + DPO + KTO + mergekit - that's the stack. Watch and save it, then merge your first two fine-tunes this week.

h100envy

13,856 次观看 • 3 个月前

The Fine$t👩🏽‍❤️‍💋‍👩🏽

Sensitive content

The Fine$t👩🏽‍❤️‍💋‍👩🏽

Dolly Wrld🦂🎀 .

31,388 次观看 • 1 个月前

OpenAI engineer explained every form of fine-tuning in 1 hour 46 minutes - better than $3000 fine-tuning bootcamps. pick your task -> try prompting first -> drop to SFT for style -> run DPO for preference -> run RFT for verifiable rewards -> ship a model 10x cheaper than GPT-5. That loop is why teams that fine-tune beat teams that only prompt on cost and speed. Prompting + SFT + DPO + RFT + OpenAI fine-tuning API - that's the stack. Watch and save it, then pick the right fine-tune for your task this week.

OpenAI engineer explained every form of fine-tuning in 1 hour 46 minutes - better than $3000 fine-tuning bootcamps. pick your task -> try prompting first -> drop to SFT for style -> run DPO for preference -> run RFT for verifiable rewards -> ship a model 10x cheaper than GPT-5. That loop is why teams that fine-tune beat teams that only prompt on cost and speed. Prompting + SFT + DPO + RFT + OpenAI fine-tuning API - that's the stack. Watch and save it, then pick the right fine-tune for your task this week.

h100envy

41,245 次观看 • 3 个月前

Over the past few days, I dug into the geometry of LoRA to understand how exactly it constraints the geometry of the search space. There's a beautiful connection to the *intrinsic dimension* of a task. Also learnt I'm saying "difficulty" wrong... 🤡 0:00 MNIST vs the walking humanoid 1:05 Do we still need fine-tuning? 3:05 What is LoRA? 6:55 What does the rank constraint actually do? (geometric intuition) 9:50 Intrinsic dimension: training along random directions 13:30 Intrinsic dimension of pre-trained LLMs 15:10 Demo: fine-tuning a query router with Qwen3.8-27B

Over the past few days, I dug into the geometry of LoRA to understand how exactly it constraints the geometry of the search space. There's a beautiful connection to the intrinsic dimension of a task. Also learnt I'm saying "difficulty" wrong... 🤡 0:00 MNIST vs the walking humanoid 1:05 Do we still need fine-tuning? 3:05 What is LoRA? 6:55 What does the rank constraint actually do? (geometric intuition) 9:50 Intrinsic dimension: training along random directions 13:30 Intrinsic dimension of pre-trained LLMs 15:10 Demo: fine-tuning a query router with Qwen3.8-27B

Julia Turc

19,922 次观看 • 6 天前

I'm fine tuning the muscle-skin setup of my T. rex model, so decided to made a little comparison video showing the effect of the different layers during animation.

I'm fine tuning the muscle-skin setup of my T. rex model, so decided to made a little comparison video showing the effect of the different layers during animation.

Dr Matt Dempsey

22,562 次观看 • 5 个月前

Holy sh*t, developers are going to lose their minds over this. Someone open sourced a repo that lets you fine-tune 100+ LLMs and VLMs without building the training pipeline yourself. Llama. Qwen. DeepSeek. Gemma. Mistral. And more. It's called LLaMA-Factory and here's why it changes everything: → 100+ open-source LLMs + VLMs → LoRA + QLoRA → SFT + DPO + PPO → multimodal fine-tuning → experiment tracking → Web UI You don't need to train the next frontier model. You can take a much smaller model and make it extremely good at one specific job. most people haven't done that yet. full guide in the article below

Holy sh*t, developers are going to lose their minds over this. Someone open sourced a repo that lets you fine-tune 100+ LLMs and VLMs without building the training pipeline yourself. Llama. Qwen. DeepSeek. Gemma. Mistral. And more. It's called LLaMA-Factory and here's why it changes everything: → 100+ open-source LLMs + VLMs → LoRA + QLoRA → SFT + DPO + PPO → multimodal fine-tuning → experiment tracking → Web UI You don't need to train the next frontier model. You can take a much smaller model and make it extremely good at one specific job. most people haven't done that yet. full guide in the article below

Rahul

34,785 次观看 • 1 个月前

LoRA training is the standout feature. You can fine tune Qwen Image directly online, export the trained LoRA, and plug it straight into ComfyUI workflows like any custom model. This replaces a lot of paid LoRA setups.

LoRA training is the standout feature. You can fine tune Qwen Image directly online, export the trained LoRA, and plug it straight into ComfyUI workflows like any custom model. This replaces a lot of paid LoRA setups.

SANI BULA

15,462 次观看 • 8 个月前

Ice-T had a heated confrontation with a cop that escalated to the point the rapper called the officer a "f**king a**hole." FULL STORY HERE:

Ice-T had a heated confrontation with a cop that escalated to the point the rapper called the officer a "fking ahole." FULL STORY HERE:

TMZ

139,394 次观看 • 1 年前

"A Transfer BAN isn't ENOUGH... A fine isn't ENOUGH! Beth T has her say on the Premier League and the punishment Man City should receive!

"A Transfer BAN isn't ENOUGH... A fine isn't ENOUGH! Beth T has her say on the Premier League and the punishment Man City should receive!

The United Stand

56,285 次观看 • 1 个月前

You can now train a 3D LoRA, exclusively on fal! TRELLIS.2 LoRA Trainer turns a folder of 3D assets into custom image-to-3D adapters, fine-tune shape and texture, then generate GLBs that match your style.

You can now train a 3D LoRA, exclusively on fal! TRELLIS.2 LoRA Trainer turns a folder of 3D assets into custom image-to-3D adapters, fine-tune shape and texture, then generate GLBs that match your style.

fal

50,927 次观看 • 4 个月前

Draymond Green just proposed that the Timberwolves need to go “FULL FORCE” back at Wemby. Calls for some member of the T-Wolves to “send a message or get rolled” over after there was no fine or suspension by the league against Victor Wembanyama. (🎥: via The Draymond Green Show)

Draymond Green just proposed that the Timberwolves need to go “FULL FORCE” back at Wemby. Calls for some member of the T-Wolves to “send a message or get rolled” over after there was no fine or suspension by the league against Victor Wembanyama. (🎥: via The Draymond Green Show)

SpursRΞPORTΞR

13,625 次观看 • 5 个月前

. Jonathan Whitaker 's talk: Napkin Math For Fine Tuning was so popular that we ended up doing an encore! He answers q's like: - When should I use LoRA? Quantization? GC? - What’s the cheapest option? most accurate? - What hardware? - What batch size / context length ..etc?

. Jonathan Whitaker 's talk: Napkin Math For Fine Tuning was so popular that we ended up doing an encore! He answers q's like: - When should I use LoRA? Quantization? GC? - What’s the cheapest option? most accurate? - What hardware? - What batch size / context length ..etc?

Hamel Husain

110,070 次观看 • 2 年前

THE RESULTS ARE IN! Barry O’s Birth C e r t i f i c a t e was a “fraudulently created document”. Watch full video for conclusion!

THE RESULTS ARE IN! Barry O’s Birth C e r t i f i c a t e was a “fraudulently created document”. Watch full video for conclusion!

Dr. Truth

63,463 次观看 • 3 年前

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

LeRobot

23,450 次观看 • 3 个月前

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.

h100envy

711,476 次观看 • 3 个月前