Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Full Fine-Tuning vs LoRA by hand ✍️ interactive diagram. Open Here's some T/F questions to test your knowledge: [ ] Growing the batch size grows the number of trainable parameters [ ] Both rows produce an output of the same shape [ ] LoRA's ΔW is a smaller matrix... show more

Tom Yeh

65,267 subscribers

10,580 görüntüleme • 14 gün önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Self-Attention vs Cross-Attention by hand ✍️ interactive diagram. Open Here's some T/F questions to test your knowledge: [ ] Making X taller widens Wq, Wk and Wv but leaves Q, K and V unchanged [ ] E has to be the same length as X [ ] Q and K must share the same height [ ] Self-attention and cross-attention must use the same value height [ ] Adding tokens grows self-attention's scores faster than cross-attention's Check your answers by dragging the diagram. ~ Prof. Tom Yeh Answers: T, F, T, F, T

Self-Attention vs Cross-Attention by hand ✍️ interactive diagram. Open Here's some T/F questions to test your knowledge: [ ] Making X taller widens Wq, Wk and Wv but leaves Q, K and V unchanged [ ] E has to be the same length as X [ ] Q and K must share the same height [ ] Self-attention and cross-attention must use the same value height [ ] Adding tokens grows self-attention's scores faster than cross-attention's Check your answers by dragging the diagram. ~ Prof. Tom Yeh Answers: T, F, T, F, T

Tom Yeh

14,021 görüntüleme • 15 gün önce

$still experimenting with LoRA based on the Thinking Machines configuration and just implemented it in colab. In this notebook I set up a fine tune of Qwen/Qwen3-0.6B on the OpenR1-Math dataset with lora rank of 1. with this setup you can get the same reward accuracy as full fine-tuning, at a fraction of vram usage.$

still experimenting with LoRA based on the Thinking Machines configuration and just implemented it in colab. In this notebook I set up a fine tune of Qwen/Qwen3-0.6B on the OpenR1-Math dataset with lora rank of 1. with this setup you can get the same reward accuracy as full fine-tuning, at a fraction of vram usage.

Ben Burtenshaw

25,624 görüntüleme • 11 ay önce

$QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.$

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.

Paolo Ardoino 🤖

42,413 görüntüleme • 4 ay önce

Draymond Green just proposed that the Timberwolves need to go “FULL FORCE” back at Wemby. Calls for some member of the T-Wolves to “send a message or get rolled” over after there was no fine or suspension by the league against Victor Wembanyama. (🎥: via The Draymond Green Show)

Draymond Green just proposed that the Timberwolves need to go “FULL FORCE” back at Wemby. Calls for some member of the T-Wolves to “send a message or get rolled” over after there was no fine or suspension by the league against Victor Wembanyama. (🎥: via The Draymond Green Show)

SpursRΞPORTΞR

13,590 görüntüleme • 3 ay önce

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

LeRobot

22,200 görüntüleme • 1 ay önce

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

The era of Stable Intelligence is here 🤖 Tether’s QVAC Fabric just released the world’s first cross-platform 1-bit LLM LoRA fine-tuning framework. QVAC Fabric extends Microsoft's ultra-efficient BitNet architecture, allowing fine-tuning and inference of LLMs directly on your smartphone—no NVIDIA GPU/CUDA lock-in or expensive server required. The Breakthrough: - Total Sovereignty: LoRA fine-tune ultra-efficient models locally on any smartphone, including iPhones, Pixel phones, Samsung Galaxy phones and any desktop/laptop operating systems using Vulkan and Metal backends. - Extreme Efficiency: 1-bit architecture uses up to 90% less memory and runs up to 11x faster than traditional models. - Universal Access: What used to require a data center now runs on the chip in your pocket. Own your intelligence. The era of stable, local AI is here. 📱🧠 Read the full details on Hugging Face and grab the binaries to build on your own hardware.

QVAC

108,835 görüntüleme • 5 ay önce

Dana White says it's a myth that fighters get paid more when they transition from MMA to boxing: "That's the bullsh*t publicly that people believe, but that's not the truth... boxing don't work... It is the most f***ed up business, and all these sound bites are all so full of sh*t that it makes everybody f***ing believe that that sh*t is true, when it's not." 🎥 Kevin Iole #UFC #MMA #Boxing

Dana White says it's a myth that fighters get paid more when they transition from MMA to boxing: "That's the bullsh*t publicly that people believe, but that's not the truth... boxing don't work... It is the most f***ed up business, and all these sound bites are all so full of sh*t that it makes everybody f***ing believe that that sh*t is true, when it's not." 🎥 Kevin Iole #UFC #MMA #Boxing

Championship Rounds

132,336 görüntüleme • 1 yıl önce

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Andrew Ng

66,435 görüntüleme • 2 yıl önce

Many people are flat wrong about DeepSeek. You might think everyone freaked out about DeepSeek because the model is really good—which it is—or because it was from China, which is also true. But there's a more subtle reason: DeepSeek showed they could achieve those results using reinforcement learning instead of supervised fine-tuning. In other words, they didn't need an expensive phase where a bunch of people tuned the model by answering questions. They were able to do all of this faster and cheaper using reinforcement fine-tuning. This is a huge deal! I recorded the attached video with the following goals in mind: 1. Explain how reinforcement fine-tuning works 2. Show you how you can fine-tune your own models 3. Walk you through a complete code example

Many people are flat wrong about DeepSeek. You might think everyone freaked out about DeepSeek because the model is really good—which it is—or because it was from China, which is also true. But there's a more subtle reason: DeepSeek showed they could achieve those results using reinforcement learning instead of supervised fine-tuning. In other words, they didn't need an expensive phase where a bunch of people tuned the model by answering questions. They were able to do all of this faster and cheaper using reinforcement fine-tuning. This is a huge deal! I recorded the attached video with the following goals in mind: 1. Explain how reinforcement fine-tuning works 2. Show you how you can fine-tune your own models 3. Walk you through a complete code example

Santiago

105,589 görüntüleme • 1 yıl önce

"Your Mac is useless for AI without CUDA, buy the $4K DGX." Half right — and the half that's wrong costs you $4,000. CUDA matters for training and some frameworks. Real. But "running local models"? I benchmarked a Mac Mini against the DGX on the same 30B: generation was 56 vs 84 t/s. Not useless — usable, faster than you read, no CUDA required. llama.cpp and Ollama don't care what logo is on the chip. Here's the honest split the video skips: - Commercial fine-tuning, CUDA-only pipelines, big-context prefill → yes, the DGX earns its price - Running and chatting with local models → your Mac already does this fine The DGX isn't overpriced. It's mis-pitched. It's a prefill-and-training machine, not a "your Mac is trash" machine. Buy it for the workload that needs it — not out of CUDA FOMO. Full 3-way benchmark (DGX vs Strix Halo vs Mac Mini, prefill vs generation) pinned.

KyzoroX

263,756 görüntüleme • 1 ay önce

We're releasing ACE-Step-v1.5(2B), a fast, high-quality open-source music model. It runs locally on a consumer-grade GPU, generates a full song in under 2 seconds(on an A100), supports LoRA fine-tuning, and beats SUNO on common eval metrics. GitHub: Key traits: Quality: beats Suno on common eval scores Speed: full song under 2s on A100 Local: ~4GB VRAM, under 10s on RTX 3090 LoRA: train your own style with a few songs License: MIT, free for commercial use Data: fully authorized plus synthetic The music AI space lacks commercial-grade open models. Many creators are forced to rely on closed-source services, and can’t fully own, run locally, or fine-tune their own models. We want to help change that.

We're releasing ACE-Step-v1.5(2B), a fast, high-quality open-source music model. It runs locally on a consumer-grade GPU, generates a full song in under 2 seconds(on an A100), supports LoRA fine-tuning, and beats SUNO on common eval metrics. GitHub: Key traits: Quality: beats Suno on common eval scores Speed: full song under 2s on A100 Local: ~4GB VRAM, under 10s on RTX 3090 LoRA: train your own style with a few songs License: MIT, free for commercial use Data: fully authorized plus synthetic The music AI space lacks commercial-grade open models. Many creators are forced to rely on closed-source services, and can’t fully own, run locally, or fine-tune their own models. We want to help change that.

ACE Music

1,057,780 görüntüleme • 6 ay önce

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

Santiago

384,510 görüntüleme • 3 yıl önce

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 görüntüleme • 2 yıl önce

New Course: Reinforcement Fine-Tuning LLMs with GRPO! Learn to use reinforcement learning to improve your LLM performance in this short course, built in collaboration with Predibase by Rubrik, and taught by Travis Addair, its Co-Founder and CTO, and Arnav Garg, its Senior Engineer and Machine Learning Lead. Reasoning models have been one of the most important developments in LLMs. Reinforcement Fine-Tuning (RFT) uses rewards to encourage LLMs to find solutions to multi-step reasoning tasks such as solving math problems and debugging code - without needing pre-existing training examples like in traditional supervised fine-tuning. Group Relative Policy Optimization (GRPO) is a reinforcement fine-tuning algorithm gaining rapid adoption. Developed by the DeepSeek team and used to train the R1 reasoning model, GRPO uses reward functions that you can write in Python to assign rewards to model responses. It’s beneficial for tasks with verifiable outcomes and can work well even with fewer than 100 training examples. It can also significantly improve the reasoning ability of smaller LLMs, making applications faster and more cost effective. In this course, you’ll take a technical deep dive into RFT with GRPO. You’ll learn to build reward functions that you can use in the GRPO training process to guide an LLM toward better performance on multi-step reasoning tasks. In detail, you’ll: - Learn when reinforcement fine-tuning is a better fit than supervised fine-tuning, especially for tasks involving multi-step reasoning or limited labeled data. - Understand how GRPO uses programmable reward functions as a more scalable alternative to the human feedback required for other reinforcement learning algorithms, such as RLHF and DPO. - Frame the Wordle game as a reinforcement fine-tuning problem and see how an LLM can learn to plan, analyze feedback, and improve its strategy over time. - Design reward functions that power the reinforcement fine-tuning process. - Learn techniques for evaluating more subjective tasks, such as rating the quality of a text summary, using an LLM as a judge. - Understand why reward hacking happens and how to avoid it by adding penalty functions to discourage undesirable behaviors. - Learn the four key components of the loss calculation in the GRPO algorithm: token probability distribution ratios, advantages, clipping, and KL-divergence. - Launch reinforcement fine-tuning jobs using Predibase’s hosted training services. By the end of this course, you’ll be able to build and fine-tune LLMs using reinforcement learning to improve reasoning without relying on large labeled datasets or subjective human feedback. Please sign up here:

New Course: Reinforcement Fine-Tuning LLMs with GRPO! Learn to use reinforcement learning to improve your LLM performance in this short course, built in collaboration with Predibase by Rubrik, and taught by Travis Addair, its Co-Founder and CTO, and Arnav Garg, its Senior Engineer and Machine Learning Lead. Reasoning models have been one of the most important developments in LLMs. Reinforcement Fine-Tuning (RFT) uses rewards to encourage LLMs to find solutions to multi-step reasoning tasks such as solving math problems and debugging code - without needing pre-existing training examples like in traditional supervised fine-tuning. Group Relative Policy Optimization (GRPO) is a reinforcement fine-tuning algorithm gaining rapid adoption. Developed by the DeepSeek team and used to train the R1 reasoning model, GRPO uses reward functions that you can write in Python to assign rewards to model responses. It’s beneficial for tasks with verifiable outcomes and can work well even with fewer than 100 training examples. It can also significantly improve the reasoning ability of smaller LLMs, making applications faster and more cost effective. In this course, you’ll take a technical deep dive into RFT with GRPO. You’ll learn to build reward functions that you can use in the GRPO training process to guide an LLM toward better performance on multi-step reasoning tasks. In detail, you’ll: - Learn when reinforcement fine-tuning is a better fit than supervised fine-tuning, especially for tasks involving multi-step reasoning or limited labeled data. - Understand how GRPO uses programmable reward functions as a more scalable alternative to the human feedback required for other reinforcement learning algorithms, such as RLHF and DPO. - Frame the Wordle game as a reinforcement fine-tuning problem and see how an LLM can learn to plan, analyze feedback, and improve its strategy over time. - Design reward functions that power the reinforcement fine-tuning process. - Learn techniques for evaluating more subjective tasks, such as rating the quality of a text summary, using an LLM as a judge. - Understand why reward hacking happens and how to avoid it by adding penalty functions to discourage undesirable behaviors. - Learn the four key components of the loss calculation in the GRPO algorithm: token probability distribution ratios, advantages, clipping, and KL-divergence. - Launch reinforcement fine-tuning jobs using Predibase’s hosted training services. By the end of this course, you’ll be able to build and fine-tune LLMs using reinforcement learning to improve reasoning without relying on large labeled datasets or subjective human feedback. Please sign up here:

Andrew Ng

86,672 görüntüleme • 1 yıl önce

This is pure, unfiltered parking-lot rage. A guy pulls up to Dollar General, sees a teal sportbike sitting dead in the marked handicap spot with the blue lines and the “$100 FINE” sign right there, and just loses it for almost a full minute. “Look at that. Look at this entitled piece of sh!t…can drive a whole ass fckn motorcycle and takes a motherfucking handicap spot…My dad, handicapped man whole life… Bet your ass he needed these motherfucking spots more than you. You’re the ultimate kind of entitled piece of fckn sh!t.” Handicap spots exist for a reason and people who take them when they don’t need them are selfish as hell. They aren’t your personal VIP parking space.

This is pure, unfiltered parking-lot rage. A guy pulls up to Dollar General, sees a teal sportbike sitting dead in the marked handicap spot with the blue lines and the “$100 FINE” sign right there, and just loses it for almost a full minute. “Look at that. Look at this entitled piece of sh!t…can drive a whole ass fckn motorcycle and takes a motherfucking handicap spot…My dad, handicapped man whole life… Bet your ass he needed these motherfucking spots more than you. You’re the ultimate kind of entitled piece of fckn sh!t.” Handicap spots exist for a reason and people who take them when they don’t need them are selfish as hell. They aren’t your personal VIP parking space.

DocumentingLibs

59,469 görüntüleme • 25 gün önce

StableRonaldo wasn’t having it after Lacy called him and made it seem like it was the last time they were ever speaking before his full knee reconstruction surgery in a few hours 😭 “I hope this is the f**king last time. I'm sick of your a**. I hope to God that when you go under anesthesia, you wake up and then you say some n*t sh*t and get canceled.”

StableRonaldo wasn’t having it after Lacy called him and made it seem like it was the last time they were ever speaking before his full knee reconstruction surgery in a few hours 😭 “I hope this is the fking last time. I'm sick of your a. I hope to God that when you go under anesthesia, you wake up and then you say some nt sht and get canceled.”

GrimAnt🪽

50,539 görüntüleme • 28 gün önce

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna 👩🏽‍💻✨, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna 👩🏽‍💻✨, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

DeepLearning.AI

54,703 görüntüleme • 1 ay önce

EDP445, who was previously caught messaging & trying to meet up with a 13 year old in 2021, questions why 18 is the legal age of consent while admitting to being attracted to some 14, 15, 16, & 17 year olds 😳💀 “If you 16 or 17 and you got your t*tties bulging tf out, a mothaf*cka might actually wack off to your pictures a few f*cken times… These girls exactly what the f*ck they’re doing… Minors are partially the f*cken blame… So what about the b*tch that messaged me first? Oh thats perfectly fine. N*gga she gets off scott-free right?… What makes 18 so goddamn magical? You talk to a 17 year old chick, she just turned 18 five seconds ago. Well guess what, she’s still the same dumba** b*tch she was 5 seconds ago.”

Sensitive content

EDP445, who was previously caught messaging & trying to meet up with a 13 year old in 2021, questions why 18 is the legal age of consent while admitting to being attracted to some 14, 15, 16, & 17 year olds 😳💀 “If you 16 or 17 and you got your ttties bulging tf out, a mothafcka might actually wack off to your pictures a few fcken times… These girls exactly what the fck they’re doing… Minors are partially the fcken blame… So what about the btch that messaged me first? Oh thats perfectly fine. N*gga she gets off scott-free right?… What makes 18 so goddamn magical? You talk to a 17 year old chick, she just turned 18 five seconds ago. Well guess what, she’s still the same dumba** b*tch she was 5 seconds ago.”

Killa 🌺

588,626 görüntüleme • 22 gün önce

NEW: Trump campaign member eviscerates Allentown, PA Mayor Matthew Tuerk, who was apparently trying to pull permits ahead of Trump's rally. Tuerk is accused of trying to "show how big & strong" he was by obstructing Trump's Allentown rally. "You are out of your f*cking mind. Don't ever pull that sh*t on me again," the Trump campaign member said. "You try to pull some fake sh*t on the permits. You told us on Saturday it was fine. You are trying to pull our permits?" "The election is 8 days away. You said we could close off this entire intersection, and then the day of you show up in a fresh suit with all your cameras trying to show how big and strong you are."

NEW: Trump campaign member eviscerates Allentown, PA Mayor Matthew Tuerk, who was apparently trying to pull permits ahead of Trump's rally. Tuerk is accused of trying to "show how big & strong" he was by obstructing Trump's Allentown rally. "You are out of your fcking mind. Don't ever pull that sht on me again," the Trump campaign member said. "You try to pull some fake sh*t on the permits. You told us on Saturday it was fine. You are trying to pull our permits?" "The election is 8 days away. You said we could close off this entire intersection, and then the day of you show up in a fresh suit with all your cameras trying to show how big and strong you are."

Collin Rugg

4,129,847 görüntüleme • 1 yıl önce

Final Lecture of our Statistical Mechanics Series. Lecture 2 showed how we move from the Full Phase-Space Density ρ(q₁, …, qₙ, p₁, …, pₙ, t) to smaller statistical objects by integrating out variables we do not want to keep. That gives reduced descriptions like the One-Particle Density f₁(q₁,p₁,t) and the Two-Particle Density f₂(q₁,p₁,q₂,p₂,t) This was the simplification. Now comes the catch. If the Full Density obeys Liouville’s Equation, the reduced densities do not evolve independently. The equation for one level depends on the next one and this is referred to as the BBGKY hierarchy. The One-Particle Density depends on the Two-Particle Density. The Two-Particle Density depends on the Three-Particle Density. And the chain keeps going. That happens because particles interact. Once one particle feels the rest, one-particle information is no longer enough. Correlations enter, and the lower level is fed from above. If the full Hamiltonian is H = Σᵢ pᵢ²/(2m) + Σᵢ U(qᵢ) + (1/2) Σᵢ Σⱼ≠ᵢ Φ(qᵢ − qⱼ) then reducing the full density does not make the interaction terms disappear. It leaves behind coupling to higher-order reduced densities. So, schematically, ∂f₁/∂t + transport of one particle = interaction term involving f₂ and more generally the heirarchy is such that ∂fₛ/∂t + s-particle transport = interaction term involving fₛ₊₁ Therefore, Lecture 3 is really about the price of reduction. We simplify the description, but the information we remove comes back as coupling to higher-order correlations. So, how do you actually compute anything if every level depends on the next one? This is the so-called Closure Problem. To make the hierarchy usable, you need an extra assumption that cuts the chain. You replace the exact higher-order object by an approximation in terms of lower-order ones. The most basic example is a factorized closure at the pair level, where the exact correlated Two-Particle Density is replaced schematically by a product of One-Particle Densities: f₂(q₁,p₁,q₂,p₂,t) ≈ f₁(q₁,p₁,t) f₁(q₂,p₂,t) That approximation is not exact. It throws away part of the correlation structure. But it gives you something the raw hierarchy does not... a closed equation for the lower-level description. That is why closure matters so much. Without it, the hierarchy is exact but open. With it, the theory becomes approximate but usable. Thus, the combined point of this final Statistical Mechanics post is simple. First, reduced descriptions are not closed because interactions generate correlations across levels. Second, if you want a workable Kinetic Theory, you must close the hierarchy by approximating those higher-order correlations. It is the bridge from formal many-body mechanics to equations people can actually solve. In the render, that is exactly the story you are seeing. The first part shows the hierarchy itself: one reduced level feeding the next, with lower descriptions inheriting structure from higher ones. The second part shows the closure step where the exact correlated pair level is replaced by a factorized ansatz, and that approximation gives back a closed one-particle description. That is, the animation moves from dependence to approximation, and from approximation to solvability. #StatisticalMechanics #BBGKY #ClosureProblem #KineticTheory #PhaseSpace #ReducedDistribution #HamiltonianMechanics #MathematicalPhysics #Mathematics #Physics

Final Lecture of our Statistical Mechanics Series. Lecture 2 showed how we move from the Full Phase-Space Density ρ(q₁, …, qₙ, p₁, …, pₙ, t) to smaller statistical objects by integrating out variables we do not want to keep. That gives reduced descriptions like the One-Particle Density f₁(q₁,p₁,t) and the Two-Particle Density f₂(q₁,p₁,q₂,p₂,t) This was the simplification. Now comes the catch. If the Full Density obeys Liouville’s Equation, the reduced densities do not evolve independently. The equation for one level depends on the next one and this is referred to as the BBGKY hierarchy. The One-Particle Density depends on the Two-Particle Density. The Two-Particle Density depends on the Three-Particle Density. And the chain keeps going. That happens because particles interact. Once one particle feels the rest, one-particle information is no longer enough. Correlations enter, and the lower level is fed from above. If the full Hamiltonian is H = Σᵢ pᵢ²/(2m) + Σᵢ U(qᵢ) + (1/2) Σᵢ Σⱼ≠ᵢ Φ(qᵢ − qⱼ) then reducing the full density does not make the interaction terms disappear. It leaves behind coupling to higher-order reduced densities. So, schematically, ∂f₁/∂t + transport of one particle = interaction term involving f₂ and more generally the heirarchy is such that ∂fₛ/∂t + s-particle transport = interaction term involving fₛ₊₁ Therefore, Lecture 3 is really about the price of reduction. We simplify the description, but the information we remove comes back as coupling to higher-order correlations. So, how do you actually compute anything if every level depends on the next one? This is the so-called Closure Problem. To make the hierarchy usable, you need an extra assumption that cuts the chain. You replace the exact higher-order object by an approximation in terms of lower-order ones. The most basic example is a factorized closure at the pair level, where the exact correlated Two-Particle Density is replaced schematically by a product of One-Particle Densities: f₂(q₁,p₁,q₂,p₂,t) ≈ f₁(q₁,p₁,t) f₁(q₂,p₂,t) That approximation is not exact. It throws away part of the correlation structure. But it gives you something the raw hierarchy does not... a closed equation for the lower-level description. That is why closure matters so much. Without it, the hierarchy is exact but open. With it, the theory becomes approximate but usable. Thus, the combined point of this final Statistical Mechanics post is simple. First, reduced descriptions are not closed because interactions generate correlations across levels. Second, if you want a workable Kinetic Theory, you must close the hierarchy by approximating those higher-order correlations. It is the bridge from formal many-body mechanics to equations people can actually solve. In the render, that is exactly the story you are seeing. The first part shows the hierarchy itself: one reduced level feeding the next, with lower descriptions inheriting structure from higher ones. The second part shows the closure step where the exact correlated pair level is replaced by a factorized ansatz, and that approximation gives back a closed one-particle description. That is, the animation moves from dependence to approximation, and from approximation to solvability. #StatisticalMechanics #BBGKY #ClosureProblem #KineticTheory #PhaseSpace #ReducedDistribution #HamiltonianMechanics #MathematicalPhysics #Mathematics #Physics

Mathelirium

10,970 görüntüleme • 4 ay önce