Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

A free MIT course about deep learning, 2026 edition: Here, MIT CSAIL researcher Alexander Amini discusses the differences between supervised, unsupervised, & reinforcement learning approaches (Lecture 5).

MIT CSAIL

347,314 subscribers

17,457 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Free MIT course breaks down foundation models & genAI: Here, CSAIL researcher & lead lecturer Rickard Brüel Gabrielsson discusses AGI & a key innovation behind AI systems: learning from observation (Lecture 1).

Free MIT course breaks down foundation models & genAI: Here, CSAIL researcher & lead lecturer Rickard Brüel Gabrielsson discusses AGI & a key innovation behind AI systems: learning from observation (Lecture 1).

MIT CSAIL

11,973 Aufrufe • vor 11 Monaten

A free MIT course about building scalable & high-performance software systems: (v/MIT OpenCourseWare) Here, MIT prof. & CSAIL principal investigator Charles Leiserson discusses why performance matters in computing (Lecture 1).

A free MIT course about building scalable & high-performance software systems: (v/MIT OpenCourseWare) Here, MIT prof. & CSAIL principal investigator Charles Leiserson discusses why performance matters in computing (Lecture 1).

MIT CSAIL

61,054 Aufrufe • vor 5 Monaten

Free MIT course breaks down deep learning: Here, MIT ass't prof. Sara Beery discusses how it's driven breakthroughs in areas like image generation, coding, & playing games (Lecture 1).

Free MIT course breaks down deep learning: Here, MIT ass't prof. Sara Beery discusses how it's driven breakthroughs in areas like image generation, coding, & playing games (Lecture 1).

MIT CSAIL

20,456 Aufrufe • vor 1 Monat

A free MIT course breaking down mathematics for computer science & engineering: (v/MIT OpenCourseWare) Here, MIT prof. Tom Leighton, who is also the CEO & co-founder of Akamai, discusses what a proof is (Lesson 1).

A free MIT course breaking down mathematics for computer science & engineering: (v/MIT OpenCourseWare) Here, MIT prof. Tom Leighton, who is also the CEO & co-founder of Akamai, discusses what a proof is (Lesson 1).

MIT CSAIL

57,854 Aufrufe • vor 7 Monaten

Watch "Endless experimentation: Building AI models in the wild." This is a 50-minute, MIT lecture about problems when deploying models and LLMs in the real world and how to prepare to solve them. This is a great lecture for those building production machine learning.

Watch "Endless experimentation: Building AI models in the wild." This is a 50-minute, MIT lecture about problems when deploying models and LLMs in the real world and how to prepare to solve them. This is a great lecture for those building production machine learning.

Santiago

117,204 Aufrufe • vor 2 Jahren

This 1 hour MIT lecture about "Machine Learning" will tell you more about how to build your own ML model than 2 months of internship at OpenAI. Bookmark this & give it 1 hour today, no matter what. This will change the way you use AI. Then read the post below.

This 1 hour MIT lecture about "Machine Learning" will tell you more about how to build your own ML model than 2 months of internship at OpenAI. Bookmark this & give it 1 hour today, no matter what. This will change the way you use AI. Then read the post below.

Movez

83,797 Aufrufe • vor 3 Monaten

Super excited to launch a new AI course! 🚀 Fine-Tuning & Reinforcement Learning for LLMs: Intro to Post-Training A collaboration between AMD 🤝 Andrew Ng’s DeepLearning.AI to give every developer the tools & compute to work with the same post-training techniques, used across today’s leading AI labs. 🎓 Learn for free → 🧵

Super excited to launch a new AI course! 🚀 Fine-Tuning & Reinforcement Learning for LLMs: Intro to Post-Training A collaboration between AMD 🤝 Andrew Ng’s DeepLearning.AI to give every developer the tools & compute to work with the same post-training techniques, used across today’s leading AI labs. 🎓 Learn for free → 🧵

Sharon Zhou

20,386 Aufrufe • vor 8 Monaten

this 1 hour MIT lecture on portfolio management is the single best video from the Mathematics in Finance series. a former Morgan Stanley trader teaches you more about diversification and position sizing than most learn in years. 26 lectures in the course. all free. save this.

this 1 hour MIT lecture on portfolio management is the single best video from the Mathematics in Finance series. a former Morgan Stanley trader teaches you more about diversification and position sizing than most learn in years. 26 lectures in the course. all free. save this.

Goshawk Trades

122,667 Aufrufe • vor 3 Monaten

Lung cancer is the leading cause of cancer-related deaths worldwide. A deep learning model developed by MIT & MGH researchers called "Sybil" can accurately predict a patient’s risk of lung cancer up to 6 years in advance by analyzing a low-dose CT scan:

Lung cancer is the leading cause of cancer-related deaths worldwide. A deep learning model developed by MIT & MGH researchers called "Sybil" can accurately predict a patient’s risk of lung cancer up to 6 years in advance by analyzing a low-dose CT scan:

MIT CSAIL

41,408 Aufrufe • vor 11 Monaten

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

Matt Turck

451,229 Aufrufe • vor 9 Monaten

Learn more about Boltz-2, the new open source AI model from MIT and Recursion capable of predicting protein binding affinity with unprecedented speed, scale and accuracy. ▪️ Boltz-2 is the first model to combine structure and binding affinity prediction, approaching the accuracy of physics-based free energy perturbation (FEP) calculations while being over 1,000 times faster and less computationally expensive. 💥 The model addresses a critical bottleneck in small molecule drug discovery. ▪️ A powerful tool built on novel machine learning: “By predicting both molecular structure and binding affinity simultaneously with unprecedented speed and scale, Boltz-2 gives R&D teams a powerful tool to triage more effectively and focus resources on the most promising compounds,” says Najat Khan, PhD, Chief R&D Officer and Chief Commercial Officer at Recursion. ▪️ The added power of open source: “Because Boltz-2 is open-source, including its training code, scientists can easily adapt it for specific types of molecules, making it even more powerful as a tool to accelerate discovery," says Regina Barzilay, PhD, MIT School of Engineering Distinguished Professor for AI and Health, AI faculty lead at MIT Jameel Clinic for AI & Health and MIT CSAIL principal investigator. 👉 Learn more, read the preprint, and access Boltz-2 here: 📅 Join us for live presentations, demos and discussions: 📎 MIT Presentation (Cambridge) – Monday, June 9. 📎 NVIDIA #GTC25 (Paris) – Wednesday, June 11. 📎 Molecular Machine Learning Conference - MoML (Montreal) – Tuesday, June 17.

Learn more about Boltz-2, the new open source AI model from MIT and Recursion capable of predicting protein binding affinity with unprecedented speed, scale and accuracy. ▪️ Boltz-2 is the first model to combine structure and binding affinity prediction, approaching the accuracy of physics-based free energy perturbation (FEP) calculations while being over 1,000 times faster and less computationally expensive. 💥 The model addresses a critical bottleneck in small molecule drug discovery. ▪️ A powerful tool built on novel machine learning: “By predicting both molecular structure and binding affinity simultaneously with unprecedented speed and scale, Boltz-2 gives R&D teams a powerful tool to triage more effectively and focus resources on the most promising compounds,” says Najat Khan, PhD, Chief R&D Officer and Chief Commercial Officer at Recursion. ▪️ The added power of open source: “Because Boltz-2 is open-source, including its training code, scientists can easily adapt it for specific types of molecules, making it even more powerful as a tool to accelerate discovery," says Regina Barzilay, PhD, MIT School of Engineering Distinguished Professor for AI and Health, AI faculty lead at MIT Jameel Clinic for AI & Health and MIT CSAIL principal investigator. 👉 Learn more, read the preprint, and access Boltz-2 here: 📅 Join us for live presentations, demos and discussions: 📎 MIT Presentation (Cambridge) – Monday, June 9. 📎 NVIDIA #GTC25 (Paris) – Wednesday, June 11. 📎 Molecular Machine Learning Conference - MoML (Montreal) – Tuesday, June 17.

Recursion

15,593 Aufrufe • vor 1 Jahr

New Course: Reinforcement Fine-Tuning LLMs with GRPO! Learn to use reinforcement learning to improve your LLM performance in this short course, built in collaboration with Predibase by Rubrik, and taught by Travis Addair, its Co-Founder and CTO, and Arnav Garg, its Senior Engineer and Machine Learning Lead. Reasoning models have been one of the most important developments in LLMs. Reinforcement Fine-Tuning (RFT) uses rewards to encourage LLMs to find solutions to multi-step reasoning tasks such as solving math problems and debugging code - without needing pre-existing training examples like in traditional supervised fine-tuning. Group Relative Policy Optimization (GRPO) is a reinforcement fine-tuning algorithm gaining rapid adoption. Developed by the DeepSeek team and used to train the R1 reasoning model, GRPO uses reward functions that you can write in Python to assign rewards to model responses. It’s beneficial for tasks with verifiable outcomes and can work well even with fewer than 100 training examples. It can also significantly improve the reasoning ability of smaller LLMs, making applications faster and more cost effective. In this course, you’ll take a technical deep dive into RFT with GRPO. You’ll learn to build reward functions that you can use in the GRPO training process to guide an LLM toward better performance on multi-step reasoning tasks. In detail, you’ll: - Learn when reinforcement fine-tuning is a better fit than supervised fine-tuning, especially for tasks involving multi-step reasoning or limited labeled data. - Understand how GRPO uses programmable reward functions as a more scalable alternative to the human feedback required for other reinforcement learning algorithms, such as RLHF and DPO. - Frame the Wordle game as a reinforcement fine-tuning problem and see how an LLM can learn to plan, analyze feedback, and improve its strategy over time. - Design reward functions that power the reinforcement fine-tuning process. - Learn techniques for evaluating more subjective tasks, such as rating the quality of a text summary, using an LLM as a judge. - Understand why reward hacking happens and how to avoid it by adding penalty functions to discourage undesirable behaviors. - Learn the four key components of the loss calculation in the GRPO algorithm: token probability distribution ratios, advantages, clipping, and KL-divergence. - Launch reinforcement fine-tuning jobs using Predibase’s hosted training services. By the end of this course, you’ll be able to build and fine-tune LLMs using reinforcement learning to improve reasoning without relying on large labeled datasets or subjective human feedback. Please sign up here:

New Course: Reinforcement Fine-Tuning LLMs with GRPO! Learn to use reinforcement learning to improve your LLM performance in this short course, built in collaboration with Predibase by Rubrik, and taught by Travis Addair, its Co-Founder and CTO, and Arnav Garg, its Senior Engineer and Machine Learning Lead. Reasoning models have been one of the most important developments in LLMs. Reinforcement Fine-Tuning (RFT) uses rewards to encourage LLMs to find solutions to multi-step reasoning tasks such as solving math problems and debugging code - without needing pre-existing training examples like in traditional supervised fine-tuning. Group Relative Policy Optimization (GRPO) is a reinforcement fine-tuning algorithm gaining rapid adoption. Developed by the DeepSeek team and used to train the R1 reasoning model, GRPO uses reward functions that you can write in Python to assign rewards to model responses. It’s beneficial for tasks with verifiable outcomes and can work well even with fewer than 100 training examples. It can also significantly improve the reasoning ability of smaller LLMs, making applications faster and more cost effective. In this course, you’ll take a technical deep dive into RFT with GRPO. You’ll learn to build reward functions that you can use in the GRPO training process to guide an LLM toward better performance on multi-step reasoning tasks. In detail, you’ll: - Learn when reinforcement fine-tuning is a better fit than supervised fine-tuning, especially for tasks involving multi-step reasoning or limited labeled data. - Understand how GRPO uses programmable reward functions as a more scalable alternative to the human feedback required for other reinforcement learning algorithms, such as RLHF and DPO. - Frame the Wordle game as a reinforcement fine-tuning problem and see how an LLM can learn to plan, analyze feedback, and improve its strategy over time. - Design reward functions that power the reinforcement fine-tuning process. - Learn techniques for evaluating more subjective tasks, such as rating the quality of a text summary, using an LLM as a judge. - Understand why reward hacking happens and how to avoid it by adding penalty functions to discourage undesirable behaviors. - Learn the four key components of the loss calculation in the GRPO algorithm: token probability distribution ratios, advantages, clipping, and KL-divergence. - Launch reinforcement fine-tuning jobs using Predibase’s hosted training services. By the end of this course, you’ll be able to build and fine-tune LLMs using reinforcement learning to improve reasoning without relying on large labeled datasets or subjective human feedback. Please sign up here:

Andrew Ng

86,457 Aufrufe • vor 1 Jahr

Here's my conversation with Edward Gibson (Ted Gibson, Language Lab MIT), a linguist and psychologist at MIT, heading the MIT Language Lab. We talk all about the human language: syntax, grammar, structure, theories of language, evolution of language, how it reflects culture, and of course LLMs, both their amazing power and their limitations. It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 1:13 - Human language 5:19 - Generalizations in language 11:06 - Dependency grammar 21:05 - Morphology 29:40 - Evolution of languages 33:00 - Noam Chomsky 1:17:06 - Thinking and language 1:30:36 - LLMs 1:43:35 - Center embedding 2:10:02 - Learning a new language 2:13:54 - Nature vs nurture 2:20:30 - Culture and language 2:34:58 - Universal language 2:39:21 - Language translation 2:42:36 - Animal communication

Here's my conversation with Edward Gibson (Ted Gibson, Language Lab MIT), a linguist and psychologist at MIT, heading the MIT Language Lab. We talk all about the human language: syntax, grammar, structure, theories of language, evolution of language, how it reflects culture, and of course LLMs, both their amazing power and their limitations. It's here on X in full, and is up on YouTube, Spotify, and everywhere else. Links in comment. Timestamps: 0:00 - Introduction 1:13 - Human language 5:19 - Generalizations in language 11:06 - Dependency grammar 21:05 - Morphology 29:40 - Evolution of languages 33:00 - Noam Chomsky 1:17:06 - Thinking and language 1:30:36 - LLMs 1:43:35 - Center embedding 2:10:02 - Learning a new language 2:13:54 - Nature vs nurture 2:20:30 - Culture and language 2:34:58 - Universal language 2:39:21 - Language translation 2:42:36 - Animal communication

Lex Fridman

340,246 Aufrufe • vor 2 Jahren

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

Andrew Ng

132,304 Aufrufe • vor 8 Monaten

The Man Who Might SOLVE AI Alignment 🦸‍♂️ Dr. Steven Byrnes (Steven Byrnes) is one of the few people who both understands why alignment is hard, and is taking a serious technical shot at solving it. He has a whopping 90% P(Doom), but argues that LLMs will plateau before becoming truly dangerous, and the real threat will come from next-generation “brain-like AGI” based on actor-critic reinforcement learning. After his UC Berkeley physics PhD & Harvard postdoc, he became an AGI safety researcher at Astera Institute. He's now deep in the neuroscience of reverse-engineering how human brains actually work, knowledge that could plausibly help us solve the technical AI alignment problem. In this unusually deep episode — because with all due respect to my other guests, I didn't have to waste any time arguing with him about obvious surface issues — we talk about his "two subsystems" model of the brain, why current AI safety approaches miss the mark, his disagreements with social evolution approaches, and why understanding human neuroscience matters for building aligned AGI.

The Man Who Might SOLVE AI Alignment 🦸‍♂️ Dr. Steven Byrnes (Steven Byrnes) is one of the few people who both understands why alignment is hard, and is taking a serious technical shot at solving it. He has a whopping 90% P(Doom), but argues that LLMs will plateau before becoming truly dangerous, and the real threat will come from next-generation “brain-like AGI” based on actor-critic reinforcement learning. After his UC Berkeley physics PhD & Harvard postdoc, he became an AGI safety researcher at Astera Institute. He's now deep in the neuroscience of reverse-engineering how human brains actually work, knowledge that could plausibly help us solve the technical AI alignment problem. In this unusually deep episode — because with all due respect to my other guests, I didn't have to waste any time arguing with him about obvious surface issues — we talk about his "two subsystems" model of the brain, why current AI safety approaches miss the mark, his disagreements with social evolution approaches, and why understanding human neuroscience matters for building aligned AGI.

Liron Shapira

10,542 Aufrufe • vor 11 Monaten

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,771 Aufrufe • vor 1 Jahr

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,533 Aufrufe • vor 6 Monaten

MIT HANDED ITS DEEP LEARNING COURSE TO A FRONTIER-LAB ENGINEER FOR 68 MINUTES BECAUSE 90% OF PEOPLE SHIPPING AI CODE CAN'T EXPLAIN HOW THE MODEL ACTUALLY WORKS This is Maxime Labonne. He runs post-training at Liquid AI and wrote the LLM Engineer's Handbook. MIT gave him the room to break down the engine sitting inside every coding agent you've ever prompted. Twenty minutes in it stops being abstract. You finally see why the model confidently invents things that don't exist, why context is everything, and why "Just tell it to try again" sometimes fixes it and sometimes makes it worse. In 2026 "I use AI to code" stopped being a skill. Knowing why the model behaves the way it does -> tokens, context, post-training, where it quietly breaks -> is what separates someone who ships from someone babysitting a black box. Understanding an LLM isn't a research-team luxury anymore -> it's the difference between driving the agent and being driven by it. Anyone can prompt. The person who knows what's under the hood is the one still standing when the prompt stops working. Save this one & actually finish it ↓

MIT HANDED ITS DEEP LEARNING COURSE TO A FRONTIER-LAB ENGINEER FOR 68 MINUTES BECAUSE 90% OF PEOPLE SHIPPING AI CODE CAN'T EXPLAIN HOW THE MODEL ACTUALLY WORKS This is Maxime Labonne. He runs post-training at Liquid AI and wrote the LLM Engineer's Handbook. MIT gave him the room to break down the engine sitting inside every coding agent you've ever prompted. Twenty minutes in it stops being abstract. You finally see why the model confidently invents things that don't exist, why context is everything, and why "Just tell it to try again" sometimes fixes it and sometimes makes it worse. In 2026 "I use AI to code" stopped being a skill. Knowing why the model behaves the way it does -> tokens, context, post-training, where it quietly breaks -> is what separates someone who ships from someone babysitting a black box. Understanding an LLM isn't a research-team luxury anymore -> it's the difference between driving the agent and being driven by it. Anyone can prompt. The person who knows what's under the hood is the one still standing when the prompt stops working. Save this one & actually finish it ↓

slash1s

236,758 Aufrufe • vor 1 Monat

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Davide Scaramuzza

27,090 Aufrufe • vor 6 Monaten

Alan Oppenheim, MIT professor: "Jump Trading pays quants $650K to pull one real signal out of market noise. I wrote the textbook on that exact math and taught it for free." this free MIT lecture is the math the whole "quant edge" is built on, and almost nobody selling you a course will name it. the man teaching it wrote the signal-processing textbook every engineer on earth learned from, and he put the lectures online for nothing. here is the idea. a price chart is a signal buried in noise. most of what you see is random static, and hidden inside is a faint, repeating pattern with real mathematics for pulling it out. that is not a metaphor. jim simons was a codebreaker who did exactly this to enemy transmissions, then pointed the same math at the market and made a hundred billion dollars. none of it is new. oppenheim taught this in 1987, the math is older, and every lecture is free. same story i told in the article above. the edge was never a secret formula. it was knowing a market is a noisy signal, and having the tools to read it. here is what the gurus skip. the math to separate signal from noise is free and everyone has it. the edge is cleaner data than the next desk, and the discipline to trust a faint signal across thousands of trades instead of chasing the static. the filter is free. the patience to use it is not.

Alan Oppenheim, MIT professor: "Jump Trading pays quants $650K to pull one real signal out of market noise. I wrote the textbook on that exact math and taught it for free." this free MIT lecture is the math the whole "quant edge" is built on, and almost nobody selling you a course will name it. the man teaching it wrote the signal-processing textbook every engineer on earth learned from, and he put the lectures online for nothing. here is the idea. a price chart is a signal buried in noise. most of what you see is random static, and hidden inside is a faint, repeating pattern with real mathematics for pulling it out. that is not a metaphor. jim simons was a codebreaker who did exactly this to enemy transmissions, then pointed the same math at the market and made a hundred billion dollars. none of it is new. oppenheim taught this in 1987, the math is older, and every lecture is free. same story i told in the article above. the edge was never a secret formula. it was knowing a market is a noisy signal, and having the tools to read it. here is what the gurus skip. the math to separate signal from noise is free and everyone has it. the edge is cleaner data than the next desk, and the discipline to trust a faint signal across thousands of trades instead of chasing the static. the filter is free. the patience to use it is not.

Rossst.03

269,381 Aufrufe • vor 14 Tagen