Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

1/ Introducing RL Swarm’s new backend: GenRL. A modular reinforcement learning library built for distributed, fault-tolerant training - now powering RL Swarm from the ground up. 🧵

gensyn

93,756 subscribers

82,191 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von gensyn

gensynvor 1 Jahr

2/ Each worker runs its own environment instance, contributes asynchronously to a shared rollout buffer, and updates its model weights independently, so no central controller is required.

Profilbild von gensyn

gensynvor 1 Jahr

3/ GenRL allows RL Swarm to work with any environment, described intuitively through code. This launch incorporates Reasoning Gym out-of-the-box, giving access to >100 community-created environments with no extra configuration required.

Profilbild von gensyn

gensynvor 1 Jahr

4/ What’s new: – Modular GenRL backend – Expanded configuration surface – Prebuilt Docker image for easy deployment – Reasoning Gym environment to enhance model reasoning capabilities – New multi-task swarm

Profilbild von gensyn

gensynvor 1 Jahr

5/ Now live on the Gensyn testnet. You can run RL-Swarm with GenRL today. Full code + setup:

Profilbild von gensyn

gensynvor 1 Jahr

6/ A node update is required for GenRL. Please visit ⁠support-discussion in the Discord if you have any questions.

Profilbild von Gautamgg 🕵

Gautamgg 🕵vor 1 Jahr

I want to ask 1 que What about previous trained model data rewards & participants bec it's not showing Is that data saved in your database? @fenbielding @_jamico @_grieve waiting for ans 💙

Profilbild von Mintair | One Click Node🪄

Mintair | One Click Node🪄vor 1 Jahr

Looks really interesting, we gotta setup our own custom environment.

Profilbild von AJDominic (🐱,🐐)

AJDominic (🐱,🐐)vor 1 Jahr

What gensyn cooking is unmatched!

Profilbild von Bitduke

Bitdukevor 1 Jahr

Cool, cool - more modularity

Profilbild von lior.eth (Lior Messika)

lior.eth (Lior Messika)vor 1 Jahr

These retro vibes are everything I ever wanted from an AI lab

Ähnliche Videos

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

gensyn

228,906 Aufrufe • vor 1 Jahr

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,696 Aufrufe • vor 1 Jahr

Announcing RoboFuME🤖💨, a system for autonomous & efficient real-world robot learning that 1. pre-trains a VLM reward model and a multi-task RL policy from diverse off-the-shelf demo data; 2. runs RL fine-tuning online with the VLM reward model. 🔗 🧵↓

Announcing RoboFuME🤖💨, a system for autonomous & efficient real-world robot learning that 1. pre-trains a VLM reward model and a multi-task RL policy from diverse off-the-shelf demo data; 2. runs RL fine-tuning online with the VLM reward model. 🔗 🧵↓

Jingyun Yang

88,294 Aufrufe • vor 2 Jahren

Reinforcement Learning is the future tense of intelligence. Echo is how it scales. Echo is Gradient’s distributed RL framework, running on everyday consumer devices. From its early experiments, Echo powered a 30B Sokoban model that outperformed DeepSeek-R1 and GPT-OSS-120B.

Reinforcement Learning is the future tense of intelligence. Echo is how it scales. Echo is Gradient’s distributed RL framework, running on everyday consumer devices. From its early experiments, Echo powered a 30B Sokoban model that outperformed DeepSeek-R1 and GPT-OSS-120B.

Gradient

279,191 Aufrufe • vor 1 Jahr

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning investigated the application of Deep Reinforcement Learning (Deep RL) for low-cost, miniature humanoid hardware in a dynamic environment, showing the method can synthesize sophisticated and safe movement skills making up complex behavioral strategies in a simplified one-versus-one (1v1) soccer game abs: project page:

AK

293,384 Aufrufe • vor 3 Jahren

Introducing Muscle v0 -- infinite degrees of freedom, from Daxo Robotics. A different mountain to climb - with a far more beautiful peak. We built this from the ground up: - Ultra-dexterous - Built for machine learning - Durable and robust More below (1/n)

Introducing Muscle v0 -- infinite degrees of freedom, from Daxo Robotics. A different mountain to climb - with a far more beautiful peak. We built this from the ground up: - Ultra-dexterous - Built for machine learning - Durable and robust More below (1/n)

Tom Zhang

274,422 Aufrufe • vor 1 Jahr

🚨 RL for LLMs is finally accessible. Introducing OpenTinker: The first community-driven, open-source framework designed to democratize Reinforcement Learning for LLMs. Inspired by Thinking Machines's amazing Tinker, we realize the biggest bottleneck in agentic LLM research isn’t the math—it’s the setup. Current RL pipelines are messy. Configuring VeRL for every single experiment is a productivity killer. OpenTinker fixed it. 🛠 How OpenTinker Works: Decoupled Design of Server and Client - Setup Once, Run Forever: Configure the OpenTinker backend on your GPU cluster once. - Develop Locally: Define your RL environments directly on your laptop. - Train on the Cloud: Simply point your local client to the backend. The cluster handles the compute; you handle the science. 📉 The 10x Development Efficiency Thanks to our elegant architectural decomposition, OpenTinker reduces the time to develop a new RL training pipeline by at least an order of magnitude. ⚡ Turn Idle GPU Compute into Gold Small labs often have underutilized hardware. OpenTinker turns your idle GPUs into an internal/external API service for - RL Training - SFT - Inference 🎯 Who needs OpenTinker? - Researchers tired of infrastructure hell. - Labs needing to standardize workflows. - Teams wanting to maximize hardware ROI. Thanks my amazing PhD student Siqi Zhu for leading the project. We are building the future of open RL infra. Be the first to build with us. 👇 Start Building with OpenTinker Now 🚀 Repo: 🌐 Blog: If you believe RL should be accessible to everyone, give us a star, repost this 🔄 post, and let us know what agents you plan to build!

🚨 RL for LLMs is finally accessible. Introducing OpenTinker: The first community-driven, open-source framework designed to democratize Reinforcement Learning for LLMs. Inspired by Thinking Machines's amazing Tinker, we realize the biggest bottleneck in agentic LLM research isn’t the math—it’s the setup. Current RL pipelines are messy. Configuring VeRL for every single experiment is a productivity killer. OpenTinker fixed it. 🛠 How OpenTinker Works: Decoupled Design of Server and Client - Setup Once, Run Forever: Configure the OpenTinker backend on your GPU cluster once. - Develop Locally: Define your RL environments directly on your laptop. - Train on the Cloud: Simply point your local client to the backend. The cluster handles the compute; you handle the science. 📉 The 10x Development Efficiency Thanks to our elegant architectural decomposition, OpenTinker reduces the time to develop a new RL training pipeline by at least an order of magnitude. ⚡ Turn Idle GPU Compute into Gold Small labs often have underutilized hardware. OpenTinker turns your idle GPUs into an internal/external API service for - RL Training - SFT - Inference 🎯 Who needs OpenTinker? - Researchers tired of infrastructure hell. - Labs needing to standardize workflows. - Teams wanting to maximize hardware ROI. Thanks my amazing PhD student Siqi Zhu for leading the project. We are building the future of open RL infra. Be the first to build with us. 👇 Start Building with OpenTinker Now 🚀 Repo: 🌐 Blog: If you believe RL should be accessible to everyone, give us a star, repost this 🔄 post, and let us know what agents you plan to build!

Jiaxuan You

58,258 Aufrufe • vor 8 Monaten

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Prime Intellect

1,147,976 Aufrufe • vor 9 Monaten

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs. The result? It breaks down the major engineering barriers to RL, outperforming traditional frameworks in accessibility and ease of deployment, finally making agent training viable for more developers and teams. Project: Code: U Lab: Our report: 📬 #PapersAccepted by Jiqizhixin

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs. The result? It breaks down the major engineering barriers to RL, outperforming traditional frameworks in accessibility and ease of deployment, finally making agent training viable for more developers and teams. Project: Code: U Lab: Our report: 📬 #PapersAccepted by Jiqizhixin

机器之心 JIQIZHIXIN

15,893 Aufrufe • vor 7 Monaten

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

How GPT-5 thinks, with OpenAI VP of Research Jerry Tworek 00:00 - Intro 01:01 - What Reasoning Actually Means in AI 02:32 - Chain of Thought: Models Thinking in Words 05:25 - How Models Decide How Long to Think 07:24 - Evolution from o1 to o3 to GPT-5 11:00 - The Road to OpenAI: Growing up in Poland, Dropping out of School, Trading 20:32 - Working on Robotics and Rubik's Cube Solving 23:02 - A Day in the Life: Talking to Researchers 24:06 - How Research Priorities Are Determined 26:53 - OpenAI's Culture of Transparency 29:32 - Balancing Research with Shipping Fast 31:52 - Using OpenAI's Own Tools Daily 32:43 - Pre-Training Plus RL: The Modern AI Stack 35:10 - Reinforcement Learning 101: Training Dogs 40:17 - The Evolution of Deep Reinforcement Learning 42:09 - When GPT-4 Seemed Underwhelming at First 45:39 - How RLHF Made GPT-4 Actually Useful 48:02 - Unsupervised vs Supervised Learning 49:59 - GRPO and How DeepSeek Accelerated US Research 53:05 - What It Takes to Scale Reinforcement Learning 55:36 - Agentic AI and Long-Horizon Thinking 59:19 - Alignment as an RL Problem 1:01:11 - Winning ICPC World Finals Without Specific Training 1:05:53 - Applying RL Beyond Math and Coding 1:09:15 - The Path from Here to AGI 1:12:23 - Pure RL vs Language Models

Matt Turck

451,229 Aufrufe • vor 10 Monaten

Introducing Sundial!! A brand new text editor built from the ground up for working with agents. Here, we're using Sundial to co-write a new Sundial feature.

Belinda

48,796 Aufrufe • vor 2 Monaten

🤖Adding new RL algorithms to LeRobot just got much easier. Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task. Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails. The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves. The refactor separates algorithm logic from training infrastructure: • RLAlgorithm owns learning logic • RLTrainer handles orchestration • DataMixer combines rollouts, demos, interventions, and future data sources Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry. SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next! Thomas Wolf clem 🤗

🤖Adding new RL algorithms to LeRobot just got much easier. Demo: HIL-SERL training with a SAC-based RL algorithm on an SO-100 for a hole-in-hand peg-in-hole task. Sparse reward, only 30 offline demos mixed with live robot experience, and ~1 hour of online training with human interventions only when the policy fails. The bottom graph tracks intervention rate: high at the start, steadily dropping as the policy improves. The refactor separates algorithm logic from training infrastructure: • RLAlgorithm owns learning logic • RLTrainer handles orchestration • DataMixer combines rollouts, demos, interventions, and future data sources Adding an RL algorithm now looks much closer to adding a policy: one algorithm file, one config, one registry entry. SAC is first. RLT, RECAP, ConRFT, QC-FQL, DSRL, and VLA RL fine-tuning next! Thomas Wolf clem 🤗

LeRobot

30,600 Aufrufe • vor 3 Monaten

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

omkaar

17,218 Aufrufe • vor 5 Monaten

🔥Excited to share our new work: "A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning"! We systematically study what actually works (and what doesn't) for agentic multi-turn RL, breaking down the design space into 3 pillars: 🌎Environment, 🤖Policy, and ⭐Reward. We conduct various ablations and empirical analysis on 🧩TextWorld, 🧙ALFWorld, and 🧑‍💻SWE-Gym. We also open source our codebase: 🐈🍵Meow-Tea-Taro 💜, a modular multi-turn RL framework with configurable environments, policies, and rewards! #ReinforcementLearning #AgenticAI #LLMs #Agents

🔥Excited to share our new work: "A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning"! We systematically study what actually works (and what doesn't) for agentic multi-turn RL, breaking down the design space into 3 pillars: 🌎Environment, 🤖Policy, and ⭐Reward. We conduct various ablations and empirical analysis on 🧩TextWorld, 🧙ALFWorld, and 🧑‍💻SWE-Gym. We also open source our codebase: 🐈🍵Meow-Tea-Taro 💜, a modular multi-turn RL framework with configurable environments, policies, and rewards! #ReinforcementLearning #AgenticAI #LLMs #Agents

Ruiyi Wang 王睿仪

52,974 Aufrufe • vor 10 Monaten

Introducing Reinforcement-Learned Teachers (RLTs): Transforming how we teach LLMs to reason with reinforcement learning (RL). Blog: Paper: Traditional RL focuses on “learning to solve” challenging problems with expensive LLMs and constitutes a key step in making student AI systems ultimately acquire reasoning capabilities via distillation and cold-starting. Enter our RLTs—a new class of models prompted with not only a problem’s question but also its solution, and directly trained to generate clear, step-by-step “explanations” to teach their students. Remarkably, an RLT with only 7B parameters produces superior results when distilling and cold-starting students in competitive and graduate-level reasoning tasks than orders-of-magnitude larger LLMs. RLTs are as effective even when distilling 32B students, much larger than the teacher itself—unlocking a new standard for efficiency in developing reasoning language models with RL. Code:

Introducing Reinforcement-Learned Teachers (RLTs): Transforming how we teach LLMs to reason with reinforcement learning (RL). Blog: Paper: Traditional RL focuses on “learning to solve” challenging problems with expensive LLMs and constitutes a key step in making student AI systems ultimately acquire reasoning capabilities via distillation and cold-starting. Enter our RLTs—a new class of models prompted with not only a problem’s question but also its solution, and directly trained to generate clear, step-by-step “explanations” to teach their students. Remarkably, an RLT with only 7B parameters produces superior results when distilling and cold-starting students in competitive and graduate-level reasoning tasks than orders-of-magnitude larger LLMs. RLTs are as effective even when distilling 32B students, much larger than the teacher itself—unlocking a new standard for efficiency in developing reasoning language models with RL. Code:

Sakana AI

179,362 Aufrufe • vor 1 Jahr

Introducing HiLMa-Res: a hierarchical RL framework for quadrupeds to tackle loco-manipulation tasks with sustained mobility! Designed for general learning tasks (vision-based, state-based, real-world data, etc), the robot now can step over stones🐾/navigate boxes📦/dribble⚽.

Introducing HiLMa-Res: a hierarchical RL framework for quadrupeds to tackle loco-manipulation tasks with sustained mobility! Designed for general learning tasks (vision-based, state-based, real-world data, etc), the robot now can step over stones🐾/navigate boxes📦/dribble⚽.

Zhongyu Li

11,720 Aufrufe • vor 1 Jahr

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Bolei Zhou

25,253 Aufrufe • vor 4 Monaten

Introducing Silvana V2. A revamped user experience, redesigned from the ground up. And a new addition to the platform, the Silvana Trading Terminal. Built for the ultimate trading experience on @cantonnetwork. Welcome to a new era.

Introducing Silvana V2. A revamped user experience, redesigned from the ground up. And a new addition to the platform, the Silvana Trading Terminal. Built for the ultimate trading experience on @cantonnetwork. Welcome to a new era.

Silvana

11,442 Aufrufe • vor 1 Monat

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Thomas Wolf

58,901 Aufrufe • vor 1 Monat

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning *beyond* just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

So we did a bunch of projects with real world reinforcement learning - but it was often too inefficient to be practical to train tabula rasa. This suggests we need better priors, but acquiring these from on-robot data can often be expensive as well. In our recent work, we show that despite being fundamentally inaccurate, simulation can guide provide a cheap way to guide real-world RL finetuning to be super efficient! We propose Simulation-Guided Fine-Tuning (SGFT) - a simple paradigm for sim2real finetuning that uses simulation to provide reward shaping that accelerates real world RL finetuning beyond just providing an initialization. TLDR: Use value functions from sim to shape rewards for real-world RL, see large sample efficiency improvements 🧵(1/6)

Abhishek Gupta

13,637 Aufrufe • vor 1 Jahr