Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs.... show more

机器之心 JIQIZHIXIN

18,848 subscribers

15,893 views • 9 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

gensyn

228,943 views • 1 year ago

SITUATION EXPLAINED: Richard Sutton, the father of reinforcement learning, just launched a new lab. The goal: a trillion-parameter agent that learns in real time on 20 watts. • Sutton wrote The Bitter Lesson in 2019, its core claim: general methods that leverage computation beat hand-crafted approaches, by a large margin • His new lab, Oak Lab, builds on the "big world hypothesis": the world is too big for any model to pre-learn everything, so trying to is wasted effort by construction • Their "batch size one" algorithm updates from a single live experience as it arrives, instead of training on giant curated datasets... no massive pretraining run at all • Claimed result: multiple orders of magnitude less compute and energy than existing methods • The holy grail number: a trillion-parameter agent that learns and plans in real time on 20 watts... roughly what the human brain runs on • It's built for agentic tasks specifically, not as a general-purpose LLM replacement sof 𓋹: "It's possible that we end up looking back at LLMs as an evolution of search... not to hate on LLMs, they're very useful, they're awesome. But eventually I think that's what it ends up looking like."

SITUATION EXPLAINED: Richard Sutton, the father of reinforcement learning, just launched a new lab. The goal: a trillion-parameter agent that learns in real time on 20 watts. • Sutton wrote The Bitter Lesson in 2019, its core claim: general methods that leverage computation beat hand-crafted approaches, by a large margin • His new lab, Oak Lab, builds on the "big world hypothesis": the world is too big for any model to pre-learn everything, so trying to is wasted effort by construction • Their "batch size one" algorithm updates from a single live experience as it arrives, instead of training on giant curated datasets... no massive pretraining run at all • Claimed result: multiple orders of magnitude less compute and energy than existing methods • The holy grail number: a trillion-parameter agent that learns and plans in real time on 20 watts... roughly what the human brain runs on • It's built for agentic tasks specifically, not as a general-purpose LLM replacement sof 𓋹: "It's possible that we end up looking back at LLMs as an evolution of search... not to hate on LLMs, they're very useful, they're awesome. But eventually I think that's what it ends up looking like."

MTS

83,455 views • 3 months ago

This is the Claude Code moment for Reinforcement Learning. Every frontier lab knows the secret: post-training is where the magic happens. If you can eval a task, you can use RL to benchmax your model. But building and scaling the RL loop has remained a dark art reserved for the top 1% of AI Labs. Until today. Introducing Optimus — an autonomous RL Agent Harness, and the first of many we are about to ship Iacon Autonomics. We are removing the manual grind of RL and adding taste and relentlessness to the process. Just hand Optimus a codebase, an API key, and a goal. It spins up a fleet of sub-agents to run hundreds of parallel experiments. It learns from its own history, autonomously building an evolutionary tree of runs where every failure sets the next batch up to win. Train LLMs, agents, robots in sim, game npcs and much more. If you can score it, Optimus can train it. Stop doing manual RL. Let the agents cook.

This is the Claude Code moment for Reinforcement Learning. Every frontier lab knows the secret: post-training is where the magic happens. If you can eval a task, you can use RL to benchmax your model. But building and scaling the RL loop has remained a dark art reserved for the top 1% of AI Labs. Until today. Introducing Optimus — an autonomous RL Agent Harness, and the first of many we are about to ship Iacon Autonomics. We are removing the manual grind of RL and adding taste and relentlessness to the process. Just hand Optimus a codebase, an API key, and a goal. It spins up a fleet of sub-agents to run hundreds of parallel experiments. It learns from its own history, autonomously building an evolutionary tree of runs where every failure sets the next batch up to win. Train LLMs, agents, robots in sim, game npcs and much more. If you can score it, Optimus can train it. Stop doing manual RL. Let the agents cook.

Dev Shah

13,730 views • 1 month ago

Chinese researchers just published a paper with a brutal title: "The End of Software Engineering." their argument: writing code is finished as a skill a human brain holds only so much state and so many dependencies before it stalls an agent has no such ceiling, and its capacity climbs with compute so the person stops being the one typing the code you turn into the intent architect - you direct the agents and audit what comes back code turns disposable, judgment turns into the job that's why i wrote the guide below on building your own agent system from scratch the paper it's based on is down there too

Chinese researchers just published a paper with a brutal title: "The End of Software Engineering." their argument: writing code is finished as a skill a human brain holds only so much state and so many dependencies before it stalls an agent has no such ceiling, and its capacity climbs with compute so the person stops being the one typing the code you turn into the intent architect - you direct the agents and audit what comes back code turns disposable, judgment turns into the job that's why i wrote the guide below on building your own agent system from scratch the paper it's based on is down there too

Mr. Buzzoni

57,204 views • 1 month ago

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

Matt Dancho (Business Science)

11,799 views • 1 year ago

AI power users are here and the case for agent-friendly software is materializing. Aaron Levie on agents outnumbering people 1000x: "If you start to imagine that we all have to build software for agents… we spend as much time now thinking about the agent interface to our tool as we do the human interface." "If you have a hundred or a thousand times more agents than people, then your software has to be built for agents… it's going to be through an API or a CLI or MCP." "What if you give a coding agent access to your SaaS tools and a coding agent access to your knowledge work, workflows and context?" "It can actually code its way or use APIs through whatever task it's trying to achieve." "That appears to be like a paradigm that is starting to compound… and I actually think it kind of makes sense as the ultimate manifestation of this stuff." Aaron Levie

AI power users are here and the case for agent-friendly software is materializing. Aaron Levie on agents outnumbering people 1000x: "If you start to imagine that we all have to build software for agents… we spend as much time now thinking about the agent interface to our tool as we do the human interface." "If you have a hundred or a thousand times more agents than people, then your software has to be built for agents… it's going to be through an API or a CLI or MCP." "What if you give a coding agent access to your SaaS tools and a coding agent access to your knowledge work, workflows and context?" "It can actually code its way or use APIs through whatever task it's trying to achieve." "That appears to be like a paradigm that is starting to compound… and I actually think it kind of makes sense as the ultimate manifestation of this stuff." Aaron Levie

a16z

42,152 views • 1 month ago

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,211 views • 1 year ago

Your next robotics project could start with these steps. Today, we're open-sourcing Asimov 1's locomotion policy and the training code behind it, giving developers a foundation to build on. Alongside the trained policy checkpoint, we're sharing our Isaac Lab-based training code, including reward definitions, hardware-tuned actuator configurations, and domain randomization settings. That foundation gives you a starting point for your own experiments. Adapt the training setup to hardware changes. Explore different walking styles. Build toward new capabilities. We've taken the first steps. Build what comes next. Asimov 1 locomotion:

Your next robotics project could start with these steps. Today, we're open-sourcing Asimov 1's locomotion policy and the training code behind it, giving developers a foundation to build on. Alongside the trained policy checkpoint, we're sharing our Isaac Lab-based training code, including reward definitions, hardware-tuned actuator configurations, and domain randomization settings. That foundation gives you a starting point for your own experiments. Adapt the training setup to hardware changes. Explore different walking styles. Build toward new capabilities. We've taken the first steps. Build what comes next. Asimov 1 locomotion:

Asimov

62,799 views • 20 days ago

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Thomas Wolf

59,254 views • 3 months ago

From lab to open-source: A new milestone for AI-driven education. 🎓 🤗 We’ve been closely following the MAIC project at Tsinghua University, and we’re thrilled to see it now open-sourced as #OpenMAIC. ✨ This isn't just another chatbot; it takes Multi-Agent orchestration to the next level by building a fully interactive classroom where AI instructors and peers collaborate in real-time. What makes it technically impressive: 🛠️ Complex Orchestration: Leveraging #LangGraph to manage spontaneous interactions—like #AI students "raising hands" during a live lecture. 🧠 Structured Planning: A dedicated "Plan Agent" that transforms raw PDFs into coherent, logically sequenced pedagogical flows. 💻 Beyond Text: A masterclass in GenUI implementation, featuring synchronized TTS, laser pointers, and real-time whiteboard demonstrations. 🥳 If you’re building complex, multi-modal #Agent workflows, this repo is a treasure trove of engineering insights. 🖥️Explore the project: 📰 Read the research:

From lab to open-source: A new milestone for AI-driven education. 🎓 🤗 We’ve been closely following the MAIC project at Tsinghua University, and we’re thrilled to see it now open-sourced as #OpenMAIC. ✨ This isn't just another chatbot; it takes Multi-Agent orchestration to the next level by building a fully interactive classroom where AI instructors and peers collaborate in real-time. What makes it technically impressive: 🛠️ Complex Orchestration: Leveraging #LangGraph to manage spontaneous interactions—like #AI students "raising hands" during a live lecture. 🧠 Structured Planning: A dedicated "Plan Agent" that transforms raw PDFs into coherent, logically sequenced pedagogical flows. 💻 Beyond Text: A masterclass in GenUI implementation, featuring synchronized TTS, laser pointers, and real-time whiteboard demonstrations. 🥳 If you’re building complex, multi-modal #Agent workflows, this repo is a treasure trove of engineering insights. 🖥️Explore the project: 📰 Read the research:

OpenBMB

152,659 views • 7 months ago

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Avi Chawla

351,173 views • 4 months ago

Can reinforcement learning from AI feedback unlock new capabilities in AI agents? Introducing Motif, an LLM-powered method for intrinsic motivation from AI feedback. Motif extracts reward functions from Llama 2's preferences and uses them to train agents with reinforcement learning. On the complex NetHack game, Motif solves previously unsolved tasks without needing any expert demonstrations. Surprisingly, Motif's reward leads to better game score than the one obtained by using the score itself as a reward. Given access to an event captioning mechanism, a few properties make Motif a general method: • it is entirely based on open models • the LLM doesn't need direct access to the environment dynamics (e.g., its source code) • the LLM doesn't need to understand observation and action spaces The best part? You can start using Motif right now, even on a small compute budget: the whole pipeline can take less than two GPU-days. Feel free to read our paper and try our code out. Paper: Code: Blog post: Work co-lead by Martin Klissarov and myself, with Shagun Sodhani Roberta Raileanu Pierre-Luc Bacon Pascal Vincent @yayitsamyzhang Mikael Henaff Learn more in the thread 🧵

Can reinforcement learning from AI feedback unlock new capabilities in AI agents? Introducing Motif, an LLM-powered method for intrinsic motivation from AI feedback. Motif extracts reward functions from Llama 2's preferences and uses them to train agents with reinforcement learning. On the complex NetHack game, Motif solves previously unsolved tasks without needing any expert demonstrations. Surprisingly, Motif's reward leads to better game score than the one obtained by using the score itself as a reward. Given access to an event captioning mechanism, a few properties make Motif a general method: • it is entirely based on open models • the LLM doesn't need direct access to the environment dynamics (e.g., its source code) • the LLM doesn't need to understand observation and action spaces The best part? You can start using Motif right now, even on a small compute budget: the whole pipeline can take less than two GPU-days. Feel free to read our paper and try our code out. Paper: Code: Blog post: Work co-lead by Martin Klissarov and myself, with Shagun Sodhani Roberta Raileanu Pierre-Luc Bacon Pascal Vincent @yayitsamyzhang Mikael Henaff Learn more in the thread 🧵

Pierluca D'Oro

312,048 views • 3 years ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 views • 4 months ago

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

idan levin

28,307 views • 1 month ago

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:

If you're looking for a weekend project, how about training your own text-to-speech model from scratch on your own GPU, and then running it on any device's CPU? We just open-sourced the entire Pocket TTS training stack: data pipeline, recipes, and evals. It learns pretty damn fast: ~15k steps: babbling starts turning into words ~50k steps: it reads anything you type (WER under 1%) ~200k steps: the voice stops sounding synthetic On a beefy consumer GPU, that's a week of training. On eight H100s: 10-20 hours. A TTS training run will cost you less than $200 if you rent your hardware, and an order of magnitude less if you just pay for power. Some things we'd love to see people try: - Train it in your own language (a few hundred hours of speech gets you surprisingly far). - Add new features to Pocket TTS (Emotion tags? Make it sing?). - Beat us at our own game: make it faster and smaller. Show us what you build! We'll highlight the best models and new languages for the whole community to enjoy. Pocket TTS has already found many use cases, from reading for people with visual impairments to making NPCs in video games talk, and we're sure there's much more to do with it! Here's an example of a Czech Pocket TTS. Try just asking your favorite agent to find data and apply the method, and you can have your own. Get started:

kyutai

106,234 views • 1 month ago

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 views • 1 year ago

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

Andrew Ng

105,839 views • 1 year ago

It's difficult for a team to become something its coach is not. “As a coach, you can’t just wear the sweatshirt, you have to sweat too. You got to do the work." -- Tara VanDerveer Culture is built through what a coach models just as much (if not more) than the messages they give. If you want a team that handles adversity, you show them what poise looks like. If you want a team that embraces feedback, you show them you’re actively learning and open to growth too. Culture doesn’t start with players. It starts with the leader. 📹: The Threshold Lab Podcast

It's difficult for a team to become something its coach is not. “As a coach, you can’t just wear the sweatshirt, you have to sweat too. You got to do the work." -- Tara VanDerveer Culture is built through what a coach models just as much (if not more) than the messages they give. If you want a team that handles adversity, you show them what poise looks like. If you want a team that embraces feedback, you show them you’re actively learning and open to growth too. Culture doesn’t start with players. It starts with the leader. 📹: The Threshold Lab Podcast

Zach Brandon

113,420 views • 10 months ago