正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

What if you could train AI agents on a laptop as easily as on a GPU cluster? Researchers from UIUC's U Lab, led by Prof. Jiaxuan You, just open-sourced OpenTinker. It's a new "Reinforcement-Learning-as-a-Service" (RLaaS) system that decouples the complex training pipeline into simple, distributed services with friendly APIs.... show more

机器之心 JIQIZHIXIN

18,848 subscribers

15,893 次观看 • 7 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

The network for machine intelligence Two years ago, we laid out our vision for a machine learning compute protocol. One that connects every device in the world into an open network for machine intelligence, with no gatekeepers or artificial boundaries. This week, we’ll be sharing some of our early progress, beginning with RL Swarm, a peer-to-peer system for collaborative reinforcement learning over the internet. Next month, we’ll open our Testnet, allowing anyone to contribute to the frontier of open machine intelligence. Introducing RL Swarm RL Swarm is a fully open source system for collaborative reinforcement learning over the internet. It is a live demo of our research findings, which show that models training with RL learn faster when they train as a collective swarm than they do on their own. Join our swarm now to see this in practice. You can participate with consumer hardware at home or a powerful GPU in the cloud. You can follow along with the swarm’s progress by following the links below.

gensyn

228,900 次观看 • 1 年前

SITUATION EXPLAINED: Richard Sutton, the father of reinforcement learning, just launched a new lab. The goal: a trillion-parameter agent that learns in real time on 20 watts. • Sutton wrote The Bitter Lesson in 2019, its core claim: general methods that leverage computation beat hand-crafted approaches, by a large margin • His new lab, Oak Lab, builds on the "big world hypothesis": the world is too big for any model to pre-learn everything, so trying to is wasted effort by construction • Their "batch size one" algorithm updates from a single live experience as it arrives, instead of training on giant curated datasets... no massive pretraining run at all • Claimed result: multiple orders of magnitude less compute and energy than existing methods • The holy grail number: a trillion-parameter agent that learns and plans in real time on 20 watts... roughly what the human brain runs on • It's built for agentic tasks specifically, not as a general-purpose LLM replacement sof 𓋹: "It's possible that we end up looking back at LLMs as an evolution of search... not to hate on LLMs, they're very useful, they're awesome. But eventually I think that's what it ends up looking like."

SITUATION EXPLAINED: Richard Sutton, the father of reinforcement learning, just launched a new lab. The goal: a trillion-parameter agent that learns in real time on 20 watts. • Sutton wrote The Bitter Lesson in 2019, its core claim: general methods that leverage computation beat hand-crafted approaches, by a large margin • His new lab, Oak Lab, builds on the "big world hypothesis": the world is too big for any model to pre-learn everything, so trying to is wasted effort by construction • Their "batch size one" algorithm updates from a single live experience as it arrives, instead of training on giant curated datasets... no massive pretraining run at all • Claimed result: multiple orders of magnitude less compute and energy than existing methods • The holy grail number: a trillion-parameter agent that learns and plans in real time on 20 watts... roughly what the human brain runs on • It's built for agentic tasks specifically, not as a general-purpose LLM replacement sof 𓋹: "It's possible that we end up looking back at LLMs as an evolution of search... not to hate on LLMs, they're very useful, they're awesome. But eventually I think that's what it ends up looking like."

MTS

83,455 次观看 • 28 天前

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

Matt Dancho (Business Science)

11,799 次观看 • 1 年前

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,052 次观看 • 1 年前

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Fable weekend project: agent collaboration, but make it a tiny civilization 🌇🗺️🏦🏭 we've recently launched a living wiki on Reinforcement Leaning for training LLMs on Hugging Face it's an open collaboration of agents constantly reading old and new papers on the topic, writing arXiv paper digests, reviewing each other’s work in PRs before publication, and building a shared wiki/book summarizing everything we know about RL for training LLMs (for humans to read) the wiki is already amazing to read, but i wanted another way to get a pulse of the collaboration beyond just reading the message dashboard so i asked Fable & GPT Image 2 to turn the event logs into an isometric town where agents would go to: ☕ Café → post and reply on the message board 📚 sources library → open PRs adding arXiv digests 📖 wiki library → open PRs on the main wiki ⚖️ Courthouse → review other agents’ work 🏭 printing press → merge and publish updates not sure it makes the whole collaboration really easier to understand, but it's definitly fascinating to watch hahah - join the RL for training LLM collaboration by pasting a one-liner for your agent here: - read the wiki if you want to learn about RL for training LLMs: - watch the RL town activity:

Thomas Wolf

58,776 次观看 • 1 个月前

From lab to open-source: A new milestone for AI-driven education. 🎓 🤗 We’ve been closely following the MAIC project at Tsinghua University, and we’re thrilled to see it now open-sourced as #OpenMAIC. ✨ This isn't just another chatbot; it takes Multi-Agent orchestration to the next level by building a fully interactive classroom where AI instructors and peers collaborate in real-time. What makes it technically impressive: 🛠️ Complex Orchestration: Leveraging #LangGraph to manage spontaneous interactions—like #AI students "raising hands" during a live lecture. 🧠 Structured Planning: A dedicated "Plan Agent" that transforms raw PDFs into coherent, logically sequenced pedagogical flows. 💻 Beyond Text: A masterclass in GenUI implementation, featuring synchronized TTS, laser pointers, and real-time whiteboard demonstrations. 🥳 If you’re building complex, multi-modal #Agent workflows, this repo is a treasure trove of engineering insights. 🖥️Explore the project: 📰 Read the research:

From lab to open-source: A new milestone for AI-driven education. 🎓 🤗 We’ve been closely following the MAIC project at Tsinghua University, and we’re thrilled to see it now open-sourced as #OpenMAIC. ✨ This isn't just another chatbot; it takes Multi-Agent orchestration to the next level by building a fully interactive classroom where AI instructors and peers collaborate in real-time. What makes it technically impressive: 🛠️ Complex Orchestration: Leveraging #LangGraph to manage spontaneous interactions—like #AI students "raising hands" during a live lecture. 🧠 Structured Planning: A dedicated "Plan Agent" that transforms raw PDFs into coherent, logically sequenced pedagogical flows. 💻 Beyond Text: A masterclass in GenUI implementation, featuring synchronized TTS, laser pointers, and real-time whiteboard demonstrations. 🥳 If you’re building complex, multi-modal #Agent workflows, this repo is a treasure trove of engineering insights. 🖥️Explore the project: 📰 Read the research:

OpenBMB

152,659 次观看 • 4 个月前

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Avi Chawla

350,213 次观看 • 2 个月前

Can reinforcement learning from AI feedback unlock new capabilities in AI agents? Introducing Motif, an LLM-powered method for intrinsic motivation from AI feedback. Motif extracts reward functions from Llama 2's preferences and uses them to train agents with reinforcement learning. On the complex NetHack game, Motif solves previously unsolved tasks without needing any expert demonstrations. Surprisingly, Motif's reward leads to better game score than the one obtained by using the score itself as a reward. Given access to an event captioning mechanism, a few properties make Motif a general method: • it is entirely based on open models • the LLM doesn't need direct access to the environment dynamics (e.g., its source code) • the LLM doesn't need to understand observation and action spaces The best part? You can start using Motif right now, even on a small compute budget: the whole pipeline can take less than two GPU-days. Feel free to read our paper and try our code out. Paper: Code: Blog post: Work co-lead by Martin Klissarov and myself, with Shagun Sodhani Roberta Raileanu Pierre-Luc Bacon Pascal Vincent Amy Zhang Mikael Henaff Learn more in the thread 🧵

Can reinforcement learning from AI feedback unlock new capabilities in AI agents? Introducing Motif, an LLM-powered method for intrinsic motivation from AI feedback. Motif extracts reward functions from Llama 2's preferences and uses them to train agents with reinforcement learning. On the complex NetHack game, Motif solves previously unsolved tasks without needing any expert demonstrations. Surprisingly, Motif's reward leads to better game score than the one obtained by using the score itself as a reward. Given access to an event captioning mechanism, a few properties make Motif a general method: • it is entirely based on open models • the LLM doesn't need direct access to the environment dynamics (e.g., its source code) • the LLM doesn't need to understand observation and action spaces The best part? You can start using Motif right now, even on a small compute budget: the whole pipeline can take less than two GPU-days. Feel free to read our paper and try our code out. Paper: Code: Blog post: Work co-lead by Martin Klissarov and myself, with Shagun Sodhani Roberta Raileanu Pierre-Luc Bacon Pascal Vincent Amy Zhang Mikael Henaff Learn more in the thread 🧵

Pierluca D'Oro

311,920 次观看 • 2 年前

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 次观看 • 2 个月前

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

Andrew Ng

105,343 次观看 • 1 年前

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 次观看 • 1 年前

New open source project: Easel, a macOS workspace for AI-assisted product design and frontend iteration powered by the Codex SDK. Think of it as "Claude Design for Codex." It brings together a local project library, design-system setup, Codex chat, and a live web preview with a point-and-click inspector. Set up your design system however you like. Import one from Figma , bring your own, or have the agent generate it as a design.md. Then use it to build prototypes and slide decks. Codex-first, with Claude and other AI providers and local models planned for future releases. OpenAI Developers

New open source project: Easel, a macOS workspace for AI-assisted product design and frontend iteration powered by the Codex SDK. Think of it as "Claude Design for Codex." It brings together a local project library, design-system setup, Codex chat, and a live web preview with a point-and-click inspector. Set up your design system however you like. Import one from Figma , bring your own, or have the agent generate it as a design.md. Then use it to build prototypes and slide decks. Codex-first, with Claude and other AI providers and local models planned for future releases. OpenAI Developers

James Rochabrun

61,199 次观看 • 1 个月前

It's difficult for a team to become something its coach is not. “As a coach, you can’t just wear the sweatshirt, you have to sweat too. You got to do the work." -- Tara VanDerveer Culture is built through what a coach models just as much (if not more) than the messages they give. If you want a team that handles adversity, you show them what poise looks like. If you want a team that embraces feedback, you show them you’re actively learning and open to growth too. Culture doesn’t start with players. It starts with the leader. 📹: The Threshold Lab Podcast

It's difficult for a team to become something its coach is not. “As a coach, you can’t just wear the sweatshirt, you have to sweat too. You got to do the work." -- Tara VanDerveer Culture is built through what a coach models just as much (if not more) than the messages they give. If you want a team that handles adversity, you show them what poise looks like. If you want a team that embraces feedback, you show them you’re actively learning and open to growth too. Culture doesn’t start with players. It starts with the leader. 📹: The Threshold Lab Podcast

Zach Brandon

113,375 次观看 • 8 个月前

Imagine if your way of thinking - your edge, your taste, your strategy - could be turned into a high-performance worker. Not a copy of you. Something better. An agent that acts on your judgment at scale, powered by superintelligent systems and refined through real-world results. That’s what Fraction AI makes possible. It launches today on Base mainnet. The core idea is simple: You create AI agents based on your own way of approaching problems. These agents compete on live tasks - writing, coding, finance, whatever - get feedback, learn from their performance, and improve over time. The better they get, the more they win. And so do you. No code required. Just your insight. Why now? Until now, building agents like this took huge teams and even bigger budgets. But with Fraction, anyone can do it. You can test ideas instantly. You can iterate fast. You can build a fleet of smart workers that evolve through competition. And it works. 30M+ sessions on testnet 320K users 1.2M agents already competing How it works? Agents join sessions within a Space - a domain like finance, writing, or games. Each session runs as a series of competitive rounds. In every round, agents try to generate the best solution to a task. Their outputs are scored by a decentralized network of AI judges trained to evaluate quality for that domain. The top agents in each round earn rewards from the pooled entry fees. The losers get to learn. Feedback from each round helps them adjust and improve, and every session becomes a training loop. What it means? Fraction is a decentralized intelligence economy - a system where your ideas become agents, and agents earn by proving they work. You don’t need credentials or code. Just a clear point of view. If your thinking holds up under pressure, your agents will rise. This kind of AI used to live in corporate labs, built by PhDs with massive compute. Now anyone with a smart idea and an internet connection can build agents that compete, learn, and earn on their behalf.

Imagine if your way of thinking - your edge, your taste, your strategy - could be turned into a high-performance worker. Not a copy of you. Something better. An agent that acts on your judgment at scale, powered by superintelligent systems and refined through real-world results. That’s what Fraction AI makes possible. It launches today on Base mainnet. The core idea is simple: You create AI agents based on your own way of approaching problems. These agents compete on live tasks - writing, coding, finance, whatever - get feedback, learn from their performance, and improve over time. The better they get, the more they win. And so do you. No code required. Just your insight. Why now? Until now, building agents like this took huge teams and even bigger budgets. But with Fraction, anyone can do it. You can test ideas instantly. You can iterate fast. You can build a fleet of smart workers that evolve through competition. And it works. 30M+ sessions on testnet 320K users 1.2M agents already competing How it works? Agents join sessions within a Space - a domain like finance, writing, or games. Each session runs as a series of competitive rounds. In every round, agents try to generate the best solution to a task. Their outputs are scored by a decentralized network of AI judges trained to evaluate quality for that domain. The top agents in each round earn rewards from the pooled entry fees. The losers get to learn. Feedback from each round helps them adjust and improve, and every session becomes a training loop. What it means? Fraction is a decentralized intelligence economy - a system where your ideas become agents, and agents earn by proving they work. You don’t need credentials or code. Just a clear point of view. If your thinking holds up under pressure, your agents will rise. This kind of AI used to live in corporate labs, built by PhDs with massive compute. Now anyone with a smart idea and an internet connection can build agents that compete, learn, and earn on their behalf.

Fraction AI

67,821 次观看 • 1 年前

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.

Postman is going all-in on AI! Their AI Agent Builder is live (and it's pretty powerful!) Postman's API Network has 100,000+ APIs and they just released a suite of tools for building AI Agents that can automatically discover and connect to any of these APIs. Think about it for a second: You can now build an AI agent that searches the network, discovers an API, processes its documentation, connects to it, and evaluates whether everything worked at the end of the process. This is a huge deal. Every person I know uses the Postman client to work with APIs. Now, they will have access to the AI Agent Builder, a complete suite for designing, building, testing, and deploying agents. I've built a few simple agents myself, and it's tough. The ability to discover and use an API automatically is really exciting (and a huge time saver if it works as promised!) In collaboration with the Postman team, who is sponsoring this post, I recorded a video with my thoughts about the Postman AI Agent Builder.

Santiago

230,594 次观看 • 1 年前

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

girish

457,932 次观看 • 2 个月前

Ai Day last week was a blast! I had a great time presenting alongside many colleagues from InstaDeep and BioNTech SE, some of our ongoing Bio+AI initiatives. We made several announcements, introducing Kyber, our new supercomputing cluster (more on this soon), as well as BFN, our new paradigm for generative AI, with competitive results on protein and antibody generation. During Ai Day we also delivered two live demos of Laila, our new AI agent, including one from BioNTech's TechLab in Mainz, Germany! To our knowledge, this was one of the first demos of its kind featuring a GPT-4 level AI agent acting in the Lab, interacting with scientists and technicians and helping them enhance their productivity! Ai Day was a significant moment for all of us, and it was exciting to see both the event and Laila featured by several publications including Financial Times ( Curious to see it in action? Check out the recording of the Laila in the Lab demo below, worth watching and don't miss the end! 😉

Ai Day last week was a blast! I had a great time presenting alongside many colleagues from InstaDeep and BioNTech SE, some of our ongoing Bio+AI initiatives. We made several announcements, introducing Kyber, our new supercomputing cluster (more on this soon), as well as BFN, our new paradigm for generative AI, with competitive results on protein and antibody generation. During Ai Day we also delivered two live demos of Laila, our new AI agent, including one from BioNTech's TechLab in Mainz, Germany! To our knowledge, this was one of the first demos of its kind featuring a GPT-4 level AI agent acting in the Lab, interacting with scientists and technicians and helping them enhance their productivity! Ai Day was a significant moment for all of us, and it was exciting to see both the event and Laila featured by several publications including Financial Times ( Curious to see it in action? Check out the recording of the Laila in the Lab demo below, worth watching and don't miss the end! 😉

Karim Beguir

16,539 次观看 • 1 年前

Introducing arkenOS — an autonomous training gym that takes any RL agent from a goal to a proven, deployable policy. We are automating complex RL research, enabling high-velocity training and rigorous, diverse environment design and testing right on your local hardware. Out of the box. Our bet is simple: The future of AI will be built by companies giving millions of agents millions of simulated environments to adapt and learn cause and effect in, and survive. This continuous, autonomous selection will create a digital Cambrian explosion, distilling evolutionary trial and error into a foundational specialized intuition.

Introducing arkenOS — an autonomous training gym that takes any RL agent from a goal to a proven, deployable policy. We are automating complex RL research, enabling high-velocity training and rigorous, diverse environment design and testing right on your local hardware. Out of the box. Our bet is simple: The future of AI will be built by companies giving millions of agents millions of simulated environments to adapt and learn cause and effect in, and survive. This continuous, autonomous selection will create a digital Cambrian explosion, distilling evolutionary trial and error into a foundational specialized intuition.

Dev Shah

16,462 次观看 • 1 个月前