Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Reinforcement learning research often depends on static benchmarks and obscure leaderboards. But real evaluation happens inside environments, with specific tools, prompts, constraints, and workflows. Today, we’re launching the Turing RL Environments Evaluation Platform. Researchers now have direct, real time access to: -The exact production RL environments used in evaluation... show more

Turing

16,474 subscribers

69,730 просмотров • 4 месяцев назад •via X (Twitter)

Образование Новости и политика Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Thrilled to be featured in Dwarkesh’s latest episode with Richard Sutton, widely regarded as the father of reinforcement learning and 2024 Turing Award winner. As Richard explains, we’re entering the Era of Experience, where training AI means creating environments that capture the complexity, unpredictability, and subtle rules of real life. What we’ve learned: building these environments takes deep expertise in data, workflows, and nuances. At Labelbox, we’re proud to help teams tackle today’s RL challenges while also shaping the next era of AI, turning ambitious projects into working systems. (Learn more in the comments)

Thrilled to be featured in Dwarkesh’s latest episode with Richard Sutton, widely regarded as the father of reinforcement learning and 2024 Turing Award winner. As Richard explains, we’re entering the Era of Experience, where training AI means creating environments that capture the complexity, unpredictability, and subtle rules of real life. What we’ve learned: building these environments takes deep expertise in data, workflows, and nuances. At Labelbox, we’re proud to help teams tackle today’s RL challenges while also shaping the next era of AI, turning ambitious projects into working systems. (Learn more in the comments)

Labelbox

293,695 просмотров • 10 месяцев назад

Weekends are for research. Inside OptimAI labs, we’ve been pushing agents beyond the screen. A first glimpse of OptimAI Prime. Not just agents that think, but agents that operate. Learning through reinforcement, coordinating across the network, and executing in real-world environments. From digital workflows → to physical actions. This is the next step for the agent-native internet.🤖 Still early. But already in motion.

Weekends are for research. Inside OptimAI labs, we’ve been pushing agents beyond the screen. A first glimpse of OptimAI Prime. Not just agents that think, but agents that operate. Learning through reinforcement, coordinating across the network, and executing in real-world environments. From digital workflows → to physical actions. This is the next step for the agent-native internet.🤖 Still early. But already in motion.

OptimAI Network

20,719 просмотров • 3 месяцев назад

Today I’m excited to introduce micro1 Intelligence, the world’s most advanced platform for training frontier AI models. Achieving AGI is bottlenecked by one main thing: high-quality data. Data based on real-world environments that capture human expert workflows, complex decision-making, and reward signals models need to learn. With micro1 Intelligence, frontier labs can train on RL environments across every subject matter all in one place.

Today I’m excited to introduce micro1 Intelligence, the world’s most advanced platform for training frontier AI models. Achieving AGI is bottlenecked by one main thing: high-quality data. Data based on real-world environments that capture human expert workflows, complex decision-making, and reward signals models need to learn. With micro1 Intelligence, frontier labs can train on RL environments across every subject matter all in one place.

Ali Ansari

332,850 просмотров • 9 месяцев назад

We took a short break from robotics to build a human-level agent to play Competitive Pokémon. Partially observed. Stochastic. Long-horizon. Now mastered with Offline RL + Transformers. Our agent, trained on 475k+ human battles, hits the top 10% on Pokémon Showdown leaderboards. No search or heuristics, just sequence modeling. Today, we're open-sourcing our Metamon platform with our algorithms, data, and environments: 🌐 We are excited to see how our work accelerates research on building generally capable AI agents, and more importantly, inspires the next generation of Pokémon trainers!

We took a short break from robotics to build a human-level agent to play Competitive Pokémon. Partially observed. Stochastic. Long-horizon. Now mastered with Offline RL + Transformers. Our agent, trained on 475k+ human battles, hits the top 10% on Pokémon Showdown leaderboards. No search or heuristics, just sequence modeling. Today, we're open-sourcing our Metamon platform with our algorithms, data, and environments: 🌐 We are excited to see how our work accelerates research on building generally capable AI agents, and more importantly, inspires the next generation of Pokémon trainers!

Yuke Zhu

50,693 просмотров • 1 год назад

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

OpenBMB

13,996 просмотров • 7 месяцев назад

Loved this 22-minute talk on continual learning for AI agents. Must watch for anyone looking to get agents performant and into production. Credit: Soheil Feizi at AI Engineer • Agent learning can happen at three layers: the model (weights), the harness (prompts, tools, skills, code, workflows), and memory (session or persistent). • Two fundamental challenges: (1) getting feedback, meaning how do we know if the agent did well and what it should have done instead, and (2) acting on that feedback, meaning deciding which layer or component to change and how. • Feedback sources differ by stage: In development you have benchmarks with evaluators that score pass/fail. In production you only have logs, which can be judged either automatically (LLMs or code analyzing the log, which is scalable) or by human experts (low volume but critical domain knowledge). • Logs plus feedback aren't enough because they're not testable: A single log with feedback is one observation of what happened. You need to lift it into a replayable learning environment, a simulation with tools, users, and defined evaluators, so candidate fixes can be run, verified, and compared. • Three ways to optimize the agent, with tradeoffs: Model-layer updates (SFT, RL post-training like DPO/GRPO, LoRA) are expensive and need benchmarks and evaluators. Harness updates (trace-to-harness coding agents, prompt search like GEPA) are flexible but either untestable and "vibe-based" or benchmark-dependent. Memory updates (fact storage like Letta/Mem0, skill distillation) are cheapest and fastest but usually unverified. • A good learning engine makes "the smallest durable change at the right layer" of the agent. • Verifiable continual learning (VCL): Improve an agent from its own experience where every fix is proven to help and proven to break nothing that already worked. It requires an executable test (replayable failure), a measured delta (score before and after), and regression tests (prior tests still pass). • Four principles of practical VCL: Replayability (turn one-off failures into rerunnable tests), holisticness (one failure can have causes in memory, prompts, tools, workflow, or model, so route the fix to the right layer), lifelongness (fix new failures subject to no regression on past environments, with regression handled inside the optimization loop rather than post-hoc), and efficiency (the loop must run frequently and cheaply, without scaling linearly as past environments accumulate). • Three takeaways: (1) Agent continual learning isn't necessarily fine-tuning; many useful updates live in the harness and memory layers. (2) Production logs are not learning environments and must be transformed into replayable ones. (3) The frontier is regression-aware improvement: fixing new failures while verifying you don't break old ones.

Loved this 22-minute talk on continual learning for AI agents. Must watch for anyone looking to get agents performant and into production. Credit: Soheil Feizi at AI Engineer • Agent learning can happen at three layers: the model (weights), the harness (prompts, tools, skills, code, workflows), and memory (session or persistent). • Two fundamental challenges: (1) getting feedback, meaning how do we know if the agent did well and what it should have done instead, and (2) acting on that feedback, meaning deciding which layer or component to change and how. • Feedback sources differ by stage: In development you have benchmarks with evaluators that score pass/fail. In production you only have logs, which can be judged either automatically (LLMs or code analyzing the log, which is scalable) or by human experts (low volume but critical domain knowledge). • Logs plus feedback aren't enough because they're not testable: A single log with feedback is one observation of what happened. You need to lift it into a replayable learning environment, a simulation with tools, users, and defined evaluators, so candidate fixes can be run, verified, and compared. • Three ways to optimize the agent, with tradeoffs: Model-layer updates (SFT, RL post-training like DPO/GRPO, LoRA) are expensive and need benchmarks and evaluators. Harness updates (trace-to-harness coding agents, prompt search like GEPA) are flexible but either untestable and "vibe-based" or benchmark-dependent. Memory updates (fact storage like Letta/Mem0, skill distillation) are cheapest and fastest but usually unverified. • A good learning engine makes "the smallest durable change at the right layer" of the agent. • Verifiable continual learning (VCL): Improve an agent from its own experience where every fix is proven to help and proven to break nothing that already worked. It requires an executable test (replayable failure), a measured delta (score before and after), and regression tests (prior tests still pass). • Four principles of practical VCL: Replayability (turn one-off failures into rerunnable tests), holisticness (one failure can have causes in memory, prompts, tools, workflow, or model, so route the fix to the right layer), lifelongness (fix new failures subject to no regression on past environments, with regression handled inside the optimization loop rather than post-hoc), and efficiency (the loop must run frequently and cheaply, without scaling linearly as past environments accumulate). • Three takeaways: (1) Agent continual learning isn't necessarily fine-tuning; many useful updates live in the harness and memory layers. (2) Production logs are not learning environments and must be transformed into replayable ones. (3) The frontier is regression-aware improvement: fixing new failures while verifying you don't break old ones.

Alex Lieberman

20,085 просмотров • 1 месяц назад

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

Pat Grady

65,104 просмотров • 9 дней назад

CONTEXT ENGINEERING > PROMPT ENGINEERING Everyone is obsessed with writing better prompts. The next generation of AI builders is focused on context engineering instead. • Prompt engineering shapes the question. Context engineering shapes everything the AI sees. • Great prompts can't save an agent missing critical context, memory, or tools. • AI failures often come from missing information, not weak models. • Context engineering combines prompts, memory, RAG, state management, and tool access into one system. What powers effective AI agents? → Memory: Remembers preferences, past interactions, and ongoing tasks. → State Management: Tracks progress across multi-step workflows. → RAG: Retrieves only the most relevant information when needed. → Tools: Connects AI to APIs, databases, code execution, and real-world actions. → Dynamic Prompts: Enriches instructions with live context at runtime. The key insight: Prompt Engineering = Better Questions Context Engineering = Better Systems The future of AI isn't building smarter prompts. It's building smarter environments for AI to think, remember, retrieve, and act.

CONTEXT ENGINEERING > PROMPT ENGINEERING Everyone is obsessed with writing better prompts. The next generation of AI builders is focused on context engineering instead. • Prompt engineering shapes the question. Context engineering shapes everything the AI sees. • Great prompts can't save an agent missing critical context, memory, or tools. • AI failures often come from missing information, not weak models. • Context engineering combines prompts, memory, RAG, state management, and tool access into one system. What powers effective AI agents? → Memory: Remembers preferences, past interactions, and ongoing tasks. → State Management: Tracks progress across multi-step workflows. → RAG: Retrieves only the most relevant information when needed. → Tools: Connects AI to APIs, databases, code execution, and real-world actions. → Dynamic Prompts: Enriches instructions with live context at runtime. The key insight: Prompt Engineering = Better Questions Context Engineering = Better Systems The future of AI isn't building smarter prompts. It's building smarter environments for AI to think, remember, retrieve, and act.

Dami-Defi

10,127 просмотров • 2 месяцев назад

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Last semester, I taught Reinforcement Learning class again at UCLA. Together with my amazing TAs Matthew and Caiyuan, we built a mini-project: MetaDrive Arena 🚗🤖 Students applied what they learned in class, trained RL agents, and competed on a live leaderboard. The results were incredible, with 94 agents, 2K submissions, and 130K matches. We saw tons of creativity, clever ideas, and real progress in learning. We’re now releasing it publicly to support RL education and experimentation. Try it out and train your own agent at 🔗

Bolei Zhou

25,253 просмотров • 4 месяцев назад

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners (formerly TPG Capital), with participation from Lightspeed and Notable Capital. 🚀 At PatronusAI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over now. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since. ⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders and researchers across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more. ✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners (formerly TPG Capital), with participation from Lightspeed and Notable Capital. 🚀 At PatronusAI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over now. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since. ⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders and researchers across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more. ✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

Anand Kannappan

40,406 просмотров • 1 месяц назад

Finance Agent Benchmark v2 is here. Finance is one of the most lucrative applications of AI where much of the busy work could be automated. That’s why we rebuilt our Finance Agent Benchmark to push frontier models even further. We designed V2 to better reflect what financial analysts actually do: refined taxonomy reflecting real workflows, an improved harness with more tools, and jury-based evaluation. The result: no model cracks 52%. Would you trust a financial analyst who’s only correct half the time?

Finance Agent Benchmark v2 is here. Finance is one of the most lucrative applications of AI where much of the busy work could be automated. That’s why we rebuilt our Finance Agent Benchmark to push frontier models even further. We designed V2 to better reflect what financial analysts actually do: refined taxonomy reflecting real workflows, an improved harness with more tools, and jury-based evaluation. The result: no model cracks 52%. Would you trust a financial analyst who’s only correct half the time?

Vals AI

11,248 просмотров • 3 месяцев назад

Mansa AI is an enterprise-grade AI + Web3 platform designed to move artificial intelligence from experimentation into real-world execution. Built for creators, developers, and businesses, it focuses on deploying AI that actually works across modern digital systems, not just in isolated demos. 🚀 Production-ready AI infrastructure Mansa AI enables teams to deploy AI systems designed for live environments, handling real workflows, real data, and real operational demands without constant manual oversight. 🧠 Autonomous AI agents At its core, Mansa AI allows users to build autonomous agents that automate decision-making, coordinate tasks, monitor live signals, and execute complex workflows across dynamic environments. ⚙️ Fully customizable logic Agents can be configured with custom behaviors, triggers, and responses. From content generation and analytics to operational automation and intelligent orchestration, logic adapts to specific business strategies. 🔗 Web3 and off-chain integration Mansa AI bridges blockchain ecosystems with traditional systems, enabling cross-chain coordination, smart contract interactions, and seamless integration with existing enterprise infrastructure. 📊 Real-world use cases The platform supports automation for operations, customer engagement, analytics, data pipelines, content workflows, and AI-driven optimization across products and teams. 📈 Built for scale Whether launching as a startup or deploying across enterprise systems, Mansa AI is designed to scale AI operations without adding complexity or fragmentation. Mansa AI transforms artificial intelligence into deployable infrastructure. By combining autonomy, customization, interoperability, and scalability, it enables teams to own, operate, and grow intelligent systems that deliver real value in production environments.

Mansa AI is an enterprise-grade AI + Web3 platform designed to move artificial intelligence from experimentation into real-world execution. Built for creators, developers, and businesses, it focuses on deploying AI that actually works across modern digital systems, not just in isolated demos. 🚀 Production-ready AI infrastructure Mansa AI enables teams to deploy AI systems designed for live environments, handling real workflows, real data, and real operational demands without constant manual oversight. 🧠 Autonomous AI agents At its core, Mansa AI allows users to build autonomous agents that automate decision-making, coordinate tasks, monitor live signals, and execute complex workflows across dynamic environments. ⚙️ Fully customizable logic Agents can be configured with custom behaviors, triggers, and responses. From content generation and analytics to operational automation and intelligent orchestration, logic adapts to specific business strategies. 🔗 Web3 and off-chain integration Mansa AI bridges blockchain ecosystems with traditional systems, enabling cross-chain coordination, smart contract interactions, and seamless integration with existing enterprise infrastructure. 📊 Real-world use cases The platform supports automation for operations, customer engagement, analytics, data pipelines, content workflows, and AI-driven optimization across products and teams. 📈 Built for scale Whether launching as a startup or deploying across enterprise systems, Mansa AI is designed to scale AI operations without adding complexity or fragmentation. Mansa AI transforms artificial intelligence into deployable infrastructure. By combining autonomy, customization, interoperability, and scalability, it enables teams to own, operate, and grow intelligent systems that deliver real value in production environments.

King

155,637 просмотров • 8 месяцев назад

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,592 просмотров • 7 месяцев назад

OptimAI Core Node v2 is live. And with it, the introduction of OptimAI Claw. From nodes as infrastructure → to nodes as execution environments. With Core Node v2, every device becomes a runtime for native personal agents. 👉 Get your node up and running. No complex setup. No fragmented tooling. Just a few clicks... and your agent is live. OptimAI Claw brings persistent, node-resident agents directly into your system: • Multi-source research across web & social • Real-time summarization and signal tracking • Autonomous workflow execution • Continuous learning through reinforcement All running locally. All under your control. Supported on Windows & macOS. This is beyond prompt-based AI. Beyond isolated tools. This is personal agent infrastructure. Where every node thinks, acts, and evolves. Claw is now part of the network. And this is only the beginning. Happy Building, Making Impact! 🔥

OptimAI Core Node v2 is live. And with it, the introduction of OptimAI Claw. From nodes as infrastructure → to nodes as execution environments. With Core Node v2, every device becomes a runtime for native personal agents. 👉 Get your node up and running. No complex setup. No fragmented tooling. Just a few clicks... and your agent is live. OptimAI Claw brings persistent, node-resident agents directly into your system: • Multi-source research across web & social • Real-time summarization and signal tracking • Autonomous workflow execution • Continuous learning through reinforcement All running locally. All under your control. Supported on Windows & macOS. This is beyond prompt-based AI. Beyond isolated tools. This is personal agent infrastructure. Where every node thinks, acts, and evolves. Claw is now part of the network. And this is only the beginning. Happy Building, Making Impact! 🔥

OptimAI Network

355,036 просмотров • 4 месяцев назад

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners, with participation from Lightspeed and Notable Capital. 🚀 At Patronus AI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since.⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more.✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners, with participation from Lightspeed and Notable Capital. 🚀 At Patronus AI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since.⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more.✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

PatronusAI

95,090 просмотров • 1 месяц назад

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Flexion

71,725 просмотров • 1 месяц назад

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

a16z

75,069 просмотров • 24 дней назад

OptimAI Search is now on Agentic(.)Market - and we’re bringing the stack to Base. Builders can plug into live, citation-backed context across web, social, and crypto data through MCP, API, and SDK - then pay seamlessly with x402. No brittle workflows. No manual stitching. Just agent-native access to real-time intelligence at scale. And with custom domain agents on the marketplace, specialized knowledge becomes reusable. Search is becoming infrastructure. Build with it: 💡 💡

OptimAI Search is now on Agentic(.)Market - and we’re bringing the stack to Base. Builders can plug into live, citation-backed context across web, social, and crypto data through MCP, API, and SDK - then pay seamlessly with x402. No brittle workflows. No manual stitching. Just agent-native access to real-time intelligence at scale. And with custom domain agents on the marketplace, specialized knowledge becomes reusable. Search is becoming infrastructure. Build with it: 💡 💡

OptimAI Network

63,663 просмотров • 2 месяцев назад

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

361,163 просмотров • 1 месяц назад