Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

From Failures to Learnable Worlds Policy-Conditioned Environment Generation for Navigation Agents paper:

_akhaliq's profile picture

AK

534,456 subscribers

17,068 Aufrufe • vor 20 Tagen •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von lewington
lewingtonvor 20 Tagen

i think it’s just a personalised curriculum for embodied agents

Profilbild von 陈放Frank🕊️
陈放Frank🕊️vor 20 Tagen

「导航 agent 失败 → 反过来生环境再训练」这招特别狠——把失败率当数据用而不是砸更多算力，之前 Voyager 那条线就在做这件事，但生成环境的可控性一直卡。 policy-conditioned 比 unconditional 难不少，主要卡在 conditional signal 怎么拿到（动作 label + 失败 case 的对齐）。如果这个方向真跑通，对 Embodied Turing Machines 那类 self-improve 也是直接补一块——agent 可以自己做 curriculum。

Profilbild von LongTian
LongTianvor 20 Tagen

failing forward is the real curriculum here, and conditioning on the policy makes the generated worlds actually bite back

Profilbild von Daniel Priscu
Daniel Priscuvor 20 Tagen

building the next environment from where the policy fails beats burning compute on levels it already solved

Profilbild von Broadstreet
Broadstreetvor 20 Tagen

⏫

Profilbild von Elio科技迷(互)
Elio科技迷(互)vor 20 Tagen

turning failed runs into new training worlds is a neat trick

Ähnliche Videos

🚨Excited to share: “𝗣𝗮𝗻𝗼𝗚𝗲𝗻: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigation”!🚨 PanoGen creates diverse 360-degree panorama via recursive outpainting, achieving SotA on R2R, R4R, CVDN. Mohit Bansal @uncnlp 🧵
0:24

🚨Excited to share: “𝗣𝗮𝗻𝗼𝗚𝗲𝗻: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigation”!🚨 PanoGen creates diverse 360-degree panorama via recursive outpainting, achieving SotA on R2R, R4R, CVDN. Mohit Bansal @uncnlp 🧵

Jialu Li

31,991 Aufrufe • vor 3 Jahren

Environment generation is the missing scaling axis for embodied AI. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D env where agents act, fail & learn. Env-agent co-evolvution improves navigation success 50% → 90%. From a prompt, our SimCoder writes code to automatically build an interactive world. Agents train inside it. And their performance shapes the next world.
1:36

Environment generation is the missing scaling axis for embodied AI. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D env where agents act, fail & learn. Env-agent co-evolvution improves navigation success 50% → 90%. From a prompt, our SimCoder writes code to automatically build an interactive world. Agents train inside it. And their performance shapes the next world.

SimWorld

2,272,272 Aufrufe • vor 5 Monaten

Introducing an approach to directly ground video generation models to policy execution without needing any action labels! Our approach uses a generic goal-conditioned exploration procedure to learn a policy that works across robots / embodiments!
1:09

Introducing an approach to directly ground video generation models to policy execution without needing any action labels! Our approach uses a generic goal-conditioned exploration procedure to learn a policy that works across robots / embodiments!

Yilun Du

21,427 Aufrufe • vor 2 Jahren

📣 New paper: Emergence of Maps in the Memories of Blind Navigation Agents Humans have the ability to navigate poorly lit spaces by relying on touch and memory. Our research shows that blind AI agents can learn to do the same. Read the paper ➡️
0:39

📣 New paper: Emergence of Maps in the Memories of Blind Navigation Agents Humans have the ability to navigate poorly lit spaces by relying on touch and memory. Our research shows that blind AI agents can learn to do the same. Read the paper ➡️

AI at Meta

89,757 Aufrufe • vor 3 Jahren

Introducing ShapeR, a method for robust conditional 3D shape generation from casually captured sequences. ShapeR leverages a rectified flow transformer conditioned on per-object multimodal data to turn casual image sequences into full metric scene reconstructions. Project Page: Paper: Links to code and huggingface below ⬇️
0:58

Introducing ShapeR, a method for robust conditional 3D shape generation from casually captured sequences. ShapeR leverages a rectified flow transformer conditioned on per-object multimodal data to turn casual image sequences into full metric scene reconstructions. Project Page: Paper: Links to code and huggingface below ⬇️

Yawar Siddiqui

71,100 Aufrufe • vor 9 Monaten

Gen2Act: Casting language-conditioned manipulation as *human video generation* followed by *closed-loop policy execution conditioned on the generated video* enables solving diverse real-world tasks unseen in the robot dataset! 1/n
0:28

Gen2Act: Casting language-conditioned manipulation as *human video generation* followed by *closed-loop policy execution conditioned on the generated video* enables solving diverse real-world tasks unseen in the robot dataset! 1/n

Homanga Bharadhwaj

71,305 Aufrufe • vor 2 Jahren

New paper on kernel generation: "AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation" Agents often waste a lot of time making the same mistakes and trying to fix unpromising solutions. AdaExplore accumulates reusable skills based on failures (Adapt) and searches with a mixture of large changes and small refinements (Explore). Outperforms popular methods like OpenEvolve!
0:22

New paper on kernel generation: "AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation" Agents often waste a lot of time making the same mistakes and trying to fix unpromising solutions. AdaExplore accumulates reusable skills based on failures (Adapt) and searches with a mixture of large changes and small refinements (Explore). Outperforms popular methods like OpenEvolve!

Sean Welleck

29,727 Aufrufe • vor 6 Monaten

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!
1:22

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!

Y Combinator

44,192 Aufrufe • vor 8 Monaten

LIFT Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation paper:
0:25

LIFT Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation paper:

AK

17,977 Aufrufe • vor 27 Tagen

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻 Ubuntu. - 🕸️ Graph evaluator - provides a fine-grain evaluation metric for agents. - 🤖 Task generation - composes subtasks to automatically generate tasks. By connecting all devices to agents, 🦀CRAB unlocks greater capabilities for human-like tasks than ever before. Use 🦀 CRAB to benchmark your multimodal agents! - 👨‍💻 Check out the repository: - 📝 Read the paper: - 🌐 Find out more via the project page: - 🐫 Join our community:
1:05

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻 Ubuntu. - 🕸️ Graph evaluator - provides a fine-grain evaluation metric for agents. - 🤖 Task generation - composes subtasks to automatically generate tasks. By connecting all devices to agents, 🦀CRAB unlocks greater capabilities for human-like tasks than ever before. Use 🦀 CRAB to benchmark your multimodal agents! - 👨‍💻 Check out the repository: - 📝 Read the paper: - 🌐 Find out more via the project page: - 🐫 Join our community:

CAMEL-AI.org

72,807 Aufrufe • vor 2 Jahren

How many failures will it take for the American government to finally change foreign policy?
2:14

How many failures will it take for the American government to finally change foreign policy?

Ron Paul

14,646 Aufrufe • vor 4 Monaten

Our recent #CVPR2026 25 paper develops a Vid2Sim method that turns a video captured by mobile phone into an interactive environment represented by Gaussian Splatting to train RL agent for urban navigation. Incredible Ziyang Xie leaded the project. Webpage:
0:51

Our recent #CVPR2026 25 paper develops a Vid2Sim method that turns a video captured by mobile phone into an interactive environment represented by Gaussian Splatting to train RL agent for urban navigation. Incredible Ziyang Xie leaded the project. Webpage:

Bolei Zhou

31,701 Aufrufe • vor 1 Jahr

Introducing the Capy Paper integration The easiest way to design with cloud coding agents Paper runs in Capy's VMs, so your agents and teammates can design together from any device
5:21

Introducing the Capy Paper integration The easiest way to design with cloud coding agents Paper runs in Capy's VMs, so your agents and teammates can design together from any device

Capy

33,932 Aufrufe • vor 26 Tagen

Every RL environment we build runs on AgentEnv. Today we’re open-sourcing it, so more people can build the worlds agents learn in.
1:00

Every RL environment we build runs on AgentEnv. Today we’re open-sourcing it, so more people can build the worlds agents learn in.

Scale

105,882 Aufrufe • vor 29 Tagen

PAMI Part Anchored Motion for Text to Human-Object Interaction Generation paper:
0:34

PAMI Part Anchored Motion for Text to Human-Object Interaction Generation paper:

AK

20,327 Aufrufe • vor 27 Tagen

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.
0:30

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.

SciTech Era

47,751 Aufrufe • vor 5 Monaten

Microsoft presents World-R1 Reinforcing 3D Constraints for Text-to-Video Generation paper:
1:11

Microsoft presents World-R1 Reinforcing 3D Constraints for Text-to-Video Generation paper:

AK

24,625 Aufrufe • vor 6 Monaten

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.
6:34

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.

Siyuan Huang

11,451 Aufrufe • vor 2 Jahren