正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Happy to share our new work on Navigation World Models! 🔥🔥 Navigation is a fundamental skill of agents with visual-motor capabilities. We train a single World Model across multiple environments and diverse agent data. w/ Gaoyue Zhou, Danny Tran, trevordarrell and Yann LeCun.

Amir Bar

3,126 subscribers

83,539 次观看 • 1 年前 •via X (Twitter)

科学技术健康养生教育

Anya Rossi• Live Now

Private livecam show

8 条评论

Amir Bar 的头像

Amir Bar1 年前

Our Navigation World Model can simulate trajectories by generating video. This capability unlocks planning: simply find the sequence of actions that leads from the input image to a target goal. In unknown environments, our model can hallucinate navigation trajectories.

Amir Bar 的头像

Amir Bar1 年前

For more information, see our paper and project page: Project Page: Preprint: work is a collaboration between @AIatMeta and @berkeley_ai 😊

Amir Bar 的头像

Amir Bar1 年前

@berkeley_ai A lot of our work is also built on the work of others. Our Conditional Diffusion Transformer model (CDiT) extends DiT by @billpeeb and @sainingxie, and much of the data and inspiration is based on the works of our @berkeley_ai colleagues Noriaki Hirose and @shahdhruv_

Amir Bar 的头像

Amir Bar1 年前

@berkeley_ai @billpeeb @sainingxie @shahdhruv_ Finally, it is exciting to see the space of world models blowing up in the past week with other cool works from @theworldlabs and the genie project by @_rockt @GoogleDeepMind. Looking forward to what comes next!

rami (🇸🇬 till 15th) 的头像

rami (🇸🇬 till 15th)1 年前

@GaoyueZhou @trevordarrell @ylecun sick

Fronesis 的头像

Fronesis1 年前

@GaoyueZhou @trevordarrell @ylecun Thank you sir 🙏🫡

Michael Cho - Rbt/Acc 的头像

Michael Cho - Rbt/Acc1 年前

@GaoyueZhou @trevordarrell @ylecun Super cool! Btw have u come across this dataset? DM me if u r keen on our next big release 🙏

Arxiv Papers 的头像

Arxiv Papers1 年前

@GaoyueZhou @trevordarrell @ylecun

相关视频

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 次观看 • 2 年前

🎮 We release VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (w/ Junyi Zhang Jiaxin Ge) 🌐 With 17 environments across multiple domains, we show systematically the brittleness of VLMs in visual interaction, and what training leads to. 🧵[1/8]

🎮 We release VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (w/ Junyi Zhang Jiaxin Ge) 🌐 With 17 environments across multiple domains, we show systematically the brittleness of VLMs in visual interaction, and what training leads to. 🧵[1/8]

Zirui "Colin" Wang

40,334 次观看 • 5 个月前

Yann LeCun (Chief AI Scientist, Meta, Yann LeCun), Pim de Witte (CEO, General Intuition), and Aude Durand (Kyutai, Aude Durand), talk about world models, embodied agents, Yann's new company, and the limitations of LLMs 0:00 - Introduction to World Models 5:00 - Why World Models, Intuition & Introducing Yann's new company 10:00 - Architectures + Merging Language & Interaction Data towards General Agents 20:00 - Open Source, Sovereign AI & kyutai Partnership Keynote for #aiPULSE2025 at Station F in Paris 🇫🇷

Yann LeCun (Chief AI Scientist, Meta, Yann LeCun), Pim de Witte (CEO, General Intuition), and Aude Durand (Kyutai, Aude Durand), talk about world models, embodied agents, Yann's new company, and the limitations of LLMs 0:00 - Introduction to World Models 5:00 - Why World Models, Intuition & Introducing Yann's new company 10:00 - Architectures + Merging Language & Interaction Data towards General Agents 20:00 - Open Source, Sovereign AI & kyutai Partnership Keynote for #aiPULSE2025 at Station F in Paris 🇫🇷

General Intuition

111,179 次观看 • 7 个月前

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"

Haider.

201,086 次观看 • 1 个月前

🥳Excited to share: Hierarchical World Models as Visual Whole-Body Humanoid Controllers Joint work with Jyothir S V Vlad Sobal Yann LeCun Xiaolong Wang Hao Su Our method, Puppeteer, learns high-dim humanoid policies that look natural, in an entirely data-driven way! 🧵👇(1/n)

🥳Excited to share: Hierarchical World Models as Visual Whole-Body Humanoid Controllers Joint work with Jyothir S V Vlad Sobal Yann LeCun Xiaolong Wang Hao Su Our method, Puppeteer, learns high-dim humanoid policies that look natural, in an entirely data-driven way! 🧵👇(1/n)

Nicklas Hansen

250,912 次观看 • 2 年前

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,524 次观看 • 11 个月前

Introducing PROWL! We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions. Those failures then become training data in an automated loop that advances world model performance.

Introducing PROWL! We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions. Those failures then become training data in an automated loop that advances world model performance.

Odyssey

63,762 次观看 • 1 个月前

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it! 🧵1/n

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict when hallucination happens and use that to fix it! 🧵1/n

Nicklas Hansen

50,250 次观看 • 10 天前

Scaling embodied AI starts with automating the environments. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D environments where agents act, fail, and learn. With coding-agent + embodied-agent co-evolution, navigation success improves from 50% → 90%. 1/

Scaling embodied AI starts with automating the environments. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D environments where agents act, fail, and learn. With coding-agent + embodied-agent co-evolution, navigation success improves from 50% → 90%. 1/

Lianhui Qin@ACL2026

58,744 次观看 • 1 个月前

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

309,942 次观看 • 1 年前

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.

LobeHub

185,032 次观看 • 5 个月前

🚨Thrilled to present VisualAgentBench (VAB) with Yu Gu and Tianjie, where we enable both TRAINING & TESTING of visual foundation agents across 5 different environments! In all 17 large multimodal models (LMMs) are tested. Find our paper, data, and more insights below 👇 Paper: Code & Data: Thanks AK for sharing on today’s arxiv on HF!

🚨Thrilled to present VisualAgentBench (VAB) with Yu Gu and Tianjie, where we enable both TRAINING & TESTING of visual foundation agents across 5 different environments! In all 17 large multimodal models (LMMs) are tested. Find our paper, data, and more insights below 👇 Paper: Code & Data: Thanks AK for sharing on today’s arxiv on HF!

Xiao Liu (Shaw)

22,594 次观看 • 1 年前

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

Amir Zamir

69,539 次观看 • 2 年前

In our new short course you’ll learn exactly how algorithms like DALL·E and Stable Diffusion work. Dive into the fascinating world of diffusion models and build your own model from scratch with Sharon Zhou ✈️ ICML in "How Diffusion Models Work." Sign up:

In our new short course you’ll learn exactly how algorithms like DALL·E and Stable Diffusion work. Dive into the fascinating world of diffusion models and build your own model from scratch with Sharon Zhou ✈️ ICML in "How Diffusion Models Work." Sign up:

DeepLearning.AI

56,075 次观看 • 3 年前

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

Roei Herzig

62,711 次观看 • 1 年前

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Haider.

331,488 次观看 • 1 个月前

About a year ago, we launched the Agent-to-Agent protocol because AI agents shouldn’t work in silos. Today, developers are using A2A to coordinate specialized agents across enterprise data systems, commerce workflows, DevOps environments, and scientific research. FoldRun is one example, simplifying petabyte-scale protein modeling through agent-to-agent collaboration. Learn how to build with it in the blog:

About a year ago, we launched the Agent-to-Agent protocol because AI agents shouldn’t work in silos. Today, developers are using A2A to coordinate specialized agents across enterprise data systems, commerce workflows, DevOps environments, and scientific research. FoldRun is one example, simplifying petabyte-scale protein modeling through agent-to-agent collaboration. Learn how to build with it in the blog:

Google for Developers

35,162 次观看 • 13 天前

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

Proof of Coverage | Tech’s Creative Agency

11,852 次观看 • 5 个月前

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

The Humanoid Hub

30,148 次观看 • 10 个月前