Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Happy to share our new work on Navigation World Models! 🔥🔥 Navigation is a fundamental skill of agents with visual-motor capabilities. We train a single World Model across multiple environments and diverse agent data. w/ Gaoyue Zhou, Danny Tran, trevordarrell and Yann LeCun.

Amir Bar

3,126 subscribers

83,539 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie Gesundheit & Wellness Bildung

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von Amir Bar

Amir Barvor 1 Jahr

Our Navigation World Model can simulate trajectories by generating video. This capability unlocks planning: simply find the sequence of actions that leads from the input image to a target goal. In unknown environments, our model can hallucinate navigation trajectories.

Profilbild von Amir Bar

Amir Barvor 1 Jahr

For more information, see our paper and project page: Project Page: Preprint: work is a collaboration between @AIatMeta and @berkeley_ai 😊

Profilbild von Amir Bar

Amir Barvor 1 Jahr

@berkeley_ai A lot of our work is also built on the work of others. Our Conditional Diffusion Transformer model (CDiT) extends DiT by @billpeeb and @sainingxie, and much of the data and inspiration is based on the works of our @berkeley_ai colleagues Noriaki Hirose and @shahdhruv_

Profilbild von Amir Bar

Amir Barvor 1 Jahr

@berkeley_ai @billpeeb @sainingxie @shahdhruv_ Finally, it is exciting to see the space of world models blowing up in the past week with other cool works from @theworldlabs and the genie project by @_rockt @GoogleDeepMind. Looking forward to what comes next!

Profilbild von rami (🇸🇬 till 15th)

rami (🇸🇬 till 15th)vor 1 Jahr

@GaoyueZhou @trevordarrell @ylecun sick

Profilbild von Fronesis

Fronesisvor 1 Jahr

@GaoyueZhou @trevordarrell @ylecun Thank you sir 🙏🫡

Profilbild von Michael Cho - Rbt/Acc

Michael Cho - Rbt/Accvor 1 Jahr

@GaoyueZhou @trevordarrell @ylecun Super cool! Btw have u come across this dataset? DM me if u r keen on our next big release 🙏

Profilbild von Arxiv Papers

Arxiv Papersvor 1 Jahr

@GaoyueZhou @trevordarrell @ylecun

Ähnliche Videos

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 Aufrufe • vor 2 Jahren

🎮 We release VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (w/ Junyi Zhang Jiaxin Ge) 🌐 With 17 environments across multiple domains, we show systematically the brittleness of VLMs in visual interaction, and what training leads to. 🧵[1/8]

🎮 We release VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (w/ Junyi Zhang Jiaxin Ge) 🌐 With 17 environments across multiple domains, we show systematically the brittleness of VLMs in visual interaction, and what training leads to. 🧵[1/8]

Zirui "Colin" Wang

40,334 Aufrufe • vor 5 Monaten

Yann LeCun (Chief AI Scientist, Meta, Yann LeCun), Pim de Witte (CEO, General Intuition), and Aude Durand (Kyutai, Aude Durand), talk about world models, embodied agents, Yann's new company, and the limitations of LLMs 0:00 - Introduction to World Models 5:00 - Why World Models, Intuition & Introducing Yann's new company 10:00 - Architectures + Merging Language & Interaction Data towards General Agents 20:00 - Open Source, Sovereign AI & kyutai Partnership Keynote for #aiPULSE2025 at Station F in Paris 🇫🇷

Yann LeCun (Chief AI Scientist, Meta, Yann LeCun), Pim de Witte (CEO, General Intuition), and Aude Durand (Kyutai, Aude Durand), talk about world models, embodied agents, Yann's new company, and the limitations of LLMs 0:00 - Introduction to World Models 5:00 - Why World Models, Intuition & Introducing Yann's new company 10:00 - Architectures + Merging Language & Interaction Data towards General Agents 20:00 - Open Source, Sovereign AI & kyutai Partnership Keynote for #aiPULSE2025 at Station F in Paris 🇫🇷

General Intuition

111,179 Aufrufe • vor 7 Monaten

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"

Haider.

201,086 Aufrufe • vor 1 Monat

🥳Excited to share: Hierarchical World Models as Visual Whole-Body Humanoid Controllers Joint work with Jyothir S V Vlad Sobal Yann LeCun Xiaolong Wang Hao Su Our method, Puppeteer, learns high-dim humanoid policies that look natural, in an entirely data-driven way! 🧵👇(1/n)

🥳Excited to share: Hierarchical World Models as Visual Whole-Body Humanoid Controllers Joint work with Jyothir S V Vlad Sobal Yann LeCun Xiaolong Wang Hao Su Our method, Puppeteer, learns high-dim humanoid policies that look natural, in an entirely data-driven way! 🧵👇(1/n)

Nicklas Hansen

250,912 Aufrufe • vor 2 Jahren

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,524 Aufrufe • vor 11 Monaten

Introducing PROWL! We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions. Those failures then become training data in an automated loop that advances world model performance.

Introducing PROWL! We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions. Those failures then become training data in an automated loop that advances world model performance.

Odyssey

63,762 Aufrufe • vor 1 Monat

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict *when* hallucination happens and use that to fix it! 🧵1/n

Super excited to share the last paper of my PhD: "Hallucination in World Models is Predictable and Preventable"✨ We train a 350M-param generative world model on a large dataset w/ 210 tasks and show that we can predict when hallucination happens and use that to fix it! 🧵1/n

Nicklas Hansen

50,250 Aufrufe • vor 10 Tagen

Scaling embodied AI starts with automating the environments. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D environments where agents act, fail, and learn. With coding-agent + embodied-agent co-evolution, navigation success improves from 50% → 90%. 1/

Scaling embodied AI starts with automating the environments. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D environments where agents act, fail, and learn. With coding-agent + embodied-agent co-evolution, navigation success improves from 50% → 90%. 1/

Lianhui Qin@ACL2026

58,744 Aufrufe • vor 1 Monat

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

309,942 Aufrufe • vor 1 Jahr

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.

LobeHub

185,032 Aufrufe • vor 5 Monaten

🚨Thrilled to present VisualAgentBench (VAB) with Yu Gu and Tianjie, where we enable both TRAINING & TESTING of visual foundation agents across 5 different environments! In all 17 large multimodal models (LMMs) are tested. Find our paper, data, and more insights below 👇 Paper: Code & Data: Thanks AK for sharing on today’s arxiv on HF!

🚨Thrilled to present VisualAgentBench (VAB) with Yu Gu and Tianjie, where we enable both TRAINING & TESTING of visual foundation agents across 5 different environments! In all 17 large multimodal models (LMMs) are tested. Find our paper, data, and more insights below 👇 Paper: Code & Data: Thanks AK for sharing on today’s arxiv on HF!

Xiao Liu (Shaw)

22,594 Aufrufe • vor 1 Jahr

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

Amir Zamir

69,539 Aufrufe • vor 2 Jahren

In our new short course you’ll learn exactly how algorithms like DALL·E and Stable Diffusion work. Dive into the fascinating world of diffusion models and build your own model from scratch with Sharon Zhou ✈️ ICML in "How Diffusion Models Work." Sign up:

In our new short course you’ll learn exactly how algorithms like DALL·E and Stable Diffusion work. Dive into the fascinating world of diffusion models and build your own model from scratch with Sharon Zhou ✈️ ICML in "How Diffusion Models Work." Sign up:

DeepLearning.AI

56,075 Aufrufe • vor 3 Jahren

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

What happens when vision🤝 robotics meet? Happy to share our new work on Pretraining Robotic Foundational Models!🔥 ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model. Berkeley AI Research😊

Roei Herzig

62,711 Aufrufe • vor 1 Jahr

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Yann LeCun says you cannot build a reliable agentic system without a world model LLMs don't have world models. They can't predict the consequences of their actions before taking them "they just act, and whatever happens next is someone else's problem" Without that, it's not intelligence

Haider.

331,488 Aufrufe • vor 1 Monat

About a year ago, we launched the Agent-to-Agent protocol because AI agents shouldn’t work in silos. Today, developers are using A2A to coordinate specialized agents across enterprise data systems, commerce workflows, DevOps environments, and scientific research. FoldRun is one example, simplifying petabyte-scale protein modeling through agent-to-agent collaboration. Learn how to build with it in the blog:

About a year ago, we launched the Agent-to-Agent protocol because AI agents shouldn’t work in silos. Today, developers are using A2A to coordinate specialized agents across enterprise data systems, commerce workflows, DevOps environments, and scientific research. FoldRun is one example, simplifying petabyte-scale protein modeling through agent-to-agent collaboration. Learn how to build with it in the blog:

Google for Developers

35,162 Aufrufe • vor 13 Tagen

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

Proof of Coverage | Tech’s Creative Agency

11,852 Aufrufe • vor 5 Monaten

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

The Humanoid Hub

30,148 Aufrufe • vor 10 Monaten