Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Google DeepMind researchers say world models allow robotic agents to learn and plan in simulated environments, cutting real-world risks and costs. Robots can also potentially experience and learn rare scenarios in a world model, like Halloween crowds.

TheHumanoidHub's profile picture

The Humanoid Hub

67,252 subscribers

30,148 görüntüleme • 11 ay önce •via X (Twitter)

Sağlık & İyilikBilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️
0:33

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,615 görüntüleme • 1 yıl önce

Our Robotic Foundation Model, trained on data from real-world warehouse operations and simulated scenarios, enables our robots to identify objects, understand 3D spaces, and predict optimal grasping and placement.
0:28

Our Robotic Foundation Model, trained on data from real-world warehouse operations and simulated scenarios, enables our robots to identify objects, understand 3D spaces, and predict optimal grasping and placement.

Covariant

33,706 görüntüleme • 1 yıl önce

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.
0:30

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.

SciTech Era

47,510 görüntüleme • 1 ay önce

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️
2:03

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

310,204 görüntüleme • 1 yıl önce

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"
1:29

Yann LeCun says that within a year to 18 months, we'll have a general method for training hierarchical world models These models would learn from video and real-world data, then help plan actions in robotics, healthcare, and other areas "then scale them toward a universal world model"

Haider.

201,090 görüntüleme • 2 ay önce

Google DeepMind introduced two foundational models for embodied reasoning, enabling robots to comprehend, react, and take action in the physical world: ⦿ Gemini Robotics – built on Gemini 2.0. Integrates vision, language, and action for real-world dexterity, . ⦿ Gemini Robotics-ER – Enhances spatial reasoning for advanced robotic control. They are working with Apptronik to develop the next generation of humanoid robots.
1:11

Google DeepMind introduced two foundational models for embodied reasoning, enabling robots to comprehend, react, and take action in the physical world: ⦿ Gemini Robotics – built on Gemini 2.0. Integrates vision, language, and action for real-world dexterity, . ⦿ Gemini Robotics-ER – Enhances spatial reasoning for advanced robotic control. They are working with Apptronik to develop the next generation of humanoid robots.

The Humanoid Hub

73,097 görüntüleme • 1 yıl önce

Action-controllable world models are evolving fast. LingBot-World-Infinity can run an unbroken session of real-time world generation for up to an hour. A distilled model can stream 720p at 60 fps. World models are going to be a critical piece in training robotic agents for unseen worlds.
1:33

Action-controllable world models are evolving fast. LingBot-World-Infinity can run an unbroken session of real-time world generation for up to an hour. A distilled model can stream 720p at 60 fps. World models are going to be a critical piece in training robotic agents for unseen worlds.

The Humanoid Hub

14,007 görüntüleme • 22 gün önce

I got to "play" a world model in real life. The Google DeepMind folks set up a crazy demo for Genie. You select glowing orbs to represent your scene and character. It loads the world in the model, and you navigate with joysticks like a video game 🕹️
0:32

I got to "play" a world model in real life. The Google DeepMind folks set up a crazy demo for Genie. You select glowing orbs to represent your scene and character. It loads the world in the model, and you navigate with joysticks like a video game 🕹️

Justine Moore

60,142 görüntüleme • 2 ay önce

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N
1:34

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Russell Mendonca

38,454 görüntüleme • 1 yıl önce

What does JEPA actually learn? We can finally prove it 🌍 So excited to share our theory of identifiable World Models: LeJEPA recovers the latent variables of the world. Plan in the learned World Model as if it were real, same shortest path. 📄:
0:31

What does JEPA actually learn? We can finally prove it 🌍 So excited to share our theory of identifiable World Models: LeJEPA recovers the latent variables of the world. Plan in the learned World Model as if it were real, same shortest path. 📄:

David Klindt

216,805 görüntüleme • 2 ay önce

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.
0:40

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Runway

13,464 görüntüleme • 5 ay önce

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄
0:32

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,413 görüntüleme • 26 gün önce

Need help in the real world? RoboVQA can guide robots and humans through long-horizon tasks on a phone via Google Meet. We release a dataset of 800k (video, question/answer) with robots & humans doing various long-horizon tasks. Data: Google DeepMind
0:49

Need help in the real world? RoboVQA can guide robots and humans through long-horizon tasks on a phone via Google Meet. We release a dataset of 800k (video, question/answer) with robots & humans doing various long-horizon tasks. Data: Google DeepMind

Pierre Sermanet

64,369 görüntüleme • 2 yıl önce

OpenDriveLab just dropped RISE — a new paradigm for robot learning. Instead of expensive real-world trials, robots learn in imagination. A compositional world model simulates futures → evaluates outcomes → updates policy.
4:04

OpenDriveLab just dropped RISE — a new paradigm for robot learning. Instead of expensive real-world trials, robots learn in imagination. A compositional world model simulates futures → evaluates outcomes → updates policy.

Robots Digest 🤖

23,754 görüntüleme • 3 ay önce

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!
0:33

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!

Tesla Recruiting

657,085 görüntüleme • 2 ay önce

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus
3:22

🦋 Last week, our team took delivery of a new quadruped robotic dog from Unitree — the Unitree Go 2 AIR. With Letta and the Animus framework, developers can build embodied agents that think, learn, and act in the real world. Watch how we plan to bring Animus to life on this new hardware. Build 🤖 on Animus

Animus

33,084 görüntüleme • 9 ay önce

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.
3:50

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Physical Intelligence

706,891 görüntüleme • 8 ay önce

Today, we’re launching AI Quests, a game-based learning experience for students to learn firsthand how AI can address real-world societal and scientific challenges, from forecasting floods to mapping the brain. Learn more in our blog →
0:35

Today, we’re launching AI Quests, a game-based learning experience for students to learn firsthand how AI can address real-world societal and scientific challenges, from forecasting floods to mapping the brain. Learn more in our blog →

Google Research

76,967 görüntüleme • 10 ay önce

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n
0:53

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

Chuang Gan

90,261 görüntüleme • 1 yıl önce

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
1:22:44

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

16,197 görüntüleme • 2 ay önce