Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes...

SawyerMerritt's profile picture

Sawyer Merritt

1,096,087 subscribers

133,168 Aufrufe • vor 6 Monaten •via X (Twitter)

Nachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

NEWS: Waymo has released a new blog post detailing their AI strategy and how it’s allowing them to bring service to more riders faster. "Achieving demonstrably safe AI — where safety is proven, not just promised — requires a holistic approach. Beyond a smart and capable Driver, you also need a closed-loop, realistic Simulator to train and rigorously test the Driver in a myriad of challenging situations, and a sharp Critic to evaluate the Driver's performance and identify areas for improvement." Waymo says that autonomous driving isn’t just a matter of building a “smart driver,” but rather creating a full AI ecosystem centered on safety from the ground up. At the core is the Waymo Foundation Model, a unified world-model that powers all major components of Waymo’s autonomous stack (Driver, Simulator, Critic). "By using a “Think Fast/Think Slow” architecture (combining rapid sensor-fusion with deep semantic reasoning), this system enables the car to detect complex and rare road scenarios (e.g. a burning vehicle ahead), reason about them, and choose safe behavior. Waymo trains large “Teacher” AI-models for driving, simulation, and evaluation, then distills them into smaller, efficient “Student” models suitable for real-world deployment, while keeping safety validation tightly integrated. The result is a continuous “flywheel” of learning: driving data (real and simulated) generate feedback, which leads to refinements, more simulation, more data, and only when safety checks pass is new code deployed. Having already exceeded 100 million fully autonomous miles, Waymo reports a more than ten-fold reduction in severe-injury crashes compared to human drivers." Full blog post:
1:33

NEWS: Waymo has released a new blog post detailing their AI strategy and how it’s allowing them to bring service to more riders faster. "Achieving demonstrably safe AI — where safety is proven, not just promised — requires a holistic approach. Beyond a smart and capable Driver, you also need a closed-loop, realistic Simulator to train and rigorously test the Driver in a myriad of challenging situations, and a sharp Critic to evaluate the Driver's performance and identify areas for improvement." Waymo says that autonomous driving isn’t just a matter of building a “smart driver,” but rather creating a full AI ecosystem centered on safety from the ground up. At the core is the Waymo Foundation Model, a unified world-model that powers all major components of Waymo’s autonomous stack (Driver, Simulator, Critic). "By using a “Think Fast/Think Slow” architecture (combining rapid sensor-fusion with deep semantic reasoning), this system enables the car to detect complex and rare road scenarios (e.g. a burning vehicle ahead), reason about them, and choose safe behavior. Waymo trains large “Teacher” AI-models for driving, simulation, and evaluation, then distills them into smaller, efficient “Student” models suitable for real-world deployment, while keeping safety validation tightly integrated. The result is a continuous “flywheel” of learning: driving data (real and simulated) generate feedback, which leads to refinements, more simulation, more data, and only when safety checks pass is new code deployed. Having already exceeded 100 million fully autonomous miles, Waymo reports a more than ten-fold reduction in severe-injury crashes compared to human drivers." Full blog post:

Sawyer Merritt

83,625 Aufrufe • vor 8 Monaten

Earlier this year, we began testing the Waymo Driver on Phoenix freeways in rider-only mode with our employees. Here’s a glimpse of freeway driving as we keep pushing forward to safely and responsibly scale our operations.
1:01

Earlier this year, we began testing the Waymo Driver on Phoenix freeways in rider-only mode with our employees. Here’s a glimpse of freeway driving as we keep pushing forward to safely and responsibly scale our operations.

Dmitri Dolgov

190,873 Aufrufe • vor 2 Jahren

Introducing GAIA-2 🌎Generative world modeling just stepped up a gear. GAIA-2 is the latest development of Wayve’s video-generative world model tailored for driving. GAIA-2 offers richer, more realistic, and highly controllable synthetic driving scenarios, accelerating Wayve’s path to safe driver assistance and automated driving at scale. Learn more about GAIA-2 in our Blog: #GAIA2 #GAIA #EmbodiedAI
0:30

Introducing GAIA-2 🌎Generative world modeling just stepped up a gear. GAIA-2 is the latest development of Wayve’s video-generative world model tailored for driving. GAIA-2 offers richer, more realistic, and highly controllable synthetic driving scenarios, accelerating Wayve’s path to safe driver assistance and automated driving at scale. Learn more about GAIA-2 in our Blog: #GAIA2 #GAIA #EmbodiedAI

Wayve

41,675 Aufrufe • vor 1 Jahr

Even my kids know that in a world full of Tesla Robotaxis, Waymo is dunzo. It’s really not that complicated. Waymo needs expensive LiDAR, radar, cameras, specialized hardware, detailed maps, and a massive amount of infrastructure just to operate in certain areas. Tesla’s approach is completely different. Cameras are cheap. Tesla can mass-produce vehicles at a scale Waymo can’t come close to, use the same AI system across millions of cars, collect massive amounts of real-world driving data, and improve the entire fleet through software updates. That’s where the economics become almost impossible to compete with. The cheaper you can build the car, the cheaper you can operate it, and the more vehicles you can put on the road, the cheaper you can eventually make every ride. Waymo might prove Robotaxis can work. Tesla’s goal is to make them cheap enough and build enough of them to be everywhere. That’s the difference people still don’t understand. This isn’t just about who has the better Robotaxi today. It’s about who can build millions of them, drive down the cost per mile, and scale across the world. Even my kids understand the end game. Tesla Robotaxis are going to be everywhere. In the world where Tesla Robotaxis are flooded all around the world, it will very difficult for Waymo to survive.
0:28

Even my kids know that in a world full of Tesla Robotaxis, Waymo is dunzo. It’s really not that complicated. Waymo needs expensive LiDAR, radar, cameras, specialized hardware, detailed maps, and a massive amount of infrastructure just to operate in certain areas. Tesla’s approach is completely different. Cameras are cheap. Tesla can mass-produce vehicles at a scale Waymo can’t come close to, use the same AI system across millions of cars, collect massive amounts of real-world driving data, and improve the entire fleet through software updates. That’s where the economics become almost impossible to compete with. The cheaper you can build the car, the cheaper you can operate it, and the more vehicles you can put on the road, the cheaper you can eventually make every ride. Waymo might prove Robotaxis can work. Tesla’s goal is to make them cheap enough and build enough of them to be everywhere. That’s the difference people still don’t understand. This isn’t just about who has the better Robotaxi today. It’s about who can build millions of them, drive down the cost per mile, and scale across the world. Even my kids understand the end game. Tesla Robotaxis are going to be everywhere. In the world where Tesla Robotaxis are flooded all around the world, it will very difficult for Waymo to survive.

Teslaconomics

34,323 Aufrufe • vor 28 Tagen

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️
0:33

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,667 Aufrufe • vor 1 Jahr

XPENG is thrilled to introduce X-World, our controllable multi-view generative world model powered by video diffusion technology. Pushing the boundaries of smart mobility, X-World delivers real-time response and continuous multi-view generation to simulate complex driving environments. Xianming Liu, Head of XPENG’s General AI Center, emphasizes: "Beyond generation, X-World serves as the scalable infrastructure critical for training and validating the next leap in autonomous driving." $XPEV
1:36

XPENG is thrilled to introduce X-World, our controllable multi-view generative world model powered by video diffusion technology. Pushing the boundaries of smart mobility, X-World delivers real-time response and continuous multi-view generation to simulate complex driving environments. Xianming Liu, Head of XPENG’s General AI Center, emphasizes: "Beyond generation, X-World serves as the scalable infrastructure critical for training and validating the next leap in autonomous driving." $XPEV

XPENG

4,070,876 Aufrufe • vor 4 Monaten

Today we're announcing #GAIA1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible. These videos are entirely synthetically generated by Wayve's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving. Why is this game-changing for autonomous driving? 1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety. 2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving. 3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario. Check out many more videos in our blog or further technical details in our paper: Or come chat with our team who are at the International Conference on Computer Vision (#ICCV2023) this week in Paris in Booth 32 Jamie Shotton
0:41

Today we're announcing #GAIA1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible. These videos are entirely synthetically generated by Wayve's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving. Why is this game-changing for autonomous driving? 1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety. 2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving. 3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario. Check out many more videos in our blog or further technical details in our paper: Or come chat with our team who are at the International Conference on Computer Vision (#ICCV2023) this week in Paris in Booth 32 Jamie Shotton

Alex Kendall

631,867 Aufrufe • vor 2 Jahren

Our friends Waymo have simulated billions of miles of driving, across countless types of scenarios and weather conditions – from harsh sun to intense snow. ❄️ By learning from those experiences, in addition to real-world driving, Waymo’s AI is ready for any rare, complex events. Hear the deep dive with Hannah Fry and Waymo's Distinguished Engineer Vincent Vanhoucke, on our podcast ↓ Timecodes 00:00 Intro 01:02 Ride around town 03:48 The driverless car problem 08:43 Sensors 13:00 3D model of the world 16:42 Closed loop problem 22:51 Multimodal models 29:31 Tokenizing 34:39 Human drivers 43:51 Safety 48:41 London and the future 52:44 Outro
53:44

Our friends Waymo have simulated billions of miles of driving, across countless types of scenarios and weather conditions – from harsh sun to intense snow. ❄️ By learning from those experiences, in addition to real-world driving, Waymo’s AI is ready for any rare, complex events. Hear the deep dive with Hannah Fry and Waymo's Distinguished Engineer Vincent Vanhoucke, on our podcast ↓ Timecodes 00:00 Intro 01:02 Ride around town 03:48 The driverless car problem 08:43 Sensors 13:00 3D model of the world 16:42 Closed loop problem 22:51 Multimodal models 29:31 Tokenizing 34:39 Human drivers 43:51 Safety 48:41 London and the future 52:44 Outro

Google DeepMind

54,916 Aufrufe • vor 9 Monaten

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 Aufrufe • vor 6 Monaten

NEW: Phoenix police officer pulls over a driverless Waymo car for driving on the wrong side of the road leading the nonexistent driver to roll the windows down to speak with the police. This is hilarious. Waymo has since released a statement, apparently blaming the incident on "inconsistent construction signage." The car "was blocked from navigating back into the correct lane" for about 30 seconds when the officer pulled it over. "In an effort to clear the intersection, the Waymo vehicle proceeded forward a short distance and pulled into the next available parking lot," the company said.
1:21

NEW: Phoenix police officer pulls over a driverless Waymo car for driving on the wrong side of the road leading the nonexistent driver to roll the windows down to speak with the police. This is hilarious. Waymo has since released a statement, apparently blaming the incident on "inconsistent construction signage." The car "was blocked from navigating back into the correct lane" for about 30 seconds when the officer pulled it over. "In an effort to clear the intersection, the Waymo vehicle proceeded forward a short distance and pulled into the next available parking lot," the company said.

Collin Rugg

10,377,839 Aufrufe • vor 2 Jahren

For the first time ever, I rode in a fully autonomous driverless ride-hailing (Waymo) vehicle in San Fransciso. The vehicle navigated the busy streets just like any vehicle with a driver. But, I got to admit, it was surreal sitting in the back and riding without a driver (and no one to talk). The vehicle’s computer controlled everything including the music, the temperature, announcements such as how long the ride takes and arrival. Waymo and similar others could be the new norm in many cities around the world in the not distant future. The question is: What would happen to the jobs of millions of drivers of taxis and ride hailing services around the world? Waymo is owned by Alphabet (Google’s parent) and has expanded beyond its original base in San Francisco to other U.S. cities including Los Angeles, Dallas, etc.
0:38

For the first time ever, I rode in a fully autonomous driverless ride-hailing (Waymo) vehicle in San Fransciso. The vehicle navigated the busy streets just like any vehicle with a driver. But, I got to admit, it was surreal sitting in the back and riding without a driver (and no one to talk). The vehicle’s computer controlled everything including the music, the temperature, announcements such as how long the ride takes and arrival. Waymo and similar others could be the new norm in many cities around the world in the not distant future. The question is: What would happen to the jobs of millions of drivers of taxis and ride hailing services around the world? Waymo is owned by Alphabet (Google’s parent) and has expanded beyond its original base in San Francisco to other U.S. cities including Los Angeles, Dallas, etc.

Zemedeneh Negatu

437,360 Aufrufe • vor 3 Monaten

3-months ago we went from only driving in the UK, California and Germany, to driving in seven different countries with very different climates. 🌍 🇺🇸 In the US, we went from driving in one state to driving in ten! From the sunny streets of California to the snowy mountain passes in Wyoming, the Wayve AI Driver is navigating a diversity of environments in the US on our journey to driving in 500 cities around the world. ~400 cities to go before the end of the year - all enabled by one global foundation model 🌍 #AI500 #GlobalRoadTrip #EmbodiedAI
0:25

3-months ago we went from only driving in the UK, California and Germany, to driving in seven different countries with very different climates. 🌍 🇺🇸 In the US, we went from driving in one state to driving in ten! From the sunny streets of California to the snowy mountain passes in Wyoming, the Wayve AI Driver is navigating a diversity of environments in the US on our journey to driving in 500 cities around the world. ~400 cities to go before the end of the year - all enabled by one global foundation model 🌍 #AI500 #GlobalRoadTrip #EmbodiedAI

Wayve

32,943 Aufrufe • vor 1 Jahr

Tesla Is Now An AI And Robotics Company ... What Does That Mean For The Model S and X - Episode #21 of The Gigacast ---------------------------------------------------------- Intro Tesla refreshes the Model S and X for the first time in almost half a decade What is the future of the Model S and X in an Autonomous World | Can/Will it ever be material for Tesla Is the Model Y and 3 just too good of a deal compared to the S and X | What does the luxury car market Look Like Tesla FSD safety Bloomberg report Omar, Waymo, and Tesla in New York City Waymo and the Jaguar I-Pace Over the air updates | Tesla's getting better over time Keeping up the pace if Innovation: Tesla vs. Apple Questions from the Comments This is the last episode before the launch of FSD in Austin, Texas (Hopefully) | What we're watching ----------------------------------------------------------
57:59

Tesla Is Now An AI And Robotics Company ... What Does That Mean For The Model S and X - Episode #21 of The Gigacast ---------------------------------------------------------- Intro Tesla refreshes the Model S and X for the first time in almost half a decade What is the future of the Model S and X in an Autonomous World | Can/Will it ever be material for Tesla Is the Model Y and 3 just too good of a deal compared to the S and X | What does the luxury car market Look Like Tesla FSD safety Bloomberg report Omar, Waymo, and Tesla in New York City Waymo and the Jaguar I-Pace Over the air updates | Tesla's getting better over time Keeping up the pace if Innovation: Tesla vs. Apple Questions from the Comments This is the last episode before the launch of FSD in Austin, Texas (Hopefully) | What we're watching ----------------------------------------------------------

The Gigacast

41,266 Aufrufe • vor 1 Jahr

Waymo’s first autonomous demo took eighteen months. The product took fifteen years. Today the Waymo Driver runs 500,000 trips a week — 4 million fully autonomous miles across 15 cities, with 17x fewer serious-injury crashes than human drivers. At Startup School 2026, Waymo co-CEO Dmitri Dolgov shares the seven lessons behind that journey, from bridging the gap between a demo and a real product to building systems that can safely operate in the physical world. 00:07 — 7 Lessons From Building Waymo 02:21 — Why Physical AI Is Different 06:52 — The Gap Between a Demo and a Product 11:34 — Why Reliability Lives on an Exponential Curve 14:17 — Pick the Right Technology Curve 16:05 — Why Waymo Uses Cameras, LiDAR, and Radar 21:07 — Ride Every Technology Wave 24:43 — Inside the Waymo Foundation Model 30:09 — The Bitter Lesson Still Wins 36:41 — Why Every Physical AI Company Needs a Simulator 41:36 — Build an AI Flywheel 43:22 — Evals Are Your Competitive Advantage 46:09 — How Waymo Became 17x Safer Than Human Drivers 47:59 — The Next Decade of AI Will Be Physical
49:24

Waymo’s first autonomous demo took eighteen months. The product took fifteen years. Today the Waymo Driver runs 500,000 trips a week — 4 million fully autonomous miles across 15 cities, with 17x fewer serious-injury crashes than human drivers. At Startup School 2026, Waymo co-CEO Dmitri Dolgov shares the seven lessons behind that journey, from bridging the gap between a demo and a real product to building systems that can safely operate in the physical world. 00:07 — 7 Lessons From Building Waymo 02:21 — Why Physical AI Is Different 06:52 — The Gap Between a Demo and a Product 11:34 — Why Reliability Lives on an Exponential Curve 14:17 — Pick the Right Technology Curve 16:05 — Why Waymo Uses Cameras, LiDAR, and Radar 21:07 — Ride Every Technology Wave 24:43 — Inside the Waymo Foundation Model 30:09 — The Bitter Lesson Still Wins 36:41 — Why Every Physical AI Company Needs a Simulator 41:36 — Build an AI Flywheel 43:22 — Evals Are Your Competitive Advantage 46:09 — How Waymo Became 17x Safer Than Human Drivers 47:59 — The Next Decade of AI Will Be Physical

Y Combinator

58,218 Aufrufe • vor 4 Tagen

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.
0:51

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,110 Aufrufe • vor 1 Jahr

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO
1:02

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO

The Humanoid Hub

68,453 Aufrufe • vor 6 Monaten

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado
1:30

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

34,650 Aufrufe • vor 9 Tagen

Applied Intuition CEO Qasar Younis on the autonomous driving approaches being taken by Tesla and Waymo, and why both can succeed in the years to come: "Generally speaking, every single car company on the planet right now is working on a product that’s like a Tesla FSD product. Many companies are working on versions of that, that would become fully autonomous within a cheap sensor suite." "So the fundamental difference, just to simplify the Tesla approach versus the Waymo approach... the Waymo approach is lots of sensors and lots of compute, and maps, and the Tesla version is very few sensors, no high fidelity maps... and cheaper compute for a lack of a better word." "And the Tesla version of a product, this in the industry is called an L2++ product, is going to be available everywhere because it’s literally cheaper and it doesn’t require HD maps. The Waymo product functions better in a geographically constrained area." "So, fast forward five years, both of these types of technologies will be much more ubiquitous. L2++ and L4 will be much more ubiquitous, not only in the Bay Area or in parts of China, but really globally - there are companies working on this globally." "We’re at that moment for L2++ systems. Where people are willing to pay thousands of dollars for a semi-automated vehicle." "It will not be a long time — you’re already seeing this in China — where the downward pricing pressure for the autonomous product, for lack of a better word, will become close to free." Qasar Younis Lenny Rachitsky
1:35

Applied Intuition CEO Qasar Younis on the autonomous driving approaches being taken by Tesla and Waymo, and why both can succeed in the years to come: "Generally speaking, every single car company on the planet right now is working on a product that’s like a Tesla FSD product. Many companies are working on versions of that, that would become fully autonomous within a cheap sensor suite." "So the fundamental difference, just to simplify the Tesla approach versus the Waymo approach... the Waymo approach is lots of sensors and lots of compute, and maps, and the Tesla version is very few sensors, no high fidelity maps... and cheaper compute for a lack of a better word." "And the Tesla version of a product, this in the industry is called an L2++ product, is going to be available everywhere because it’s literally cheaper and it doesn’t require HD maps. The Waymo product functions better in a geographically constrained area." "So, fast forward five years, both of these types of technologies will be much more ubiquitous. L2++ and L4 will be much more ubiquitous, not only in the Bay Area or in parts of China, but really globally - there are companies working on this globally." "We’re at that moment for L2++ systems. Where people are willing to pay thousands of dollars for a semi-automated vehicle." "It will not be a long time — you’re already seeing this in China — where the downward pricing pressure for the autonomous product, for lack of a better word, will become close to free." Qasar Younis Lenny Rachitsky

a16z

60,385 Aufrufe • vor 5 Monaten

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.
1:52

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.

reactor

12,735,559 Aufrufe • vor 2 Monaten