Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing Driver — a transformer-based, real-time, vision-only path prediction model for highway and urban driving. It predicts the vehicle’s future path using only camera-based perception—no Lidar, HD maps, or additional sensors required. We deployed it in closed-loop simulation with CARLA Simulator + GenSim-2, our generative AI model that re-renders... show more

Helm.ai

1,469 subscribers

4,040,896 views • 1 year ago •via X (Twitter)

News & Politics Science & Technology Autos & Vehicles

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Introducing GAIA-2 🌎Generative world modeling just stepped up a gear. GAIA-2 is the latest development of Wayve’s video-generative world model tailored for driving. GAIA-2 offers richer, more realistic, and highly controllable synthetic driving scenarios, accelerating Wayve’s path to safe driver assistance and automated driving at scale. Learn more about GAIA-2 in our Blog: #GAIA2 #GAIA #EmbodiedAI

Introducing GAIA-2 🌎Generative world modeling just stepped up a gear. GAIA-2 is the latest development of Wayve’s video-generative world model tailored for driving. GAIA-2 offers richer, more realistic, and highly controllable synthetic driving scenarios, accelerating Wayve’s path to safe driver assistance and automated driving at scale. Learn more about GAIA-2 in our Blog: #GAIA2 #GAIA #EmbodiedAI

Wayve

41,608 views • 1 year ago

Hot take: retained-mode was a premature optimization. We built Membrane using rust and egui and never looked back. When our IDE was react-based we had to worry about reflows and re-renders. With immediate-mode every frame is a re-render! But it only takes 900us

Hot take: retained-mode was a premature optimization. We built Membrane using rust and egui and never looked back. When our IDE was react-based we had to worry about reflows and re-renders. With immediate-mode every frame is a re-render! But it only takes 900us

Juan Campa

89,828 views • 1 year ago

At #GTC26 we are showing AlpaDreams — generative, interactive closed-loop simulation for autonomous driving. World models generate environments in real time, policies or drivers interact, and the simulator updates the world. If you are at GTC, come by the booth and try it out!

At #GTC26 we are showing AlpaDreams — generative, interactive closed-loop simulation for autonomous driving. World models generate environments in real time, policies or drivers interact, and the simulator updates the world. If you are at GTC, come by the booth and try it out!

Zian Wang

16,730 views • 2 months ago

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

Sawyer Merritt

133,082 views • 4 months ago

Excited to share this demo from Over the Reality! Watch our Unitree Go2 robodog navigating our office while reconstructing the 3D space in real-time using the VGGT-based foundation vision model. This is a prime example of machine perception in action, turning raw RGB camera feeds into rich, detailed 3D maps! The robodog's RGB cam generates a dense, textured 3D reconstruction via VGGT from a few photograms, capturing nuances like object shapes and surfaces with impressive fidelity (main view). Compare that to the standard LiDAR system (top right), it's sparser, more point-cloud focused, lacking the visual richness. Vision models are closing the gap fast! What's powering this? VGGT, a cutting-edge foundation model for 3D perception, trained on datasets orders of magnitude smaller than our massive OVER 3D maps dataset. Imagine the leap when we apply OVER's scale to VGGT-like transformer based architectures, denser reconstructions, better generalization, revolutionary for robotics, machine perception & AR! Stay tuned for more breakthroughs at the intersection of AI, robotics, and DePIN. We're building the future of Physical AI and Spatial Computing at Over the Reality 🌐 What do you think, ready for robodogs in your world? Drop your thoughts! 🤖🌐

Excited to share this demo from Over the Reality! Watch our Unitree Go2 robodog navigating our office while reconstructing the 3D space in real-time using the VGGT-based foundation vision model. This is a prime example of machine perception in action, turning raw RGB camera feeds into rich, detailed 3D maps! The robodog's RGB cam generates a dense, textured 3D reconstruction via VGGT from a few photograms, capturing nuances like object shapes and surfaces with impressive fidelity (main view). Compare that to the standard LiDAR system (top right), it's sparser, more point-cloud focused, lacking the visual richness. Vision models are closing the gap fast! What's powering this? VGGT, a cutting-edge foundation model for 3D perception, trained on datasets orders of magnitude smaller than our massive OVER 3D maps dataset. Imagine the leap when we apply OVER's scale to VGGT-like transformer based architectures, denser reconstructions, better generalization, revolutionary for robotics, machine perception & AR! Stay tuned for more breakthroughs at the intersection of AI, robotics, and DePIN. We're building the future of Physical AI and Spatial Computing at Over the Reality 🌐 What do you think, ready for robodogs in your world? Drop your thoughts! 🤖🌐

Over the Reality 🌐

359,500 views • 9 months ago

Google is offering a Generative AI Learning Path with 10 courses for FREE! - Intro to Generative AI - Intro to LLMs - Intro to Image Generation - Encoder-Decoder Architecture - Transformer Models and more A Thread 🧵👇

Google is offering a Generative AI Learning Path with 10 courses for FREE! - Intro to Generative AI - Intro to LLMs - Intro to Image Generation - Encoder-Decoder Architecture - Transformer Models and more A Thread 🧵👇

Afiz ⚡️

249,513 views • 2 years ago

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,327 views • 8 months ago

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

AGIBOT

28,219 views • 2 months ago

SAM 2 from Meta FAIR is the first unified model for real-time, promptable object segmentation in images & videos. Using the model in our web-based demo you can segment, track and apply effects to objects in video in just a few clicks. Try SAM 2 ➡️

SAM 2 from Meta FAIR is the first unified model for real-time, promptable object segmentation in images & videos. Using the model in our web-based demo you can segment, track and apply effects to objects in video in just a few clicks. Try SAM 2 ➡️

AI at Meta

88,918 views • 1 year ago

Introducing Lemon Slice Live: the first real-time audio-video AI model Anyone can video chat with any character by just uploading a single photo. Every pixel on the screen is rendered at 25fps in real-time by a transformer model. Here’s how we built it + a live demo! 👇

Introducing Lemon Slice Live: the first real-time audio-video AI model Anyone can video chat with any character by just uploading a single photo. Every pixel on the screen is rendered at 25fps in real-time by a transformer model. Here’s how we built it + a live demo! 👇

LemonSlice

170,779 views • 1 year ago

Update v1.1.0 for Camera Path is out ;D - Added the ability to reverse the path direction using the Speed Slider. - Improved camera snapping when looping on an open path. - Fix the flicker that sometimes occurred in the livelink, making it more stable #VRChat #CameraPath

Update v1.1.0 for Camera Path is out ;D - Added the ability to reverse the path direction using the Speed Slider. - Improved camera snapping when looping on an open path. - Fix the flicker that sometimes occurred in the livelink, making it more stable #VRChat #CameraPath

DX

98,514 views • 1 year ago

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

LiveKit

126,804 views • 1 year ago

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!

World Labs

339,636 views • 7 months ago

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

15,907 views • 4 months ago

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Kaifeng Zhang

45,946 views • 11 months ago

Introducing Motion Prediction - a new AI model that accurately predicts an actor’s movement, even when they're partially or completely hidden by objects or other actors in a scene.🎬🔮 Motion Prediction enhances AI markerless motion capture by filling in movement gaps caused by occlusions so you can deliver more realistic, lifelike animation. Try it now in Wonder Animation. #wonderdynamics #wonderstudio #animation #ai #CGI

Introducing Motion Prediction - a new AI model that accurately predicts an actor’s movement, even when they're partially or completely hidden by objects or other actors in a scene.🎬🔮 Motion Prediction enhances AI markerless motion capture by filling in movement gaps caused by occlusions so you can deliver more realistic, lifelike animation. Try it now in Wonder Animation. #wonderdynamics #wonderstudio #animation #ai #CGI

Wonder Dynamics

29,068 views • 1 year ago

Tesla FSD vs Competition in China Tesla is the only one with pure vision, meaning no LiDAR or radar sensors.

Tesla FSD vs Competition in China Tesla is the only one with pure vision, meaning no LiDAR or radar sensors.

Nic Cruz Patane

112,401 views • 2 months ago

Transformer Explainer Interactive Learning of Text-Generative Models discuss: Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present Transformer Explainer, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and enabling smooth transitions across abstraction levels of mathematical operations and model structures. It runs a live GPT-2 instance locally in the user's browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. Our tool requires no installation or special hardware, broadening the public's education access to modern generative AI techniques.

Transformer Explainer Interactive Learning of Text-Generative Models discuss: Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present Transformer Explainer, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and enabling smooth transitions across abstraction levels of mathematical operations and model structures. It runs a live GPT-2 instance locally in the user's browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. Our tool requires no installation or special hardware, broadening the public's education access to modern generative AI techniques.

AK

90,723 views • 1 year ago

Can vision-only self-driving compete with LiDAR? Yes, and we think it might be the only solution. Sam Korus explains why for askARK.

Can vision-only self-driving compete with LiDAR? Yes, and we think it might be the only solution. Sam Korus explains why for askARK.

ARK Invest

34,501 views • 9 months ago

Dreamina’s new MultiFrames feature lets creators generate a full 54-second video from just 10 images, with custom frame durations, camera path settings, and keyframe-level prompts — no stitching, no cuts. Here an exemple using 10 images. All details in thread 👇 Did you try it ? Dreamina AI #DreaminaMultiFrames 1/5

Dreamina’s new MultiFrames feature lets creators generate a full 54-second video from just 10 images, with custom frame durations, camera path settings, and keyframe-level prompts — no stitching, no cuts. Here an exemple using 10 images. All details in thread 👇 Did you try it ? Dreamina AI #DreaminaMultiFrames 1/5

LudovicCreator

52,029 views • 6 months ago