Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

World models should not just look real. They should obey physics. CrashTwin stress-tests generative world models on safety-critical crashes, measuring temporal consistency, momentum and energy conservation, and identity stability. Data + code:

zhiwen_fan_'s profile picture

Zhiwen(Aaron) Fan

1,836 subscribers

16,933 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Japan shows the world what real safety and trust for children should look like everywhere.
1:43

Japan shows the world what real safety and trust for children should look like everywhere.

GABRIEL 🪽

24,581 Aufrufe • vor 2 Monaten

Data for the Real World Diana & Austin Tindle AI has become superhuman at code, language, and images because it has enormous amounts of data to learn from. But our understanding of the physical world is still limited by sparse sensors and models built largely on intuition. Cheaper sensors and better foundation models could unlock entirely new datasets for energy, agriculture, logistics, construction, and the climate. Once you can accurately model the physical world, you can begin to control it.
1:05

Data for the Real World Diana & Austin Tindle AI has become superhuman at code, language, and images because it has enormous amounts of data to learn from. But our understanding of the physical world is still limited by sparse sensors and models built largely on intuition. Cheaper sensors and better foundation models could unlock entirely new datasets for energy, agriculture, logistics, construction, and the climate. Once you can accurately model the physical world, you can begin to control it.

Y Combinator

22,185 Aufrufe • vor 1 Monat

Why can't “world models” just be sim? A real-time real2sim demo in the upcoming “ManiDreams” Foundation Stereo + SAM2 + Newton = Generalizable 3D physics prediction in a fast (15FPS) and lightweight (3070) way. No data or object mesh needed
0:21

Why can't “world models” just be sim? A real-time real2sim demo in the upcoming “ManiDreams” Foundation Stereo + SAM2 + Newton = Generalizable 3D physics prediction in a fast (15FPS) and lightweight (3070) way. No data or object mesh needed

Vector Wang

30,474 Aufrufe • vor 5 Monaten

Current world models often separate spatial reasoning from temporal modeling, leading to consistency bottlenecks and scaling limits. We developed RAYNOVA, a 4D world foundation model, to address this by unifying space and time into a single representation via a pure auto-regressive framework. The result: Data-driven high-fidelity simulation data that scales across any vehicles, anywhere. Read the full technical breakdown:
0:27

Current world models often separate spatial reasoning from temporal modeling, leading to consistency bottlenecks and scaling limits. We developed RAYNOVA, a 4D world foundation model, to address this by unifying space and time into a single representation via a pure auto-regressive framework. The result: Data-driven high-fidelity simulation data that scales across any vehicles, anywhere. Read the full technical breakdown:

Applied Intuition

11,252 Aufrufe • vor 3 Monaten

Been thinking about what this paper really means. "Video diffusion" and "World Models" are becoming synonymous. Neural Computers are basically video diffusion world models for terminal envs and GUI. Lots of talk last week about automating Manim videos. In theory, we should be able to train these world models on a 10000 hours of diverse manim videos and "see where it goes" If a NC can generate outputs to terminal commands, it should be able to generate videos like this directly from prompt too. Without writing code.
0:41

Been thinking about what this paper really means. "Video diffusion" and "World Models" are becoming synonymous. Neural Computers are basically video diffusion world models for terminal envs and GUI. Lots of talk last week about automating Manim videos. In theory, we should be able to train these world models on a 10000 hours of diverse manim videos and "see where it goes" If a NC can generate outputs to terminal commands, it should be able to generate videos like this directly from prompt too. Without writing code.

AVB

119,949 Aufrufe • vor 4 Monaten

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.
1:45

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

AK

32,849 Aufrufe • vor 2 Jahren

Yann LeCun says we're never gonna get to human-level intelligence by just training on text AI must learn from high-bandwidth sensory data like video to build true world models Current models look PhD-smart but mostly regurgitate, with no real understanding "even a cat understands the physical world better"
1:29

Yann LeCun says we're never gonna get to human-level intelligence by just training on text AI must learn from high-bandwidth sensory data like video to build true world models Current models look PhD-smart but mostly regurgitate, with no real understanding "even a cat understands the physical world better"

Haider.

138,265 Aufrufe • vor 10 Monaten

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,676 Aufrufe • vor 3 Monaten

Nexar’s real-world data just built the best incident prediction model — Meet BADAS 1.0. It beat state-of-the-art models by learning from 10B+ real miles and 60M+ real events, not simulations. 🚀 Discover what it can do for you
0:32

Nexar’s real-world data just built the best incident prediction model — Meet BADAS 1.0. It beat state-of-the-art models by learning from 10B+ real miles and 60M+ real events, not simulations. 🚀 Discover what it can do for you

Nexar

709,509 Aufrufe • vor 10 Monaten

"I think that the holodeck is near again minus the hardlight holograms etc. - We’ll be able to have that, with our 3D glasses, with the ocular occlusion and more…” Emad breaks down how diffusion models are learning real-world physics, why open-source world models are arriving faster than expected, and why access to compute becomes the critical layer as these experiences scale. #RC2026
21:28

"I think that the holodeck is near again minus the hardlight holograms etc. - We’ll be able to have that, with our 3D glasses, with the ocular occlusion and more…” Emad breaks down how diffusion models are learning real-world physics, why open-source world models are arriving faster than expected, and why access to compute becomes the critical layer as these experiences scale. #RC2026

The Render Network

12,435 Aufrufe • vor 4 Monaten

Given the strong community adoption and real-world deployment of MolmoAct2 on YAM, we're introducing zero-shot evaluation of MolmoAct2 Bimanual YAM in simulation. Now you can test out our models without a real-world robot and build on them! Code: Simulation built on Maniskill!
0:20

Given the strong community adoption and real-world deployment of MolmoAct2 on YAM, we're introducing zero-shot evaluation of MolmoAct2 Bimanual YAM in simulation. Now you can test out our models without a real-world robot and build on them! Code: Simulation built on Maniskill!

Jiafei Duan

37,104 Aufrufe • vor 2 Monaten

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.
1:16:13

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

abdel

38,919 Aufrufe • vor 3 Monaten

.Trenton Bricken explains how we know LLMs are actually generalizing - aka they're not just stochastic parrots: - Training models on code makes them better at reasoning in language. - Models fine tuned on math problems become better at entity detection. - We can just straightforwardly read the world-models developed by smaller NNs which are easier to interpret (Othello). Transfer learning shows models are developing a deeper understanding of their data. Full episode out Thursday.
1:37

.Trenton Bricken explains how we know LLMs are actually generalizing - aka they're not just stochastic parrots: - Training models on code makes them better at reasoning in language. - Models fine tuned on math problems become better at entity detection. - We can just straightforwardly read the world-models developed by smaller NNs which are easier to interpret (Othello). Transfer learning shows models are developing a deeper understanding of their data. Full episode out Thursday.

Dwarkesh Patel

237,812 Aufrufe • vor 2 Jahren

The fact that you can create a video like this in just a few minutes is pretty insane and magical. Anyone can place themselves into any world they want, real or imagined, instantly and soon in real time. The Moore's law of generative AI was right: the quality, resolution, and control of models doubles about every year.
0:52

The fact that you can create a video like this in just a few minutes is pretty insane and magical. Anyone can place themselves into any world they want, real or imagined, instantly and soon in real time. The Moore's law of generative AI was right: the quality, resolution, and control of models doubles about every year.

Cristóbal Valenzuela

20,713 Aufrufe • vor 4 Monaten

Meet LongCat-Video-Avatar 1.5🐱—our upgraded, open-source digital human framework. Built for real production, not just short demos. What's New: 🔹 Upgraded Audio Encoder: Replaces Wav2Vec2 with Whisper-Large, yielding significantly smoother and more natural lip dynamics. 🔹 Production-Ready Stability: Achieves accurate lip-synchronization, full-body temporal stability, and robust long-video generation with strict identity consistency. 🔹 Stylized Domain Generalization: Robustly generalizes to anime, animals, and complex real-world conditions such as multi-person interactions and object handling. 🔹 Efficient 8-Step Inference: Advanced step distillation accelerates inference to 8 NFE, balancing cost-effective serving with exceptional visual fidelity. 📊 LongCat-Video-Avatar 1.5 performs strongly in realism, naturalness, and stability, outperforming leading open-source models and closed systems. 🐱 Avatar 1.5 framework is now open source: 🔗 Weights & Code: 🔗 HuggingFace: 🔗 Tech Report: 🔗 Project Page:
0:13

Meet LongCat-Video-Avatar 1.5🐱—our upgraded, open-source digital human framework. Built for real production, not just short demos. What's New: 🔹 Upgraded Audio Encoder: Replaces Wav2Vec2 with Whisper-Large, yielding significantly smoother and more natural lip dynamics. 🔹 Production-Ready Stability: Achieves accurate lip-synchronization, full-body temporal stability, and robust long-video generation with strict identity consistency. 🔹 Stylized Domain Generalization: Robustly generalizes to anime, animals, and complex real-world conditions such as multi-person interactions and object handling. 🔹 Efficient 8-Step Inference: Advanced step distillation accelerates inference to 8 NFE, balancing cost-effective serving with exceptional visual fidelity. 📊 LongCat-Video-Avatar 1.5 performs strongly in realism, naturalness, and stability, outperforming leading open-source models and closed systems. 🐱 Avatar 1.5 framework is now open source: 🔗 Weights & Code: 🔗 HuggingFace: 🔗 Tech Report: 🔗 Project Page:

Meituan LongCat

31,437 Aufrufe • vor 3 Monaten

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:
1:00

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Heng Yang

19,166 Aufrufe • vor 5 Monaten

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️
25:05

1/ BREAKING – DePIN's Biggest New Deal: Valeo x NATIX Network with Alireza Ghods Host: Connor Lovely 🎬 We discuss: - Why a $3B, Fortune 500 company wants Natix data - What is a World Foundational Model? - Why autonomous driving needs synthetic data Timestamps: 00:00 - Introduction 01:14 - Updates on VX360 and Data Sales 02:09 - Vision Language Model and Its Applications 04:39 - Importance of Diverse Data for World Models 06:20 - Details on the Valeo Deal 08:15 - Use Cases and Future Plans for World Models 10:45 - Synthetic Data and Real-World Applications 19:37 - Future of Physical AI and Autonomous Driving 23:51 - Conclusion Watch on other platforms below ⬇️

Proof of Coverage | Tech’s Creative Agency

11,869 Aufrufe • vor 7 Monaten

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:
0:44

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,078,824 Aufrufe • vor 3 Monaten