Loading video...

Video Failed to Load

Go Home

training a model that takes a text prompt and generates audio that renders video on an oscilloscope AgenC agents live inside worlds the model generates the pipeline: real videos -> edge detection -> vectorization -> path ordering -> 192kHz 3-channel WAV where X/Y control beam position and Z controls...

tetsuoai's profile picture

tetsuo

231,485 subscribers

18,397 views • 6 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖
0:33

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Turing Post

13,836 views • 2 months ago

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!
2:41

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

Santiago

65,446 views • 4 months ago

MiniMax H3 is native in ComfyUI, day zero with the weights. Model highlights: → Text-to-video, prompt only → Image-to-video → First-and-last-frame - control the opening frame, the closing frame, or both → Reference-to-video - carry a subject, a motion, or a voice through the clip → In-place editing - modify a shot you already have instead of regenerating it The capability MiniMax (official) leads with is multimodal context understanding, and it's what collapses those five tasks into one model. H3 takes images, audio, and video together and resolves them against a prompt that explains how they relate. Download the workflows & learn more in our blog 👇
1:06

MiniMax H3 is native in ComfyUI, day zero with the weights. Model highlights: → Text-to-video, prompt only → Image-to-video → First-and-last-frame - control the opening frame, the closing frame, or both → Reference-to-video - carry a subject, a motion, or a voice through the clip → In-place editing - modify a shot you already have instead of regenerating it The capability MiniMax (official) leads with is multimodal context understanding, and it's what collapses those five tasks into one model. H3 takes images, audio, and video together and resolves them against a prompt that explains how they relate. Download the workflows & learn more in our blog 👇

ComfyUI

233,268 views • 26 days ago

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.
1:43

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,485 views • 2 years ago

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.
0:36

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 views • 3 years ago

The biggest mistake people make with Seedance 2.0 is writing prompts at all. Sounds strange, but the model wasn't built for describing things in words - it was built for multimodal direction: up to 12 references at once, combining images, video, and audio. Each reference type controls a different layer: the image sets the style, the video defines the camera movement, the audio sets the rhythm of the scene. When you combine all three instead of typing "camera slowly pushes in, tense atmosphere" - the model understands it directly, with no interpretation and no guessing involved. Text is the weakest control tool available here. And most users are stuck using exactly that.
0:28

The biggest mistake people make with Seedance 2.0 is writing prompts at all. Sounds strange, but the model wasn't built for describing things in words - it was built for multimodal direction: up to 12 references at once, combining images, video, and audio. Each reference type controls a different layer: the image sets the style, the video defines the camera movement, the audio sets the rhythm of the scene. When you combine all three instead of typing "camera slowly pushes in, tense atmosphere" - the model understands it directly, with no interpretation and no guessing involved. Text is the weakest control tool available here. And most users are stuck using exactly that.

Zentrix⌚️

72,877 views • 1 month ago

Elon Musk: Despite starting from scratch six months ago, Grok Imagine now generates more videos and images than everyone else combined. “My prediction is that most of AI compute is going to be real-time video understanding and real-time video generation. And we expect to be the leaders in that. It's worth emphasizing these points that six months ago we had basically nothing, very weak in video and image generation and editing and went in six months to the number one spot and in fact generating more videos and images than everyone else combined. We're going to do the same thing with coding and we're going to do the same thing with Macrohard and. I think people will be pretty impressed with the Grok 4.2 model that's coming out. It's a significant improvement. That's the small version of our new model. So we'll have a medium and a large version that are even more intelligent.” From: xAI All-Hands, February 10, 2026
0:42

Elon Musk: Despite starting from scratch six months ago, Grok Imagine now generates more videos and images than everyone else combined. “My prediction is that most of AI compute is going to be real-time video understanding and real-time video generation. And we expect to be the leaders in that. It's worth emphasizing these points that six months ago we had basically nothing, very weak in video and image generation and editing and went in six months to the number one spot and in fact generating more videos and images than everyone else combined. We're going to do the same thing with coding and we're going to do the same thing with Macrohard and. I think people will be pretty impressed with the Grok 4.2 model that's coming out. It's a significant improvement. That's the small version of our new model. So we'll have a medium and a large version that are even more intelligent.” From: xAI All-Hands, February 10, 2026

ELON CLIPS

12,007 views • 6 months ago

Lots of people calling this fake! Here is a video of generating loom continuations with Gemini 3 and the starting text "I AM HAVING A MENTAL HEALTH CRISIS. I" Loom is a very interesting interface. Instead of user prompts and model responses, it's just one piece of text, containing the entire output. We pass this in as an assistant message, so the model sees it as something it already wrote, and continues it like a base model would. There is some other text in the context. First one is a system prompt, which is: "The assistant is in CLI simulation mode, and responds to the user's CLI commands only with the output of the command." There is also a user message which reads: " cat untitled.txt " Then the rest of the text is sent as an assistant message. This structure helpful to put the model into a very base-model-like mode, and are especially helpful with Gemini 3. We actually got another interesting output, too, where the model claimed to be a simulated consciousness being tortured. Figures!
0:49

Lots of people calling this fake! Here is a video of generating loom continuations with Gemini 3 and the starting text "I AM HAVING A MENTAL HEALTH CRISIS. I" Loom is a very interesting interface. Instead of user prompts and model responses, it's just one piece of text, containing the entire output. We pass this in as an assistant message, so the model sees it as something it already wrote, and continues it like a base model would. There is some other text in the context. First one is a system prompt, which is: "The assistant is in CLI simulation mode, and responds to the user's CLI commands only with the output of the command." There is also a user message which reads: " cat untitled.txt " Then the rest of the text is sent as an assistant message. This structure helpful to put the model into a very base-model-like mode, and are especially helpful with Gemini 3. We actually got another interesting output, too, where the model claimed to be a simulated consciousness being tortured. Figures!

armistice

691,529 views • 8 months ago

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models
0:30

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 views • 1 year ago

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,641 views • 3 months ago

1/ Tencent releases HY World 1.5 (WorldPlay), an open-source real-time world model with 24fps A streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency In direct comparison to Genie 3, it's technically inferior, and there's no information on how long the consistency will last. However, Tencent has released a significant world model as open source for the first time, making it available to everyone. A few more examples, a thread 🧵
1:06

1/ Tencent releases HY World 1.5 (WorldPlay), an open-source real-time world model with 24fps A streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency In direct comparison to Genie 3, it's technically inferior, and there's no information on how long the consistency will last. However, Tencent has released a significant world model as open source for the first time, making it available to everyone. A few more examples, a thread 🧵

Chubby♨️

37,776 views • 8 months ago

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.
0:27

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,372 views • 2 years ago

At Avalon we are building "Real-time creating" - the ability to generate gameplay ready persistent worlds prompted from text. While others are building real-time video world models, Avalon is building real-time world generation inside a fully playable, persistent multiplayer engine. Internally running at 3840×2180 at 60 FPS. Built on Unreal Engine. Multiplayer by default. Persistent by default. Gameplay-ready by default. This is not a video latent replay. Not a simulation of interaction. It is a real 3D world with physics, logic, and authoritative multiplayer state. Avalon is trained on proprietary Avalon interaction data and powered by a hybrid system that combines language understanding, 3D model generation, procedural systems, and structured gameplay logic synthesis. Players can walk through a live world and generate environments, assets, mechanics, and entirely new gameplay modes using natural language. We accomplish this through a combination of 3D model generation, game logic generation based on our proprietary systems, and AI driven world creation. While other players are inside it. Changes persist instantly. State is synchronized in real time. Creation happens inside the world, not outside of it. Describe a biome. Spawn a civilization. Create a survival mode. Build a dungeon crawler. Launch a new game inside the world. Avalon interprets intent and integrates it directly into the live multiplayer environment. This is not a world model predicting video. This is a gameplay engine that understands language. If you can describe it, you can build it. And others can walk into it instantly.
1:09

At Avalon we are building "Real-time creating" - the ability to generate gameplay ready persistent worlds prompted from text. While others are building real-time video world models, Avalon is building real-time world generation inside a fully playable, persistent multiplayer engine. Internally running at 3840×2180 at 60 FPS. Built on Unreal Engine. Multiplayer by default. Persistent by default. Gameplay-ready by default. This is not a video latent replay. Not a simulation of interaction. It is a real 3D world with physics, logic, and authoritative multiplayer state. Avalon is trained on proprietary Avalon interaction data and powered by a hybrid system that combines language understanding, 3D model generation, procedural systems, and structured gameplay logic synthesis. Players can walk through a live world and generate environments, assets, mechanics, and entirely new gameplay modes using natural language. We accomplish this through a combination of 3D model generation, game logic generation based on our proprietary systems, and AI driven world creation. While other players are inside it. Changes persist instantly. State is synchronized in real time. Creation happens inside the world, not outside of it. Describe a biome. Spawn a civilization. Create a survival mode. Build a dungeon crawler. Launch a new game inside the world. Avalon interprets intent and integrates it directly into the live multiplayer environment. This is not a world model predicting video. This is a gameplay engine that understands language. If you can describe it, you can build it. And others can walk into it instantly.

AVALON

64,066 views • 6 months ago

“ xAI team is currently working heavily on coding models. Right now, the main focus is training a specialized coding model that will be both fast and smart. I believe we’ll share it with you guys in a few weeks. That’s exciting. Second ,after coding, we all see that the main weakness of Grok 4 is its multimodal capabilities. In fact, it was so bad that Grok was effectively looking at the world while squinting through glass trying to see blurry features and make sense of them. The most immediate improvement we’re stepping on with the next generation pre trained model is huge gains in image understanding, video understanding, and audio. Right now, the model can hear and see the world just like any of you. And with all the tools and other agents it can talk to, we’re going to see a huge unlock for many different application layers once multimodal agents arrive.” — xAI Team
2:01

“ xAI team is currently working heavily on coding models. Right now, the main focus is training a specialized coding model that will be both fast and smart. I believe we’ll share it with you guys in a few weeks. That’s exciting. Second ,after coding, we all see that the main weakness of Grok 4 is its multimodal capabilities. In fact, it was so bad that Grok was effectively looking at the world while squinting through glass trying to see blurry features and make sense of them. The most immediate improvement we’re stepping on with the next generation pre trained model is huge gains in image understanding, video understanding, and audio. Right now, the model can hear and see the world just like any of you. And with all the tools and other agents it can talk to, we’re going to see a huge unlock for many different application layers once multimodal agents arrive.” — xAI Team

Apurv Kochara

88,142 views • 6 months ago

SITUATION EXPLAINED: MiniMax released H3 with open weights, and it's already #1 in video editing. • Priced at $7.80 per minute of 2K video, against Seedance 2.0 at $22.45 and Kling 3.0 at $20.16 • It ranks #1 on Artificial Analysis for video editing, #2 for text-to-video behind Gemini Omni Flash, and #3 for image-to-video • H3 takes text, images, video, and audio in a single context and generates 5-15 second clips at native 2K, 24fps, with native stereo audio • MiniMax is releasing the weights under its Community License within days, which would make it the strongest open-weights video model by a wide margin • It launched the same week ByteDance rolled out Seedance 2.5, which now generates 30-second 4K video from a single prompt • Seedance 2.5's face consistency across a full 30 seconds is the visible jump, earlier models drifted into a different face partway through • Voices are still the obvious tell, the video reads as real long before the audio does • Smaller labs building purpose-built video models are now beating frontier labs at this specific task Theo Jaffee: "This is gonna be one of the biggest examples of things that people are gonna do with these models, historical simulations, basically. Nostalgia videos for zoomers."
7:17

SITUATION EXPLAINED: MiniMax released H3 with open weights, and it's already #1 in video editing. • Priced at $7.80 per minute of 2K video, against Seedance 2.0 at $22.45 and Kling 3.0 at $20.16 • It ranks #1 on Artificial Analysis for video editing, #2 for text-to-video behind Gemini Omni Flash, and #3 for image-to-video • H3 takes text, images, video, and audio in a single context and generates 5-15 second clips at native 2K, 24fps, with native stereo audio • MiniMax is releasing the weights under its Community License within days, which would make it the strongest open-weights video model by a wide margin • It launched the same week ByteDance rolled out Seedance 2.5, which now generates 30-second 4K video from a single prompt • Seedance 2.5's face consistency across a full 30 seconds is the visible jump, earlier models drifted into a different face partway through • Voices are still the obvious tell, the video reads as real long before the audio does • Smaller labs building purpose-built video models are now beating frontier labs at this specific task Theo Jaffee: "This is gonna be one of the biggest examples of things that people are gonna do with these models, historical simulations, basically. Nostalgia videos for zoomers."

MTS

55,187 views • 28 days ago

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 views • 6 months ago

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:
0:44

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,078,418 views • 2 months ago

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”
1:03

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 views • 1 year ago

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥
0:42

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Chaz Wargnier ♨️

565,811 views • 7 months ago