Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Nvidia's PID (Pixel diffusion Decoder) It decodes latent representations into high-resolution images, replacing the decode–then–super-resolve cascade while achieving lower latency and higher visual quality page: HF repo:

Stable Diffusion Tutorials

3,572 subscribers

37,247 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

AK

32,849 views • 2 years ago

🚨 BREAKING: A research lab just released a 15B model that generates multilingual talking human videos with synced audio, beats every competitor in human evaluation, and runs in 38 seconds on one GPU. It's called daVinci-MagiHuman. The key insight is that every other model in this category stacks cross-attention, multi-stream pipelines, and separate conditioning branches to handle video and audio together. This one throws all of that out and uses a single unified self-attention stream across all modalities. Super-resolution happens in latent space rather than pixel space so there's no extra VAE decode-encode round trip. The turbo VAE decoder cuts decoding overhead even further. The distilled version runs in 8 steps with no CFG at all. Visual quality, text alignment, and word error rate all beat Ovi 1.1 and LTX 2.3 on the benchmark table. 100% Opensource. Apache 2.0. Repo and research paper links are in the comments.

🚨 BREAKING: A research lab just released a 15B model that generates multilingual talking human videos with synced audio, beats every competitor in human evaluation, and runs in 38 seconds on one GPU. It's called daVinci-MagiHuman. The key insight is that every other model in this category stacks cross-attention, multi-stream pipelines, and separate conditioning branches to handle video and audio together. This one throws all of that out and uses a single unified self-attention stream across all modalities. Super-resolution happens in latent space rather than pixel space so there's no extra VAE decode-encode round trip. The turbo VAE decoder cuts decoding overhead even further. The distilled version runs in 8 steps with no CFG at all. Visual quality, text alignment, and word error rate all beat Ovi 1.1 and LTX 2.3 on the benchmark table. 100% Opensource. Apache 2.0. Repo and research paper links are in the comments.

Ihtesham Ali

45,152 views • 3 months ago

🤔The more I studied diffusion language models, the more I came to appreciate the simplicity of autoregressive (AR) language models. AR models are trained to agree with what they generate, and their serving stacks are built to preserve that structure. DLMs often do neither: they lack introspective consistency, and high TPF does not necessarily translate into high real-world TPS. We propose Introspective Diffusion Language Model (I-DLM), which unifies introspection and generation in a single pass: 1. 🧑‍🎓I-DLM brings introspective consistency to DLMs with only 5B training tokens, achieving AR-thinking-level quality. 2. 🚀 I-DLM carefully trades compute for higher TPF while converting that advantage into real TPS under high-concurrency serving. 📖Website: ⌨️Code:

🤔The more I studied diffusion language models, the more I came to appreciate the simplicity of autoregressive (AR) language models. AR models are trained to agree with what they generate, and their serving stacks are built to preserve that structure. DLMs often do neither: they lack introspective consistency, and high TPF does not necessarily translate into high real-world TPS. We propose Introspective Diffusion Language Model (I-DLM), which unifies introspection and generation in a single pass: 1. 🧑‍🎓I-DLM brings introspective consistency to DLMs with only 5B training tokens, achieving AR-thinking-level quality. 2. 🚀 I-DLM carefully trades compute for higher TPF while converting that advantage into real TPS under high-concurrency serving. 📖Website: ⌨️Code:

Chenfeng_X

344,484 views • 3 months ago

🤩 30% more speed and increased quality in high-resolution image upscaling I've improved clarity upscaling model by reducing the creativity by 20% in every internal upscaling step. It makes the amount of hallucination much more controllable, thus increasing the quality while also being 30% faster ⏩Now live: App: Code: Replicate: 🏞️ It's great how the blurry background stays blurry, but the cat becomes super sharp This example was made with a creativity setting of 0.25, and you can go lower if you want less hallucination

🤩 30% more speed and increased quality in high-resolution image upscaling I've improved clarity upscaling model by reducing the creativity by 20% in every internal upscaling step. It makes the amount of hallucination much more controllable, thus increasing the quality while also being 30% faster ⏩Now live: App: Code: Replicate: 🏞️ It's great how the blurry background stays blurry, but the cat becomes super sharp This example was made with a creativity setting of 0.25, and you can go lower if you want less hallucination

philz1337x

28,236 views • 2 years ago

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,590 views • 9 months ago

We are excited to unveil HunyuanVideo 1.5, the strongest open-source video generation model. Built upon DiT architecture, it redefines the open-source SOTA for accessibility and performance.🚀🚀🚀 HunyuanVideo 1.5 delivers state-of-the-art visual quality and motion coherence while drastically lowering the entry barrier for developers and creators: ⚡️ Unmatched Accessibility: Ultra-light 8.3B parameters, deployable on consumer GPUs with only 14GB VRAM. 🖥️ HD Cinematic Quality: Natively generates 5–10 second 480p/720p HD videos, with super-resolution support for 1080p cinematic quality. By merging SOTA performance with high hardware efficiency, HunyuanVideo 1.5 sets the new technical baseline for the open-source community. 🌐Project Page: 🔗Github: 🤗Hugging Face： 📄Technical Report:

We are excited to unveil HunyuanVideo 1.5, the strongest open-source video generation model. Built upon DiT architecture, it redefines the open-source SOTA for accessibility and performance.🚀🚀🚀 HunyuanVideo 1.5 delivers state-of-the-art visual quality and motion coherence while drastically lowering the entry barrier for developers and creators: ⚡️ Unmatched Accessibility: Ultra-light 8.3B parameters, deployable on consumer GPUs with only 14GB VRAM. 🖥️ HD Cinematic Quality: Natively generates 5–10 second 480p/720p HD videos, with super-resolution support for 1080p cinematic quality. By merging SOTA performance with high hardware efficiency, HunyuanVideo 1.5 sets the new technical baseline for the open-source community. 🌐Project Page: 🔗Github: 🤗Hugging Face： 📄Technical Report:

Tencent Hy

128,008 views • 8 months ago

SqueezeMe: Efficient Gaussian Avatars for VR TL;DR: Three of these Gaussian Splatting avatars can be run at 72 frames per second. It runs locally on a Meta Quest 3 VR headset. Abstract (excerpt): While previous methods require a desktop GPU for real-time inference of a single avatar, we aim to squeeze multiple Gaussian avatars onto a portable virtual reality headset with real-time drivable inference. We begin by training a previous work, Animatable Gaussians, on a high-quality dataset captured with 512 cameras. The Gaussians are animated by controlling a base set of Gaussians with linear blend skinning (LBS) motion, and then further adjusting them with a neural network decoder to correct their appearance. When deploying the model on a Meta Quest 3 VR headset, we find two major computational bottlenecks: the decoder and the rendering. To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space and distill the decoder to a single neural network layer. Further, we discover that neighborhoods of Gaussians can share a single corrective from the decoder, providing an additional speedup. To accelerate the rendering, we develop a custom pipeline in Vulkan that runs on the mobile GPU. Putting it all together, we run 3 Gaussian avatars concurrently at 72 FPS on a VR headset.

MrNeRF

27,104 views • 1 year ago

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

Tencent Hy

89,257 views • 10 months ago

The PMs screenshotting Claude Code outputs and saying "fix the alignment" five times are playing QA on a system that can QA itself. The default workflow for anything visual: generate a slide, screenshot it, give feedback, screenshot again, give more feedback. Five rounds of you being the quality layer between Claude and the final output. A Puppeteer skill eliminates that loop completely. You embed a screenshot tool directly in the skill file. Claude renders HTML, captures its own output, measures whether text overflows the page dimensions, and iterates three times before you ever see it. Pixel-level self-correction with zero human review. One detail that stuck with me: a recent Google paper showed that literally running the same prompt twice improves output quality. The serious Claude Code builders are encoding that into every skill. Generate, re-validate against the original rules, generate again. Quality compounds automatically with each pass. This is the builder-validator pattern. The AI does the work, then the AI audits the work against the same spec. You stop functioning as the manual checkpoint. The setup cost is one skill file with Puppeteer embedded. The return is every visual task from that point forward shipping at production quality without a screenshot loop. Build the check into the system. Stop being the system.

The PMs screenshotting Claude Code outputs and saying "fix the alignment" five times are playing QA on a system that can QA itself. The default workflow for anything visual: generate a slide, screenshot it, give feedback, screenshot again, give more feedback. Five rounds of you being the quality layer between Claude and the final output. A Puppeteer skill eliminates that loop completely. You embed a screenshot tool directly in the skill file. Claude renders HTML, captures its own output, measures whether text overflows the page dimensions, and iterates three times before you ever see it. Pixel-level self-correction with zero human review. One detail that stuck with me: a recent Google paper showed that literally running the same prompt twice improves output quality. The serious Claude Code builders are encoding that into every skill. Generate, re-validate against the original rules, generate again. Quality compounds automatically with each pass. This is the builder-validator pattern. The AI does the work, then the AI audits the work against the same spec. You stop functioning as the manual checkpoint. The setup cost is one skill file with Puppeteer embedded. The return is every visual task from that point forward shipping at production quality without a screenshot loop. Build the check into the system. Stop being the system.

Aakash Gupta

61,435 views • 3 months ago

Elon Musk: Starship is achieving something that's physically barely possible. “We're pushing the limits of physics here. Starship is the first time that there is a design for a rocket where full and rapid reusability is actually possible. There's not even been a design before where it was possible. Certainly not a design that got made into any hardware at all. We live on a planet where the gravity is quite high. Earth's gravity is really quite high. And if the gravity were even 10% or 20% higher, we'd be stuck on Earth forever. We certainly couldn't use conventional rockets. You'd have to blow yourself off the surface with a nuclear bomb, or something crazy. So, on the other hand, if Earth's gravity was just a little lower, like even 10%, 20% lower, then getting to orbit would be easy. If this was a video game, it's set to maximum difficulty, but not impossible.” The Joe Rogan Experience, October 31, 2025

Elon Musk: Starship is achieving something that's physically barely possible. “We're pushing the limits of physics here. Starship is the first time that there is a design for a rocket where full and rapid reusability is actually possible. There's not even been a design before where it was possible. Certainly not a design that got made into any hardware at all. We live on a planet where the gravity is quite high. Earth's gravity is really quite high. And if the gravity were even 10% or 20% higher, we'd be stuck on Earth forever. We certainly couldn't use conventional rockets. You'd have to blow yourself off the surface with a nuclear bomb, or something crazy. So, on the other hand, if Earth's gravity was just a little lower, like even 10%, 20% lower, then getting to orbit would be easy. If this was a video game, it's set to maximum difficulty, but not impossible.” The Joe Rogan Experience, October 31, 2025

ELON CLIPS

47,064 views • 8 months ago

Hey Joe, I've been away from the space for some time, but it seems you still haven't fixed what are clearly development errors on your platform. Earlier this year (or possibly late 2025), I returned to Duelbits and was quite active on the platform. One day, while playing HiLo, I noticed something unusual. At first, I thought I was imagining it because I was deep into a gambling session, but considering I've wagered well over $5 billion on HiLo alone across various sites, I decided to pay closer attention. What was the issue? I discovered what appeared to be a critical UI error. The "Lower" option was displayed as having a 92% hit chance when, based on the card shown, it should have been approximately 15% (or whatever the correct probability was). Instead, the platform displayed "Lower" as the higher-probability outcome and "Higher" as the lower-probability outcome. Any experienced HiLo player makes decisions based on the displayed percentages rather than waiting to calculate the probabilities manually. If the percentages shown are incorrect, that has the potential to mislead players and affect their decisions. You might not believe me, but I started recording my screen to see if it would happen again. It did. I captured it on video, sent the evidence to you, and what happened next? Nothing. So I'd like to ask the public for their opinion. Below, I'll attach screenshots and recordings of the gameplay. I asked you to investigate the issue, fix the error, and compensate me because I believe this mistake cost me approximately $20,000. What's even more concerning is that, to this day, another issue still exists: if you have an active HiLo session and refresh the page, the round can disappear entirely. How is that considered normal behavior? And while we're discussing this, let's also address the RTP claims. You promote your house games as having a 99% RTP, yet in the video, a HiLo sequence starting on a 5 (bet High, then Low on a J, then High on a 2, followed by a cashout) resulted in a net payout of 1.79x. Performing the same sequence and achieving the same outcome on Stake pays 1.83x. Can you explain that discrepancy?

Hey Joe, I've been away from the space for some time, but it seems you still haven't fixed what are clearly development errors on your platform. Earlier this year (or possibly late 2025), I returned to Duelbits and was quite active on the platform. One day, while playing HiLo, I noticed something unusual. At first, I thought I was imagining it because I was deep into a gambling session, but considering I've wagered well over $5 billion on HiLo alone across various sites, I decided to pay closer attention. What was the issue? I discovered what appeared to be a critical UI error. The "Lower" option was displayed as having a 92% hit chance when, based on the card shown, it should have been approximately 15% (or whatever the correct probability was). Instead, the platform displayed "Lower" as the higher-probability outcome and "Higher" as the lower-probability outcome. Any experienced HiLo player makes decisions based on the displayed percentages rather than waiting to calculate the probabilities manually. If the percentages shown are incorrect, that has the potential to mislead players and affect their decisions. You might not believe me, but I started recording my screen to see if it would happen again. It did. I captured it on video, sent the evidence to you, and what happened next? Nothing. So I'd like to ask the public for their opinion. Below, I'll attach screenshots and recordings of the gameplay. I asked you to investigate the issue, fix the error, and compensate me because I believe this mistake cost me approximately $20,000. What's even more concerning is that, to this day, another issue still exists: if you have an active HiLo session and refresh the page, the round can disappear entirely. How is that considered normal behavior? And while we're discussing this, let's also address the RTP claims. You promote your house games as having a 99% RTP, yet in the video, a HiLo sequence starting on a 5 (bet High, then Low on a J, then High on a 2, followed by a cashout) resulted in a net payout of 1.79x. Performing the same sequence and achieving the same outcome on Stake pays 1.83x. Can you explain that discrepancy?

Scarface

17,310 views • 1 month ago

karpathy just broke the internet with something called auto research it’s basically an ai research agent that runs experiments for you 24/7 you give it a goal like “make this model better” “find a higher converting landing page” “lower customer acquisition cost” then it runs a loop: 1) plan an experiment 2) edit the code or config 3) run a short test on a gpu 4) read the metrics 5) keep the winner 6) try again over and over while you sleep by the morning you wake up to the best version actual tested improvements think of it like a robot research intern that runs hundreds of experiments and only keeps the winners this is link to his repo for your to mess around with it in the latest episode of The Startup Ideas Podcast (SIP) 🧃 i break down: • what auto research actually is • how it works step by step • 10 business ideas you can build with it • how to install it and start using it this one is saucy because tools like this change how startups get built watch

karpathy just broke the internet with something called auto research it’s basically an ai research agent that runs experiments for you 24/7 you give it a goal like “make this model better” “find a higher converting landing page” “lower customer acquisition cost” then it runs a loop: 1) plan an experiment 2) edit the code or config 3) run a short test on a gpu 4) read the metrics 5) keep the winner 6) try again over and over while you sleep by the morning you wake up to the best version actual tested improvements think of it like a robot research intern that runs hundreds of experiments and only keeps the winners this is link to his repo for your to mess around with it in the latest episode of The Startup Ideas Podcast (SIP) 🧃 i break down: • what auto research actually is • how it works step by step • 10 business ideas you can build with it • how to install it and start using it this one is saucy because tools like this change how startups get built watch

GREG ISENBERG

435,542 views • 4 months ago

Today, we introduce HunyuanImage 3.0-Instruct, a native multimodal model focusing on image-editing by integrating visual understanding with precise image synthesis! 🚀 It understands input images and reasons before generating images. Built on an 80B-parameter MoE architecture (13B activated), it natively unifies deep multimodal comprehension and high-fidelity generation. 🧠 A "Thinking" Model with Native CoT & MixGRPO: The model doesn’t just execute commands, it processes them through a Native Chain-of-Thought (CoT) schema. Enhanced by our self-developed MixGRPO algorithm, it reasons through complex instructions to achieve flawless intent alignment and human-preference consistency. 🎨 Precise Editing & Multi-Image Fusion: The model enables accurate image editing by adding, removing, or modifying elements while keeping non-target areas perfectly intact. It also excels at seamless multi-image fusion, synthesizing complex scenes by extracting and blending elements from multiple sources into a unified, consistent output. 🏆 SOTA Performance: HunyuanImage 3.0-Instruct sets a new benchmark in visual quality and alignment, delivering performance that matches leading proprietary models. We aim to enable the community to explore new ideas with a state-of-the-art foundation model, fostering a dynamic and vibrant image generation ecosystem. 🛠️🎨 💻Try it at (PC only):

Today, we introduce HunyuanImage 3.0-Instruct, a native multimodal model focusing on image-editing by integrating visual understanding with precise image synthesis! 🚀 It understands input images and reasons before generating images. Built on an 80B-parameter MoE architecture (13B activated), it natively unifies deep multimodal comprehension and high-fidelity generation. 🧠 A "Thinking" Model with Native CoT & MixGRPO: The model doesn’t just execute commands, it processes them through a Native Chain-of-Thought (CoT) schema. Enhanced by our self-developed MixGRPO algorithm, it reasons through complex instructions to achieve flawless intent alignment and human-preference consistency. 🎨 Precise Editing & Multi-Image Fusion: The model enables accurate image editing by adding, removing, or modifying elements while keeping non-target areas perfectly intact. It also excels at seamless multi-image fusion, synthesizing complex scenes by extracting and blending elements from multiple sources into a unified, consistent output. 🏆 SOTA Performance: HunyuanImage 3.0-Instruct sets a new benchmark in visual quality and alignment, delivering performance that matches leading proprietary models. We aim to enable the community to explore new ideas with a state-of-the-art foundation model, fostering a dynamic and vibrant image generation ecosystem. 🛠️🎨 💻Try it at (PC only):

Tencent Hy

125,803 views • 5 months ago

ELON: "WE'RE PUSHING THE LIMITS OF PHYSICS WITH STARSHIP" "Starship is achieving something that's physically barely possible. We're pushing the limits of physics here. Starship is the first time that there is a design for a rocket where full and rapid reusability is actually possible. There's not even been a design before where it was possible. Certainly not a design that got made into any hardware at all. We live on a planet where the gravity is quite high. Earth's gravity is really quite high. And if the gravity were even 10 percent or 20 percent higher, we'd be stuck on Earth forever. We certainly couldn't use conventional rockets. You'd have to blow yourself off the surface with a nuclear bomb, or something crazy. So, on the other hand, if Earth's gravity was just a little lower, like even 10 percent, 20 percent lower, then getting to orbit would be easy. If this was a video game, it's set to maximum difficulty, but not impossible." Source: Elon Musk, Joe Rogan, The Joe Rogan Experience, October 31, 2025

ELON: "WE'RE PUSHING THE LIMITS OF PHYSICS WITH STARSHIP" "Starship is achieving something that's physically barely possible. We're pushing the limits of physics here. Starship is the first time that there is a design for a rocket where full and rapid reusability is actually possible. There's not even been a design before where it was possible. Certainly not a design that got made into any hardware at all. We live on a planet where the gravity is quite high. Earth's gravity is really quite high. And if the gravity were even 10 percent or 20 percent higher, we'd be stuck on Earth forever. We certainly couldn't use conventional rockets. You'd have to blow yourself off the surface with a nuclear bomb, or something crazy. So, on the other hand, if Earth's gravity was just a little lower, like even 10 percent, 20 percent lower, then getting to orbit would be easy. If this was a video game, it's set to maximum difficulty, but not impossible." Source: Elon Musk, Joe Rogan, The Joe Rogan Experience, October 31, 2025

Mario Nawfal

119,763 views • 8 months ago

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering

We are excited to share our #CORL2024 paper on learning quadrotor obstacle avoidance from the visual stream of a single #eventcamera! Trained entirely in simulation! We demonstrate obstacle avoidance both in the dark and in a forest up to 5m/s. PDF: Video: Project page: Event cameras are sensors that output per-pixel-level intensity changes at microsecond latency resolution; they feature nearly zero motion blur and high dynamic range but produce a very large volume of events under significant ego-motion and further lack a high-fidelity continuous-time sensor model in simulation, making direct #sim2real transfer not possible. By leveraging depth prediction as a pretext task, we pre-train a reactive obstacle avoidance policy with “approximated” simulated events and then fine-tune the perception component with limited events-and-depth real-world data. This technique bridges the sim2real gap for #eventcameras! As at the current state, there is no continuous-time sensor model for event cameras, we hope that this work can finally spur future research leveraging simulation for training event-vision-based policies to create faster, agile robots! Kudos to Anish Bhattacharya, @marcocannic, Vijay Kumar Nikolai Matni UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) GRASP Laboratory Penn Engineering

Davide Scaramuzza

17,219 views • 1 year ago

PHOTON COUNTING CT is NOT a better CT It is a NEW imaging modality Photon Counting CT (PCCT) represents a transformative leap in medical imaging, not only as a molecular imaging modality but also as a technology offering ultra-high resolution and functional imaging capabilities. It is fundamentally more than just an enhanced version of traditional CT—PCCT introduces new ways of seeing and understanding the human body, providing critical insights at the molecular, structural, and functional levels. This positions PCCT as a unique imaging modality that requires a fresh approach to technical implementation, operational workflows, and financial planning. Despite the larger upfront investment, PCCT’s ability to drastically reduce downstream healthcare costs makes it a highly valuable investment in the long run. 1. Technical Innovations • Molecular Imaging and Energy Discrimination: Unlike traditional CT, which simply measures the total absorbed energy, PCCT counts individual X-ray photons and differentiates their energy levels. This allows for precise molecular imaging, revealing the composition of tissues and materials at a biochemical level. By distinguishing between different tissue types and contrast agents, PCCT opens up new diagnostic possibilities, such as identifying molecular biomarkers in tumors or distinguishing between stable and unstable plaque in coronary arteries. This capability shifts the focus of imaging from purely anatomical to both anatomical and molecular, offering more comprehensive diagnostic information. • Ultra-High Spatial Resolution: PCCT features significantly smaller detector elements compared to conventional CT scanners, allowing for ultra-high resolution imaging. This means clinicians can visualize fine structures such as microcalcifications in arteries, small lesions in soft tissues, or the intricate architecture of bones. This level of detail was previously unattainable with traditional CT. When combined with molecular imaging, this ultra-high resolution allows for the precise localization and characterization of disease at very early stages, which is essential for early diagnosis and intervention. • Functional Imaging Capabilities: PCCT also excels as a functional imaging modality. By capturing energy-resolved information, PCCT can provide insights into tissue functionality and dynamic physiological processes. For instance, it can detect changes in blood flow, tissue perfusion, and oxygenation without the need for additional contrast agents or scans. This functionality allows for real-time assessment of physiological processes, making it particularly valuable in cardiology, oncology, and neurology for evaluating organ function and monitoring disease progression. • Reduced Noise and Artifact Reduction: Photon-counting technology dramatically reduces electronic noise and imaging artifacts, such as beam hardening, resulting in clearer and more accurate images. The ability to deliver ultra-high resolution images with minimal artifacts improves diagnostic accuracy, reducing the need for repeat scans and ensuring that even subtle abnormalities are detected. 2. Operational Considerations • New Workflow for Molecular, High-Resolution, and Functional Imaging: The integration of molecular, ultra-high resolution, and functional imaging into routine clinical workflows introduces complexity that requires adaptation. Radiologists and technicians need specialized training to interpret and analyze multi-energy datasets that include molecular and functional information. PCCT produces a vast amount of detailed data, requiring clinicians to adopt new imaging protocols and refine their diagnostic approaches to fully leverage its capabilities. • Post-Processing and Data Management: PCCT generates richer, more complex datasets, which necessitates advanced post-processing tools and data management systems. Existing PACS and imaging software may not be equipped to handle such large volumes of data or to process functional and molecular information effectively. This means healthcare institutions must invest in robust IT infrastructure, including upgraded software and storage solutions, as well as provide additional training for staff on new imaging analysis techniques. • Revised Clinical Protocols: The molecular, functional, and ultra-high resolution imaging capabilities of PCCT will likely prompt changes in clinical protocols. For instance, the need for contrast agents may be reduced, simplifying patient preparation and decreasing the risk of adverse reactions. Additionally, the ability to monitor physiological functions in real-time through functional imaging could lead to more dynamic diagnostic procedures, such as assessing the effectiveness of interventions or treatments in real-time. 3. Financial Impact • Higher Initial Investment: PCCT systems are more expensive than traditional CT scanners due to their advanced technology, which includes photon-counting detectors and the computational power required for high-resolution, molecular, and functional imaging. While this upfront cost is significant, it is crucial to view it in the broader context of the downstream benefits and cost reductions that PCCT offers. • Downstream Cost Reductions: Although the initial capital investment is higher, PCCT’s ability to combine molecular, functional, and ultra-high resolution imaging leads to substantial reductions in downstream healthcare costs. Its superior diagnostic accuracy minimizes the need for follow-up tests, repeat scans, or invasive diagnostic procedures, such as diagnostic coronary angiographies. For example, in cardiology, PCCT can precisely differentiate between types of coronary plaque, reducing the need for invasive procedures to assess risk. • Lower Overall Healthcare Expenditures: By enabling earlier, more accurate diagnoses, PCCT can reduce the overall cost of patient care. Early detection of disease, particularly through its molecular and functional imaging capabilities, allows for more targeted treatments, potentially preventing the need for more aggressive and expensive interventions down the line. For instance, early-stage tumor detection via molecular imaging could lead to less invasive treatments, reducing hospital stays and improving patient outcomes, ultimately driving down healthcare costs. • Increased ROI Through Enhanced Patient Outcomes: Over time, the combination of molecular, functional, and ultra-high resolution imaging enhances diagnostic precision, which translates into better patient outcomes. Improved diagnostic accuracy reduces the incidence of unnecessary procedures, minimizes treatment delays, and results in more personalized and effective care. This leads to increased patient satisfaction, better healthcare outcomes, and greater patient throughput—all factors that improve the institution’s return on investment (ROI). • Competitive Advantage and New Revenue Streams: By adopting PCCT, healthcare institutions position themselves at the forefront of advanced imaging technologies. The ability to offer molecular, functional, and ultra-high resolution imaging creates a competitive advantage, attracting more complex and high-value cases. This can boost the institution’s reputation for excellence in diagnostics, leading to increased referrals, new patient populations, and expanded revenue opportunities. Summary Photon Counting CT (PCCT) is not just an evolution of existing CT technology—it is a molecular, ultra-high resolution, and functional imaging modality that fundamentally transforms the diagnostic landscape. Its ability to capture detailed molecular data, visualize minute anatomical structures with ultra-high resolution, and provide real-time functional imaging opens new possibilities for earlier and more precise diagnoses. While the financial investment in PCCT is larger, the reduction in downstream healthcare costs through improved diagnostic accuracy, fewer unnecessary interventions, and earlier disease detection far outweighs the initial expense. For institutions committed to advancing patient care and improving long-term financial outcomes, PCCT is an essential investment in the future of medical imaging. The video attached shows a patient accessing the Hospital for ACS. PCCT can provide ALL the imaging information of the concurrent imaging modalities (CXR, CAG, Echo, CMR) that you see around it... that's a lot! #PhotonCountingCT #MolecularImaging #UltraHighResolution #FunctionalImaging #FutureOfImaging #AdvancedMedicalImaging #EarlyDiseaseDetection #InnovativeCT #CuttingEdgeHealthcare #PrecisionDiagnostics #HealthcareInnovation #MedicalTechnology #CostEffectiveImaging #NextGenCT #PatientCareRevolution

PHOTON COUNTING CT is NOT a better CT It is a NEW imaging modality Photon Counting CT (PCCT) represents a transformative leap in medical imaging, not only as a molecular imaging modality but also as a technology offering ultra-high resolution and functional imaging capabilities. It is fundamentally more than just an enhanced version of traditional CT—PCCT introduces new ways of seeing and understanding the human body, providing critical insights at the molecular, structural, and functional levels. This positions PCCT as a unique imaging modality that requires a fresh approach to technical implementation, operational workflows, and financial planning. Despite the larger upfront investment, PCCT’s ability to drastically reduce downstream healthcare costs makes it a highly valuable investment in the long run. 1. Technical Innovations • Molecular Imaging and Energy Discrimination: Unlike traditional CT, which simply measures the total absorbed energy, PCCT counts individual X-ray photons and differentiates their energy levels. This allows for precise molecular imaging, revealing the composition of tissues and materials at a biochemical level. By distinguishing between different tissue types and contrast agents, PCCT opens up new diagnostic possibilities, such as identifying molecular biomarkers in tumors or distinguishing between stable and unstable plaque in coronary arteries. This capability shifts the focus of imaging from purely anatomical to both anatomical and molecular, offering more comprehensive diagnostic information. • Ultra-High Spatial Resolution: PCCT features significantly smaller detector elements compared to conventional CT scanners, allowing for ultra-high resolution imaging. This means clinicians can visualize fine structures such as microcalcifications in arteries, small lesions in soft tissues, or the intricate architecture of bones. This level of detail was previously unattainable with traditional CT. When combined with molecular imaging, this ultra-high resolution allows for the precise localization and characterization of disease at very early stages, which is essential for early diagnosis and intervention. • Functional Imaging Capabilities: PCCT also excels as a functional imaging modality. By capturing energy-resolved information, PCCT can provide insights into tissue functionality and dynamic physiological processes. For instance, it can detect changes in blood flow, tissue perfusion, and oxygenation without the need for additional contrast agents or scans. This functionality allows for real-time assessment of physiological processes, making it particularly valuable in cardiology, oncology, and neurology for evaluating organ function and monitoring disease progression. • Reduced Noise and Artifact Reduction: Photon-counting technology dramatically reduces electronic noise and imaging artifacts, such as beam hardening, resulting in clearer and more accurate images. The ability to deliver ultra-high resolution images with minimal artifacts improves diagnostic accuracy, reducing the need for repeat scans and ensuring that even subtle abnormalities are detected. 2. Operational Considerations • New Workflow for Molecular, High-Resolution, and Functional Imaging: The integration of molecular, ultra-high resolution, and functional imaging into routine clinical workflows introduces complexity that requires adaptation. Radiologists and technicians need specialized training to interpret and analyze multi-energy datasets that include molecular and functional information. PCCT produces a vast amount of detailed data, requiring clinicians to adopt new imaging protocols and refine their diagnostic approaches to fully leverage its capabilities. • Post-Processing and Data Management: PCCT generates richer, more complex datasets, which necessitates advanced post-processing tools and data management systems. Existing PACS and imaging software may not be equipped to handle such large volumes of data or to process functional and molecular information effectively. This means healthcare institutions must invest in robust IT infrastructure, including upgraded software and storage solutions, as well as provide additional training for staff on new imaging analysis techniques. • Revised Clinical Protocols: The molecular, functional, and ultra-high resolution imaging capabilities of PCCT will likely prompt changes in clinical protocols. For instance, the need for contrast agents may be reduced, simplifying patient preparation and decreasing the risk of adverse reactions. Additionally, the ability to monitor physiological functions in real-time through functional imaging could lead to more dynamic diagnostic procedures, such as assessing the effectiveness of interventions or treatments in real-time. 3. Financial Impact • Higher Initial Investment: PCCT systems are more expensive than traditional CT scanners due to their advanced technology, which includes photon-counting detectors and the computational power required for high-resolution, molecular, and functional imaging. While this upfront cost is significant, it is crucial to view it in the broader context of the downstream benefits and cost reductions that PCCT offers. • Downstream Cost Reductions: Although the initial capital investment is higher, PCCT’s ability to combine molecular, functional, and ultra-high resolution imaging leads to substantial reductions in downstream healthcare costs. Its superior diagnostic accuracy minimizes the need for follow-up tests, repeat scans, or invasive diagnostic procedures, such as diagnostic coronary angiographies. For example, in cardiology, PCCT can precisely differentiate between types of coronary plaque, reducing the need for invasive procedures to assess risk. • Lower Overall Healthcare Expenditures: By enabling earlier, more accurate diagnoses, PCCT can reduce the overall cost of patient care. Early detection of disease, particularly through its molecular and functional imaging capabilities, allows for more targeted treatments, potentially preventing the need for more aggressive and expensive interventions down the line. For instance, early-stage tumor detection via molecular imaging could lead to less invasive treatments, reducing hospital stays and improving patient outcomes, ultimately driving down healthcare costs. • Increased ROI Through Enhanced Patient Outcomes: Over time, the combination of molecular, functional, and ultra-high resolution imaging enhances diagnostic precision, which translates into better patient outcomes. Improved diagnostic accuracy reduces the incidence of unnecessary procedures, minimizes treatment delays, and results in more personalized and effective care. This leads to increased patient satisfaction, better healthcare outcomes, and greater patient throughput—all factors that improve the institution’s return on investment (ROI). • Competitive Advantage and New Revenue Streams: By adopting PCCT, healthcare institutions position themselves at the forefront of advanced imaging technologies. The ability to offer molecular, functional, and ultra-high resolution imaging creates a competitive advantage, attracting more complex and high-value cases. This can boost the institution’s reputation for excellence in diagnostics, leading to increased referrals, new patient populations, and expanded revenue opportunities. Summary Photon Counting CT (PCCT) is not just an evolution of existing CT technology—it is a molecular, ultra-high resolution, and functional imaging modality that fundamentally transforms the diagnostic landscape. Its ability to capture detailed molecular data, visualize minute anatomical structures with ultra-high resolution, and provide real-time functional imaging opens new possibilities for earlier and more precise diagnoses. While the financial investment in PCCT is larger, the reduction in downstream healthcare costs through improved diagnostic accuracy, fewer unnecessary interventions, and earlier disease detection far outweighs the initial expense. For institutions committed to advancing patient care and improving long-term financial outcomes, PCCT is an essential investment in the future of medical imaging. The video attached shows a patient accessing the Hospital for ACS. PCCT can provide ALL the imaging information of the concurrent imaging modalities (CXR, CAG, Echo, CMR) that you see around it... that's a lot! #PhotonCountingCT #MolecularImaging #UltraHighResolution #FunctionalImaging #FutureOfImaging #AdvancedMedicalImaging #EarlyDiseaseDetection #InnovativeCT #CuttingEdgeHealthcare #PrecisionDiagnostics #HealthcareInnovation #MedicalTechnology #CostEffectiveImaging #NextGenCT #PatientCareRevolution

Dr. Filippo Cademartiri

11,820 views • 1 year ago

This BlenderFusion paper basically says "screw trying to describe 3D edits through text" and just... use Blender :-) The idea is pretty straightforward -- instead of trying to cram 3D understanding into a diffusion model, use depth estimation & segmentation to project 2D images into 2.5D meshes, edit them in actual 3D software, then use a fine-tuned diffusion model to make the results photorealistic again. The clever bit is their "dual-stream architecture" -- the model sees both the original scene AND the edited Blender render in parallel, learning to preserve what matters while fixing the inevitable artifacts from transforming imperfect 2.5D/3D reconstructions. They train it with smart masking strategies so it learns when to ignore the original scene (for removals/replacements) and can manipulate objects independently of camera motion. What you get is pretty impressive control -- not just moving objects around, but changing materials, deforming shapes, swapping backgrounds, all while maintaining visual coherence. Neural Assets (one of my favorite papers last year) tried to crack this with learned object tokens, but it struggled with overlapping objects and loses fine details (due to low res DINO encodings). BlenderFusion just sidesteps the whole problem -- want to rotate something 173.5 degrees? Just rotate it in Blender. Want to duplicate an object 8 times? Copy paste away. The diffusion model's only job is making it look photorealistic, not figuring out the 3D underpinnings. The catch? Lacks temporal consistency for animation. Each viewpoint is generated independently, so while a single edit looks great, smoothly animating a car or camera down the street won't work -- you'd get flickering and inconsistencies between frames. That said, this approach is so much more intuitive for finer grain image editing than trying to describe your changes in text prompts. It's the kind of thing that makes you wonder why we're trying to do everything inside neural networks when perfectly good 3D tools already exist -- giving you the best of both worlds.

This BlenderFusion paper basically says "screw trying to describe 3D edits through text" and just... use Blender :-) The idea is pretty straightforward -- instead of trying to cram 3D understanding into a diffusion model, use depth estimation & segmentation to project 2D images into 2.5D meshes, edit them in actual 3D software, then use a fine-tuned diffusion model to make the results photorealistic again. The clever bit is their "dual-stream architecture" -- the model sees both the original scene AND the edited Blender render in parallel, learning to preserve what matters while fixing the inevitable artifacts from transforming imperfect 2.5D/3D reconstructions. They train it with smart masking strategies so it learns when to ignore the original scene (for removals/replacements) and can manipulate objects independently of camera motion. What you get is pretty impressive control -- not just moving objects around, but changing materials, deforming shapes, swapping backgrounds, all while maintaining visual coherence. Neural Assets (one of my favorite papers last year) tried to crack this with learned object tokens, but it struggled with overlapping objects and loses fine details (due to low res DINO encodings). BlenderFusion just sidesteps the whole problem -- want to rotate something 173.5 degrees? Just rotate it in Blender. Want to duplicate an object 8 times? Copy paste away. The diffusion model's only job is making it look photorealistic, not figuring out the 3D underpinnings. The catch? Lacks temporal consistency for animation. Each viewpoint is generated independently, so while a single edit looks great, smoothly animating a car or camera down the street won't work -- you'd get flickering and inconsistencies between frames. That said, this approach is so much more intuitive for finer grain image editing than trying to describe your changes in text prompts. It's the kind of thing that makes you wonder why we're trying to do everything inside neural networks when perfectly good 3D tools already exist -- giving you the best of both worlds.

Bilawal Sidhu

34,440 views • 1 year ago

🚨 THIS IS NOT NORMAL S&P 500 just hit a new all-time high at $7,050. All while the worst crisis in decades is still unfolding in real time. Markets are surging through an inflation shock like nothing is wrong. This pump is NOT bullish - it’s manipulated. Because this isn’t how real markets behave. But almost nobody is willing to say what’s actually happening. The market is acting like the crisis is finished. It isn’t. Oil is still above $90. The pressure hasn’t gone anywhere. Yet equities keep climbing like the system isn’t under stress. And while the real economy is getting squeezed... Stocks are being pushed higher to manufacture exit liquidity for smart money. This widening gap between price and reality is not just unusual. It’s DANGEROUS. I’ve seen this setup before: When markets rally into an energy shock instead of reacting to it, the unwind doesn’t drift lower… …it collapses straight down. Vertical. And fast. This kind of price action during a crisis isn’t strength. It’s distortion. And distortion doesn’t resolve gently. Remember, I’ve called every major market top and bottom for the past decade. Turn on notifications. When it's time to sell, I’ll call it publicly here. Don’t become the exit liquidity.

🚨 THIS IS NOT NORMAL S&P 500 just hit a new all-time high at $7,050. All while the worst crisis in decades is still unfolding in real time. Markets are surging through an inflation shock like nothing is wrong. This pump is NOT bullish - it’s manipulated. Because this isn’t how real markets behave. But almost nobody is willing to say what’s actually happening. The market is acting like the crisis is finished. It isn’t. Oil is still above $90. The pressure hasn’t gone anywhere. Yet equities keep climbing like the system isn’t under stress. And while the real economy is getting squeezed... Stocks are being pushed higher to manufacture exit liquidity for smart money. This widening gap between price and reality is not just unusual. It’s DANGEROUS. I’ve seen this setup before: When markets rally into an energy shock instead of reacting to it, the unwind doesn’t drift lower… …it collapses straight down. Vertical. And fast. This kind of price action during a crisis isn’t strength. It’s distortion. And distortion doesn’t resolve gently. Remember, I’ve called every major market top and bottom for the past decade. Turn on notifications. When it's time to sell, I’ll call it publicly here. Don’t become the exit liquidity.

0xNobler

124,355 views • 3 months ago

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Rohan Paul

10,996 views • 8 days ago

🔴 Finally! NVIDIA has finally made the code for Neuralangelo public! It has the ability to transform any video into a highly detailed 3D environment, and it's a technology related to but DIFFERENT from NeRF. 💡 Here's how it works: It takes a 2D video as input, showing an object, monument, building, landscape, etc., from various perspectives and analyzes details such as depth, size, and the shapes of objects. From this, the AI sketches an initial 3D model, similar to how an artist molds a figure. This representation is then refined to highlight more details, just as an artist would make the final touches when sculpting. The result is a 3D environment/model, perfect for use in any environment. Imagine the applications it will have for video games, cinema, virtual environments, VR, and more! 📽️🎮 💡 More details: A year ago, an article was presented on a groundbreaking technique called NVIDIA's Instant NeRF. This technique turns images into stunning 3D scenes in a short time, ideal for creating realistic models for video games and other applications. Although Instant NeRF had a lot of potential, the generated models were not perfect and often lacked detailed structures, appearing somewhat cartoonish. A year on, NVIDIA releases a new technique based on Instant NeRF, named Neuralangelo. This enhances the fidelity of surface structures. While NeRF reconstructs real objects in virtual environments from images or videos, Instant NeRF speeds up this process, and Neuralangelo further improves the quality, making the generated objects appear even more realistic when examined up close. Neuralangelo improves Instant NeRF's approach in two key ways related to the hash grid encoding technique: 1⃣ Numerical gradients have been used to compute higher-order derivatives as a smoothing operation. This optimizes the "hash grid" encoding using numerical rather than analytical gradients, providing a smoother input to the network that produces the 3D model. 2⃣ A "coarse-to-fine" optimization has been implemented in the hash grids to control different levels of detail. That is, they first focus on a smoothed version of the scene, and then refine it with more detailed updates. Well, as Arthur C. Clarke said, "Any sufficiently advanced technology is indistinguishable from magic."

🔴 Finally! NVIDIA has finally made the code for Neuralangelo public! It has the ability to transform any video into a highly detailed 3D environment, and it's a technology related to but DIFFERENT from NeRF. 💡 Here's how it works: It takes a 2D video as input, showing an object, monument, building, landscape, etc., from various perspectives and analyzes details such as depth, size, and the shapes of objects. From this, the AI sketches an initial 3D model, similar to how an artist molds a figure. This representation is then refined to highlight more details, just as an artist would make the final touches when sculpting. The result is a 3D environment/model, perfect for use in any environment. Imagine the applications it will have for video games, cinema, virtual environments, VR, and more! 📽️🎮 💡 More details: A year ago, an article was presented on a groundbreaking technique called NVIDIA's Instant NeRF. This technique turns images into stunning 3D scenes in a short time, ideal for creating realistic models for video games and other applications. Although Instant NeRF had a lot of potential, the generated models were not perfect and often lacked detailed structures, appearing somewhat cartoonish. A year on, NVIDIA releases a new technique based on Instant NeRF, named Neuralangelo. This enhances the fidelity of surface structures. While NeRF reconstructs real objects in virtual environments from images or videos, Instant NeRF speeds up this process, and Neuralangelo further improves the quality, making the generated objects appear even more realistic when examined up close. Neuralangelo improves Instant NeRF's approach in two key ways related to the hash grid encoding technique: 1⃣ Numerical gradients have been used to compute higher-order derivatives as a smoothing operation. This optimizes the "hash grid" encoding using numerical rather than analytical gradients, providing a smoother input to the network that produces the 3D model. 2⃣ A "coarse-to-fine" optimization has been implemented in the hash grids to control different levels of detail. That is, they first focus on a smoothed version of the scene, and then refine it with more detailed updates. Well, as Arthur C. Clarke said, "Any sufficiently advanced technology is indistinguishable from magic."

Javi Lopez ⛩️

689,128 views • 2 years ago