Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)

ShuangL13799063's profile picture

Shuang Li

7,675 subscribers

67,506 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The Sora 2 feed is fascinating, seeing how everyone is using it for the first time. This generation stood out a mile from all the cameos and pokemon, by jesperagain: > 1960s black-and-white BBC report on Sora 2 video generation model launch. Grainy video to match the time
0:15

The Sora 2 feed is fascinating, seeing how everyone is using it for the first time. This generation stood out a mile from all the cameos and pokemon, by jesperagain: > 1960s black-and-white BBC report on Sora 2 video generation model launch. Grainy video to match the time

fofr

283,743 Aufrufe • vor 10 Monaten

this is not a drill 🚨, real-time open source video generation is here 🔥 Self-Forcing - a real-time video distilled model from Wan 2.1 by Adobe is out, and they open sourced it 🐐 I've built a live real time demo on Hugging Face Spaces 📹💨
0:12

this is not a drill 🚨, real-time open source video generation is here 🔥 Self-Forcing - a real-time video distilled model from Wan 2.1 by Adobe is out, and they open sourced it 🐐 I've built a live real time demo on Hugging Face Spaces 📹💨

apolinario 🌐

52,153 Aufrufe • vor 1 Jahr

Real-time video generation is officially here 👀 Krea AI just opened the waitlist for their new real-time video engine 😮 👉 What they’re promising: 1️⃣ 12+ FPS generation: True real-time 2️⃣ Multi-input control: Use text prompts, images, your screen, or even your webcam as a source 3️⃣ “World Model” Engine: “systems that learn how scenes evolve and how actions ripple forward in time” (c) A huge step for real-time storytelling and interactive experiences is here Take a look yourself 👇 (Video credit: Krea AI)
0:17

Real-time video generation is officially here 👀 Krea AI just opened the waitlist for their new real-time video engine 😮 👉 What they’re promising: 1️⃣ 12+ FPS generation: True real-time 2️⃣ Multi-input control: Use text prompts, images, your screen, or even your webcam as a source 3️⃣ “World Model” Engine: “systems that learn how scenes evolve and how actions ripple forward in time” (c) A huge step for real-time storytelling and interactive experiences is here Take a look yourself 👇 (Video credit: Krea AI)

AmirMušić

17,328 Aufrufe • vor 11 Monaten

🔥Spatial intelligence requires world generation, and now we have the first comprehensive evaluation benchmark📏 for it! Introducing WorldScore: Unifying evaluation for 3D, 4D, and video models on world generation! 🧵1/7 Web: arxiv:
0:14

🔥Spatial intelligence requires world generation, and now we have the first comprehensive evaluation benchmark📏 for it! Introducing WorldScore: Unifying evaluation for 3D, 4D, and video models on world generation! 🧵1/7 Web: arxiv:

Hong-Xing (Koven) Yu

66,876 Aufrufe • vor 1 Jahr

🏆 Cosmos 3 just topped 7 physical AI leaderboards. NVIDIA Cosmos™ 3, the open omni-model for physical AI, ranks #1 across world generation, robot action policy, and industrial vision understanding. 🌎 World generation: Artificial Analysis, PAI-Bench, Physics-IQ, R-Bench 🤖 Robot policy: RoboLab 👁️ Vision: VANTAGE-Bench, TAR One model. Every domain of physical AI. Available now on Hugging Face ▶️
0:14

🏆 Cosmos 3 just topped 7 physical AI leaderboards. NVIDIA Cosmos™ 3, the open omni-model for physical AI, ranks #1 across world generation, robot action policy, and industrial vision understanding. 🌎 World generation: Artificial Analysis, PAI-Bench, Physics-IQ, R-Bench 🤖 Robot policy: RoboLab 👁️ Vision: VANTAGE-Bench, TAR One model. Every domain of physical AI. Available now on Hugging Face ▶️

NVIDIA

58,848 Aufrufe • vor 2 Monaten

Seedance 2.0 is going to be a game changer for video creation. The powerful video generation tool empowers creativity and drives efficiency. Built for social content, product ads, and marketing at scale. Get 60% OFF on Pollo AI! Time is running out. Don't miss it!
0:14

Seedance 2.0 is going to be a game changer for video creation. The powerful video generation tool empowers creativity and drives efficiency. Built for social content, product ads, and marketing at scale. Get 60% OFF on Pollo AI! Time is running out. Don't miss it!

Pollo AI

16,533,552 Aufrufe • vor 4 Monaten

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction
0:17

🤖 NVIDIA’s Gr00t N1.5 is now available in LeRobot! This is the result of a great collaboration between the Hugging Face LeRobot team and NVIDIA Robotics ! Gr00t N1.5 highlights: 🦾 Cross-embodiment foundation model for robots 🧠 Multimodal inputs: vision, language, and proprioception 🪛Tested on the Libero benchmark and real-world hardware tasks 🌍Trained on real robot, synthetic, and internet-scale video data ⚙️ Flow matching action transformer for action prediction

LeRobot

115,194 Aufrufe • vor 9 Monaten

We are rolling out access to Runway Aleph, our new in-context video model, setting a new frontier for multi-task visual generation. Aleph has the ability to perform a wide range of edits and generations on existing videos. Here are a few of the things Aleph is capable of doing. (1/7)
0:19

We are rolling out access to Runway Aleph, our new in-context video model, setting a new frontier for multi-task visual generation. Aleph has the ability to perform a wide range of edits and generations on existing videos. Here are a few of the things Aleph is capable of doing. (1/7)

Runway

99,594 Aufrufe • vor 1 Jahr

ICYM ⚠️ Check out the live and upcoming features on our 12th Development Update video! ⚙️ You can also find the transcript to this video here:
0:17

ICYM ⚠️ Check out the live and upcoming features on our 12th Development Update video! ⚙️ You can also find the transcript to this video here:

Mount & Blade

14,621 Aufrufe • vor 3 Jahren

introducing FLUX 3. beyond video, this is the first model to support interactions with the real world through a new action-prediction system. available now on Krea, more modalities coming soon.
0:19

introducing FLUX 3. beyond video, this is the first model to support interactions with the real world through a new action-prediction system. available now on Krea, more modalities coming soon.

Krea

31,550 Aufrufe • vor 11 Tagen

Cool new work on real-time video generation with motion control. MotionStream enables sub-second latency streaming on a single GPU. You can also control what parts of an image do & don't move, like the video below. 👏 to Xun Huang, Joonghyuk Shin, and team!
0:16

Cool new work on real-time video generation with motion control. MotionStream enables sub-second latency streaming on a single GPU. You can also control what parts of an image do & don't move, like the video below. 👏 to Xun Huang, Joonghyuk Shin, and team!

Justine Moore

21,780 Aufrufe • vor 9 Monaten

REAL OR FAKE? I'm 50/50 but I will say this; The appearance of the Being in this video definitely fits both; ETE-1 & ETE-3 (which are both biomechanical). Interesting.. Video has been out for a while. It's been called both Real & Fake. LMK what you think:
0:11

REAL OR FAKE? I'm 50/50 but I will say this; The appearance of the Being in this video definitely fits both; ETE-1 & ETE-3 (which are both biomechanical). Interesting.. Video has been out for a while. It's been called both Real & Fake. LMK what you think:

WHISTLEBLOWER

64,788 Aufrufe • vor 1 Monat

. Runway just added a new feature that you may not have noticed and we wanted to compare. You can now upscale your generation to 4K! Our favorite has typically been Topaz Labs, so we wanted to compare the out of both. Here is the original video from Runway
0:10

. Runway just added a new feature that you may not have noticed and we wanted to compare. You can now upscale your generation to 4K! Our favorite has typically been Topaz Labs, so we wanted to compare the out of both. Here is the original video from Runway

Curious Refuge

33,308 Aufrufe • vor 1 Jahr

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -
0:15

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 Aufrufe • vor 1 Jahr

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang
0:11

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang

LeRobot

24,727 Aufrufe • vor 2 Monaten

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6
0:10

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6

Hong-Xing (Koven) Yu

30,070 Aufrufe • vor 5 Monaten

We’re amazed: AMD now beats Nvidia in inference for Image-to-Video generation. 20% faster and 35% cheaper when using our Higgsfield DoP model. AMD MI300X also has a very easy setup. Details and tests are below. 🧩 1/8
0:15

We’re amazed: AMD now beats Nvidia in inference for Image-to-Video generation. 20% faster and 35% cheaper when using our Higgsfield DoP model. AMD MI300X also has a very easy setup. Details and tests are below. 🧩 1/8

Higgsfield AI 🧩

841,949 Aufrufe • vor 1 Jahr

💡HunyuanVideo1.5 Update: We are now releasing the 480p I2V step-distilled model, which generates videos in 8 or 12 steps (recommended)! On RTX 4090, end-to-end generation time is reduced by 75%, and a single RTX 4090 can generate videos within 75 seconds. The step-distilled model maintains comparable quality to the original model while achieving significant speedup. For even faster generation, you can also try 4 steps (faster speed with slightly reduced quality). 🔗Check out the GitHub Repo:
0:14

💡HunyuanVideo1.5 Update: We are now releasing the 480p I2V step-distilled model, which generates videos in 8 or 12 steps (recommended)! On RTX 4090, end-to-end generation time is reduced by 75%, and a single RTX 4090 can generate videos within 75 seconds. The step-distilled model maintains comparable quality to the original model while achieving significant speedup. For even faster generation, you can also try 4 steps (faster speed with slightly reduced quality). 🔗Check out the GitHub Repo:

Tencent Hy

39,011 Aufrufe • vor 8 Monaten

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 Aufrufe • vor 1 Jahr

Dive into the action anytime—we’re on 24/7! 🚨Check out our step-by-step guide on monetizing your gameplay👉🏻 Let’s level up together! Video from the stream by Velrooks
0:11

Dive into the action anytime—we’re on 24/7! 🚨Check out our step-by-step guide on monetizing your gameplay👉🏻 Let’s level up together! Video from the stream by Velrooks

Exylos

37,037 Aufrufe • vor 1 Jahr