Загрузка видео...

Не удалось загрузить видео

На главную

Introducing MotionForesight: A very simple approach re-purposing video models for forecasting future 3D scene flow given a short observed video context Trained with just 40k RGB human videos, it generalizes well to completely new home/office scenes! 1/n

mangahomanga's profile picture

Homanga Bharadhwaj

3,668 subscribers

16,653 просмотров • 4 месяцев назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]
0:15

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]

Homanga Bharadhwaj

21,118 просмотров • 10 месяцев назад

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n
0:12

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n

Chen Geng

43,596 просмотров • 10 месяцев назад

The context size of video world models is only a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3
0:12

The context size of video world models is only a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3

Gordon Wetzstein

35,137 просмотров • 1 год назад

Introducing “FlowCam: Training Generalizable 3D Radiance Fields w/o Camera Poses via Pixel-Aligned Scene Flow”! We train a generalizable 3D scene representation self-supervised on datasets of raw videos, without any pre-computed camera poses or SFM! 1/n
0:12

Introducing “FlowCam: Training Generalizable 3D Radiance Fields w/o Camera Poses via Pixel-Aligned Scene Flow”! We train a generalizable 3D scene representation self-supervised on datasets of raw videos, without any pre-computed camera poses or SFM! 1/n

Vincent Sitzmann

88,640 просмотров • 3 лет назад

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses
0:13

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses

Hao AI Lab

79,140 просмотров • 1 год назад

📢📢📢 Check out XCube, a generative model for very-high resolution 3D shapes and scenes! Project Page: Project Video: Props to Xuanchi Ren for leading this project!!!! [1/N]
0:14

📢📢📢 Check out XCube, a generative model for very-high resolution 3D shapes and scenes! Project Page: Project Video: Props to Xuanchi Ren for leading this project!!!! [1/N]

Francis Williams

42,417 просмотров • 2 лет назад

Just observed that Gemini 3.7 Flash in Antigravity can now analyze videos natively.. strange that it wasn't explicitly mentioned.. Anyways, a very great addition.. earlier the model had to use tools like ffmpeg or py scripts to extract frames and view individual images do build an context about the given video.. Now it's just so clean...
0:19

Just observed that Gemini 3.7 Flash in Antigravity can now analyze videos natively.. strange that it wasn't explicitly mentioned.. Anyways, a very great addition.. earlier the model had to use tools like ffmpeg or py scripts to extract frames and view individual images do build an context about the given video.. Now it's just so clean...

Bee

20,230 просмотров • 2 месяцев назад

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]
0:16

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]

Xuanchi Ren

26,638 просмотров • 1 год назад

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 просмотров • 2 лет назад

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 просмотров • 1 год назад

Short video about the light falloff in this scene i learned it from "Jose Vega" on youtube, It's a simple trick :D #b3d #environmentart #blender #3d #stylizedrender
0:14

Short video about the light falloff in this scene i learned it from "Jose Vega" on youtube, It's a simple trick :D #b3d #environmentart #blender #3d #stylizedrender

Ramy

63,355 просмотров • 5 месяцев назад

MiniMax H3 is setting a new standard for AI video editing. Ranked #1 in multiple video editing benchmarks, it can transform existing footage with just a reference video and a prompt. Change text, backgrounds, outfits or create a completely new visual direction, with incredible consistency.
0:10

MiniMax H3 is setting a new standard for AI video editing. Ranked #1 in multiple video editing benchmarks, it can transform existing footage with just a reference video and a prompt. Change text, backgrounds, outfits or create a completely new visual direction, with incredible consistency.

KATE

482,687 просмотров • 3 месяцев назад

my new AI filmmaking experiment: skipping video models entirely and building the whole scene in 3D with Mirage why run up your credits spamming seedance on higgsfield when you can just create a scene with tesseract that has editable geometry, models, textures, lighting lol
0:13

my new AI filmmaking experiment: skipping video models entirely and building the whole scene in 3D with Mirage why run up your credits spamming seedance on higgsfield when you can just create a scene with tesseract that has editable geometry, models, textures, lighting lol

Julian

23,455 просмотров • 9 дней назад

I am blown away 🤯. Check this out! CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains dynamic capabilities, accurate camera control, and scene consistency throughout the extended exploration." "Our model enables precise camera control across diverse scenarios while preserving dynamic scene elements, e.g." "Our method can generate videos with strong 3D consistency, which enables high-quality 3D reconstruction using the camera-controlled videos." Contributions: 1) A systematic data curation pipeline for constructing a dynamic video dataset with camera trajectory annotations; 2) A lightweight camera control injection module and corresponding training strategy that preserves dynamic video generation capabilities while adding camera control effect; 3) A clip-wise autoregressive generation recipe that enables extended range exploration of generated scenes.
0:10

I am blown away 🤯. Check this out! CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains dynamic capabilities, accurate camera control, and scene consistency throughout the extended exploration." "Our model enables precise camera control across diverse scenarios while preserving dynamic scene elements, e.g." "Our method can generate videos with strong 3D consistency, which enables high-quality 3D reconstruction using the camera-controlled videos." Contributions: 1) A systematic data curation pipeline for constructing a dynamic video dataset with camera trajectory annotations; 2) A lightweight camera control injection module and corresponding training strategy that preserves dynamic video generation capabilities while adding camera control effect; 3) A clip-wise autoregressive generation recipe that enables extended range exploration of generated scenes.

MrNeRF

12,633 просмотров • 1 год назад

We recently obtained the highest-resolution 3D images of the human brain ever taken from outside the skull. This is the first look. Introducing Aleph, a research lab building brain interfaces for the telepathic future. (1/n)
0:13

We recently obtained the highest-resolution 3D images of the human brain ever taken from outside the skull. This is the first look. Introducing Aleph, a research lab building brain interfaces for the telepathic future. (1/n)

Aleph

1,655,967 просмотров • 4 месяцев назад

FramePack: Generate Video Forever [NVIDIA ONLY] The creator of ControlNet just dropped a new approach to generating videos in a PROGRESSIVE way--you can generate loooong videos with LOW VRAM (just 6GB) SUP3RMASS1VE wrote a 1-click launcher to run the Gradio app on your PC!
0:13

FramePack: Generate Video Forever [NVIDIA ONLY] The creator of ControlNet just dropped a new approach to generating videos in a PROGRESSIVE way--you can generate loooong videos with LOW VRAM (just 6GB) SUP3RMASS1VE wrote a 1-click launcher to run the Gradio app on your PC!

cocktail peanut

202,544 просмотров • 1 год назад

🔥🚨BREAKING: A petri dish filled with human brain cells just learned to how to play DOOM Scientists at Cortical Labs have successfully trained a cluster of approximately 200,000 living human neurons, grown on a microelectrode array chip, to play the classic 3D video game DOOM.
0:15

🔥🚨BREAKING: A petri dish filled with human brain cells just learned to how to play DOOM Scientists at Cortical Labs have successfully trained a cluster of approximately 200,000 living human neurons, grown on a microelectrode array chip, to play the classic 3D video game DOOM.

Dom Lucre | Breaker of Narratives

2,796,850 просмотров • 8 месяцев назад

We got a new custom LED screen table for future convention setups… we knew just what video to test it with first 🤣
0:19

Sensitive content

We got a new custom LED screen table for future convention setups… we knew just what video to test it with first 🤣

14 Werewolves 🔜 FM, AqF, FurPoc

20,438 просмотров • 8 месяцев назад

🚨 Seedance 2.5 just dropped, and AI video generation has reached a new level. I tested it by creating a 3D Pixar-style short film 🤯 From one simple prompt, it generated impressive visuals, smooth motion, and a complete story flow. The best part? You can try Seedance 2.5 on Dola AI for free. No complicated setup. No technical workflow. Just describe your idea and start creating. #dola #ai #seedance2.5
0:10

🚨 Seedance 2.5 just dropped, and AI video generation has reached a new level. I tested it by creating a 3D Pixar-style short film 🤯 From one simple prompt, it generated impressive visuals, smooth motion, and a complete story flow. The best part? You can try Seedance 2.5 on Dola AI for free. No complicated setup. No technical workflow. Just describe your idea and start creating. #dola #ai #seedance2.5

KICHOCHEZI

25,460 просмотров • 3 месяцев назад

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:
0:14

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:

Lior Alexander

158,600 просмотров • 3 лет назад