Loading video...

Video Failed to Load

Go Home

Text-To-4D Dynamic Scene Generation Presents MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions. proj: abs:

arankomatsuzaki's profile picture

Aran Komatsuzaki

170,595 subscribers

54,664 views • 3 years ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Introducing Make-A-Video3D! Generating 3D dynamic (mini) scenes from input text. That is, text --> 4D! Needs no 4D data (i.e., no dynamic 3D data), no static 3D data, no paired text-video data. Paper: Website:
0:16

Introducing Make-A-Video3D! Generating 3D dynamic (mini) scenes from input text. That is, text --> 4D! Needs no 4D data (i.e., no dynamic 3D data), no static 3D data, no paired text-video data. Paper: Website:

Devi Parikh

305,729 views • 3 years ago

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n
0:12

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n

Chen Geng

43,132 views • 5 months ago

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]
0:16

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]

Xuanchi Ren

26,602 views • 1 year ago

For dynamic, explosive text-to-video, Kling is still an absolute masterclass.
0:10

For dynamic, explosive text-to-video, Kling is still an absolute masterclass.

Pierrick Chevallier | IA

12,525 views • 5 months ago

📢DNF: Generating 4D animations with dictionary-based neural fields! Xinyi Zhang presents a new dictionary-based neural field for unconditional 4D generation of deforming shapes -- generating motions with high-quality shape and temporal consistency.
0:19

📢DNF: Generating 4D animations with dictionary-based neural fields! Xinyi Zhang presents a new dictionary-based neural field for unconditional 4D generation of deforming shapes -- generating motions with high-quality shape and temporal consistency.

Angela Dai

14,639 views • 1 year ago

Google presents MobileDiffusion Subsecond Text-to-Image Generation on Mobile Devices paper page: MobileDiffusion achieves a remarkable sub-second inference speed for generating a 512 × 512 image on mobile devices, establishing a new state of the art.
0:20

Google presents MobileDiffusion Subsecond Text-to-Image Generation on Mobile Devices paper page: MobileDiffusion achieves a remarkable sub-second inference speed for generating a 512 × 512 image on mobile devices, establishing a new state of the art.

AK

150,538 views • 2 years ago

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.
0:13

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

MrNeRF

22,698 views • 1 year ago

Excited to share our #CoRL2024 paper on humanoid motion generation! Combining human motion priors with VLM feedback, Harmon generates natural, expressive, and text-aligned humanoid motion from freeform text descriptions. 👇(1/4)
0:14

Excited to share our #CoRL2024 paper on humanoid motion generation! Combining human motion priors with VLM feedback, Harmon generates natural, expressive, and text-aligned humanoid motion from freeform text descriptions. 👇(1/4)

Zhenyu Jiang

36,868 views • 1 year ago

This is such a cool interactive website to learn how LLMs work. It’s combines step-by-step text explanations with dynamic 3D visualizations for an immersive learning experience. Available to everyone for free!
0:12

This is such a cool interactive website to learn how LLMs work. It’s combines step-by-step text explanations with dynamic 3D visualizations for an immersive learning experience. Available to everyone for free!

Alvaro Cintas

305,446 views • 1 year ago

Google presents LayoutVLM a model that can generate 3D layouts from text instructions
0:13

Google presents LayoutVLM a model that can generate 3D layouts from text instructions

Dreaming Tulpa 🥓👑

27,205 views • 1 year ago

⚡️ CSS ::highlight() + TreeWalker for dynamic text search: - No DOM manipulation - Pure CSS ::highlight() styling - Efficient TreeWalker traversal [Code Snippet 👇]
0:18

⚡️ CSS ::highlight() + TreeWalker for dynamic text search: - No DOM manipulation - Pure CSS ::highlight() styling - Efficient TreeWalker traversal [Code Snippet 👇]

Bharat Kara

39,253 views • 11 months ago

MoVieS Motion-Aware 4D Dynamic View Synthesis in One Second
0:14

MoVieS Motion-Aware 4D Dynamic View Synthesis in One Second

AK

29,601 views • 11 months ago

this looks insane, LLaDA app just dropped by apolinario (poli) A text generation method different from the traditional left-to-right approach a diffusion model with an unprecedented 8B scale, trained entirely from scratch, rivaling LLaMA3 8B in performance
0:17

this looks insane, LLaDA app just dropped by apolinario (poli) A text generation method different from the traditional left-to-right approach a diffusion model with an unprecedented 8B scale, trained entirely from scratch, rivaling LLaMA3 8B in performance

AK

26,694 views • 1 year ago

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,039 views • 1 year ago

We're excited to introduce a new dataset, 4D-DRESS, at #CVPR2024. It features diverse human subjects, realistic clothing deformations, and dynamic body motion in 4D. (1/6) 🧵👇
0:18

We're excited to introduce a new dataset, 4D-DRESS, at #CVPR2024. It features diverse human subjects, realistic clothing deformations, and dynamic body motion in 4D. (1/6) 🧵👇

AIT Lab

10,631 views • 2 years ago

The secret to dynamic poses is all in the box method!
0:14

The secret to dynamic poses is all in the box method!

GOODPOP 🍭

84,849 views • 2 months ago

Today we're sharing details on AudioCraft, a new family of generative AI models built for generating high-quality, realistic audio & music from text. AudioCraft is a single code base that works for music, sound, compression & generation — all in the same place. More details ⬇️
0:17

Today we're sharing details on AudioCraft, a new family of generative AI models built for generating high-quality, realistic audio & music from text. AudioCraft is a single code base that works for music, sound, compression & generation — all in the same place. More details ⬇️

AI at Meta

677,765 views • 2 years ago

WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments TL;DR: We present WildGS-SLAM, a robust monocular RGB SLAM system. - Utilizes uncertainty-aware tracking and mapping to handle dynamic scenes. - Leverages DINOv2-based uncertainty maps for dynamic object removal. - Improves tracking and mapping. - Enables high-quality view synthesis.
0:19

WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments TL;DR: We present WildGS-SLAM, a robust monocular RGB SLAM system. - Utilizes uncertainty-aware tracking and mapping to handle dynamic scenes. - Leverages DINOv2-based uncertainty maps for dynamic object removal. - Improves tracking and mapping. - Enables high-quality view synthesis.

MrNeRF

17,382 views • 1 year ago