Loading video...

Video Failed to Load

Go Home

Introducing 👀Stereo4D👀 A method for mining 4D from internet stereo videos. It enables large-scale, high-quality, dynamic, *metric* 3D reconstructions, with camera poses and long-term 3D motion trajectories. We used Stereo4D to make a dataset of over 100k real-world 4D scenes.

94,683 views • 1 year ago •via X (Twitter)

16 Comments

Linyi Jin's profile picture
Linyi Jin1 year ago

This type of data is ideal for learning the structure and dynamics of the real world. We gave this a shot — extending DUSt3R to model 3D motion, and training on our dataset. Given a pair of frames, our model predicts a 3D point cloud, and corresponding 3D motion trajectories.

Linyi Jin's profile picture
Linyi Jin1 year ago

See more scenes & details of how it works on our website: Paper: Thanks to the great team! Richard Tucker, @zhengqi_li, David Fouhey, @Jimantha, @holynski_ Please stay tuned for updates on data & code.

Sherwin Bahmani's profile picture
Sherwin Bahmani1 year ago

Congrats, will be super useful for the 4D reconstruction/generation community!!

Chen Wang's profile picture
Chen Wang1 year ago

Very amazing work! Wondering how to predict the 3D point trajectory of a video after pairwise prediction of DynaDust3r?

Linyi Jin's profile picture
Linyi Jin1 year ago

Thanks Chen! We've tried using DynaDust3r to predict motion at any time between two frames. We haven't tried to extend it to take all frames of a video, but could be a cool future direction.

Pedro Milcent's profile picture
Pedro Milcent1 year ago

Excellent paper @jin_linyi ! Exciting potential for models leveraging multi-modal data 🦾

IceTTT's profile picture
IceTTT1 year ago

Very interesting work!

Dejvuzzz's profile picture
Dejvuzzz1 year ago

This should be Good for 3D tracking . @jin_linyi . both 3D camera tracking and 3D mesh tracking.

Feiyu Yao's profile picture
Feiyu Yao1 year ago

great work! when will it be released? can't wait to use it.

☀️ Leon-Gerard Vandenberg 🇺🇸 🇳🇱 🇨🇦 🇦🇺's profile picture
☀️ Leon-Gerard Vandenberg 🇺🇸 🇳🇱 🇨🇦 🇦🇺1 year ago

🤩 wow 🤩

☀️ Leon-Gerard Vandenberg 🇺🇸 🇳🇱 🇨🇦 🇦🇺's profile picture
☀️ Leon-Gerard Vandenberg 🇺🇸 🇳🇱 🇨🇦 🇦🇺1 year ago

The world simulation and the singularity is near @PeterDiamandis @salimismail

Nikodeam.eth's profile picture
Nikodeam.eth1 year ago

Me patiently waiting for code

Time is Light's profile picture
Time is Light1 year ago

@zhengqi_li Time is Light !

HU Wenbo's profile picture
HU Wenbo1 year ago

Cool!

Maya N's profile picture
Maya N1 year ago

I'm blown away by the scale of your dataset! How do you plan to make it available for others to use?

Bilawal Sidhu's profile picture
Bilawal Sidhu1 year ago

Really cool to see VR180 videos have this impact -- this'll be huge for the computer vision community!

Related Videos