Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing St4RTrack!🖖 Simultaneous 4D Reconstruction and Tracking in the world coordinate feed-forwardly, just by changing the meaning of two pointmaps!

junyi42's profile picture

Junyi Zhang

2,730 subscribers

52,426 Aufrufe • vor 1 Jahr •via X (Twitter)

KunstWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

From static to dynamic, MonST3R reconstructs pointmaps at their own times. St4RTrack instead estimates both at the same moment—predicting how points in frame1 move to frame2 and reconstructing geometry of frame2. Same architecture, now for simultaneous tracking + reconstruction.

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

Such representation can be learned supervisedly from small-scale, synthetic 4D datasets. But to better generalize to real scenes, St4RTrack can also adapt to new videos *without any 4D labels*, using only 2D reprojection cues like trajectories & monocular depth.

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

To evaluate the method, we propose a benchmark, *WroldTrack*, for world coordinate tracking and dynamic 3d reconstruction

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

More interestingly is our joint reconstruction and tracking results, even the fully feed-forward mode gives promising results ⬇️

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

We present a "more interactive" results on our webpage. Come and check it out!

Profilbild von Junyi Zhang @ICLR'25
Junyi Zhang @ICLR'25vor 1 Jahr

Great collaboration with @HavenFeng (co-lead), @QianqianWang5, @yufei_ye, @pengcheng_147, @Michael_J_Black, @trevordarrell, @akanazawa 🥰

Profilbild von CoinStats
CoinStatsvor 1 Jahr

With CoinStats, you can track all your assets in one place and get advanced analytics. Connect your portfolio to start tracking your Profit & Loss today:

Profilbild von Shuo Chen
Shuo Chenvor 1 Jahr

cool work!

Profilbild von David McAllister
David McAllistervor 1 Jahr

Congrats Junyi!

Profilbild von Alexandre Devaux
Alexandre Devauxvor 1 Jahr

🔥

Ähnliche Videos

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.
0:48

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

779,506 Aufrufe • vor 1 Monat

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw
0:42

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

Matthias Niessner

61,537 Aufrufe • vor 2 Monaten

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:
1:42

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Yuan Liu

19,919 Aufrufe • vor 4 Monaten

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!
0:22

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!

PGA of America

95,333 Aufrufe • vor 1 Monat

The world has rejected the meaning of suffering, the pope and the church must lead by example.
1:07

The world has rejected the meaning of suffering, the pope and the church must lead by example.

Catholic Arena

10,901 Aufrufe • vor 1 Jahr

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:
1:00

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 Aufrufe • vor 1 Jahr

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now
1:47

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now

Hikmet Hajiyev

38,859 Aufrufe • vor 2 Jahren

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit
2:43

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit

Bilawal Sidhu

37,145 Aufrufe • vor 3 Monaten

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI
0:22

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI

VECTOR

25,334 Aufrufe • vor 1 Jahr

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.
1:01

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.

CBS News Texas

639,612 Aufrufe • vor 5 Monaten

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.
5:41

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.

The Lead CNN

29,233 Aufrufe • vor 1 Monat

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.
0:25

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.

BLIFE Protocol

21,068 Aufrufe • vor 1 Jahr

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨
2:00

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨

Reality Labs at Meta

16,973 Aufrufe • vor 1 Jahr

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.
5:19

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 Aufrufe • vor 1 Jahr

“In Gaza, men taught the world the meaning of pride!”
2:27

“In Gaza, men taught the world the meaning of pride!”

أجيج

108,095 Aufrufe • vor 2 Jahren

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.
0:20

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.

Black Hole

19,972 Aufrufe • vor 2 Monaten

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?
2:05

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?

Yorùbáness

12,913 Aufrufe • vor 1 Jahr

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024
1:28

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024

Yuval Noah Harari

63,824 Aufrufe • vor 4 Monaten

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:
1:19

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:

Dreaming Tulpa 🥓👑

106,083 Aufrufe • vor 1 Jahr

Opportunity is the first step to being great at just about anything, and this woman has provided 27,000 children the chance to do just that by introducing them to the world of gymnastics.
2:41

Opportunity is the first step to being great at just about anything, and this woman has provided 27,000 children the chance to do just that by introducing them to the world of gymnastics.

CBS Evening News

20,722 Aufrufe • vor 1 Jahr