Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Introducing St4RTrack!🖖 Simultaneous 4D Reconstruction and Tracking in the world coordinate feed-forwardly, just by changing the meaning of two pointmaps!

junyi42's profile picture

Junyi Zhang

2,730 subscribers

52,426 görüntüleme • 1 yıl önce •via X (Twitter)

SanatBilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

10 Yorum

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

From static to dynamic, MonST3R reconstructs pointmaps at their own times. St4RTrack instead estimates both at the same moment—predicting how points in frame1 move to frame2 and reconstructing geometry of frame2. Same architecture, now for simultaneous tracking + reconstruction.

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

Such representation can be learned supervisedly from small-scale, synthetic 4D datasets. But to better generalize to real scenes, St4RTrack can also adapt to new videos *without any 4D labels*, using only 2D reprojection cues like trajectories & monocular depth.

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

To evaluate the method, we propose a benchmark, *WroldTrack*, for world coordinate tracking and dynamic 3d reconstruction

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

More interestingly is our joint reconstruction and tracking results, even the fully feed-forward mode gives promising results ⬇️

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

We present a "more interactive" results on our webpage. Come and check it out!

Junyi Zhang @ICLR'25 profil fotoğrafı
Junyi Zhang @ICLR'251 yıl önce

Great collaboration with @HavenFeng (co-lead), @QianqianWang5, @yufei_ye, @pengcheng_147, @Michael_J_Black, @trevordarrell, @akanazawa 🥰

CoinStats profil fotoğrafı
CoinStats1 yıl önce

With CoinStats, you can track all your assets in one place and get advanced analytics. Connect your portfolio to start tracking your Profit & Loss today:

Shuo Chen profil fotoğrafı
Shuo Chen1 yıl önce

cool work!

David McAllister profil fotoğrafı
David McAllister1 yıl önce

Congrats Junyi!

Alexandre Devaux profil fotoğrafı
Alexandre Devaux1 yıl önce

🔥

Benzer Videolar

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.
0:48

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

779,568 görüntüleme • 1 ay önce

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw
0:42

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

Matthias Niessner

61,537 görüntüleme • 2 ay önce

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:
1:42

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Yuan Liu

19,919 görüntüleme • 4 ay önce

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!
0:22

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!

PGA of America

95,333 görüntüleme • 1 ay önce

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:
1:00

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 görüntüleme • 1 yıl önce

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now
1:47

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now

Hikmet Hajiyev

38,859 görüntüleme • 2 yıl önce

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit
2:43

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit

Bilawal Sidhu

37,145 görüntüleme • 3 ay önce

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.
1:01

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.

CBS News Texas

639,612 görüntüleme • 5 ay önce

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI
0:22

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI

VECTOR

25,334 görüntüleme • 1 yıl önce

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.
5:41

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.

The Lead CNN

29,233 görüntüleme • 1 ay önce

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.
0:25

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.

BLIFE Protocol

21,068 görüntüleme • 1 yıl önce

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨
2:00

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨

Reality Labs at Meta

16,973 görüntüleme • 1 yıl önce

“In Gaza, men taught the world the meaning of pride!”
2:27

“In Gaza, men taught the world the meaning of pride!”

أجيج

108,095 görüntüleme • 2 yıl önce

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.
5:19

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 görüntüleme • 1 yıl önce

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.
0:20

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.

Black Hole

19,972 görüntüleme • 2 ay önce

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?
2:05

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?

Yorùbáness

12,913 görüntüleme • 1 yıl önce

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024
1:28

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024

Yuval Noah Harari

63,824 görüntüleme • 4 ay önce

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:
1:19

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:

Dreaming Tulpa 🥓👑

106,083 görüntüleme • 1 yıl önce

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:
0:32

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 görüntüleme • 1 yıl önce