正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing St4RTrack!🖖 Simultaneous 4D Reconstruction and Tracking in the world coordinate feed-forwardly, just by changing the meaning of two pointmaps!

Junyi Zhang

2,730 subscribers

52,426 次观看 • 1 年前 •via X (Twitter)

艺术科学技术教育

Anya Rossi• Live Now

Private livecam show

10 条评论

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

From static to dynamic, MonST3R reconstructs pointmaps at their own times. St4RTrack instead estimates both at the same moment—predicting how points in frame1 move to frame2 and reconstructing geometry of frame2. Same architecture, now for simultaneous tracking + reconstruction.

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

Such representation can be learned supervisedly from small-scale, synthetic 4D datasets. But to better generalize to real scenes, St4RTrack can also adapt to new videos *without any 4D labels*, using only 2D reprojection cues like trajectories & monocular depth.

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

To evaluate the method, we propose a benchmark, *WroldTrack*, for world coordinate tracking and dynamic 3d reconstruction

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

More interestingly is our joint reconstruction and tracking results, even the fully feed-forward mode gives promising results ⬇️

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

We present a "more interactive" results on our webpage. Come and check it out!

Junyi Zhang @ICLR'25 的头像

Junyi Zhang @ICLR'251 年前

Great collaboration with @HavenFeng (co-lead), @QianqianWang5, @yufei_ye, @pengcheng_147, @Michael_J_Black, @trevordarrell, @akanazawa 🥰

CoinStats 的头像

CoinStats1 年前

With CoinStats, you can track all your assets in one place and get advanced analytics. Connect your portfolio to start tracking your Profit & Loss today:

Shuo Chen 的头像

Shuo Chen1 年前

cool work!

David McAllister 的头像

David McAllister1 年前

Congrats Junyi!

Alexandre Devaux 的头像

Alexandre Devaux1 年前

🔥

相关视频

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

779,506 次观看 • 1 个月前

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

Matthias Niessner

61,537 次观看 • 2 个月前

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Yuan Liu

19,919 次观看 • 4 个月前

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!

Kyle Berkshire’s swing in 4D 🤯 The 3x World Long Drive Champion stopped by the Radiant Images 4D stage at the PGA Championship … and didn’t disappoint!

PGA of America

95,333 次观看 • 1 个月前

The world has rejected the meaning of suffering, the pope and the church must lead by example.

The world has rejected the meaning of suffering, the pope and the church must lead by example.

Catholic Arena

10,901 次观看 • 1 年前

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 次观看 • 1 年前

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now

Operationality, functionality, and regularity of Agdam-Khankandi road must be ensured following the simultaneous passage by ICRC convoy. TRT World Now

Hikmet Hajiyev

38,859 次观看 • 2 年前

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit

i built this 4d reconstruction of iran's chokehold on the world's oil. almost nothing is getting through the strait of hormuz. you can clearly see the path ships took before/after the blockade, and can even detect & track "dark vessels" getting through. 0:00 - clearly see the before/after route change 0:26 - tracking iran's new tollbooth 1:31 - dark vessel detection 2:17 - tracking ships going dark mid-transit

Bilawal Sidhu

37,145 次观看 • 3 个月前

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI

INTRODUCING THE ‘FOR YOU’ FEED YOUR PERSONALIZED FEED TO SPOT THE BEST AND EARLIEST TRADES FROM ALL USERS ON THE APP – ALL POWERED BY AI

VECTOR

25,334 次观看 • 1 年前

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.

CBS News Texas Exclusive: Two simultaneous raids on jewelry stores on Thursday in Irving and Frisco were the culmination of a year-long investigation by the Collin County Sheriff's Office and federal authorities.

CBS News Texas

639,612 次观看 • 5 个月前

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.

Two marine veterans wounded in Iraq speak with Jake Tapper 🦅 about the meaning of Memorial Day and the sacrifices made by US servicemembers.

The Lead CNN

29,233 次观看 • 1 个月前

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.

Introducing WiseWorld: A revolutionary solution for generating procedural worlds at scale! Developed by Darewise 🟧, this cutting-edge software is changing the game in world-building.

BLIFE Protocol

21,068 次观看 • 1 年前

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨

Introducing MV-DUSt3R+: Transform sparse images into photorealistic replicas of actual spaces in just seconds! 🚀 Seamlessly bridge the gap between the physical and digital worlds with groundbreaking single-stage scene reconstruction. 🌍✨

Reality Labs at Meta

16,973 次观看 • 1 年前

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 次观看 • 1 年前

“In Gaza, men taught the world the meaning of pride!”

“In Gaza, men taught the world the meaning of pride!”

أجيج

108,095 次观看 • 2 年前

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.

The legendary simultaneous landing of two Falcon Heavy boosters—a masterpiece of precision engineering and visual harmony.

Black Hole

19,972 次观看 • 2 个月前

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?

Mogbo moya by TWO, Do you know the meaning of Mogbo moya?

Yorùbáness

12,913 次观看 • 1 年前

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024

The feminist revolution is one of the hopeful examples of how people, just by changing the story, also change the world. @thedailyshow Ronny Chieng 🗓️ March 2024

Yuval Noah Harari

63,824 次观看 • 4 个月前

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:

forget about physical camera stabilizations why don't we reconstruct the world in 4d and rerecord the footage:

Dreaming Tulpa 🥓👑

106,083 次观看 • 1 年前

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 次观看 • 1 年前