Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

SpatialTrackerV2: unified, end-to-end 3D point tracking model which simultaneously estimates Camera Motion, Consistent Geometry and Pixel-wise 3D Trajectories.

Bilawal Sidhu

103,911 subscribers

20,346 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Comments

Bilawal Sidhu1 year ago

New ICCV paper. Here's a HuggingFace demo you can try today, as well as code for the offline tracker. Full paper and code release forthcoming.

Prashant1 year ago

spatial ai is so important for building the digiverse

Himanshu Kumar1 year ago

Impressive, but true spatial awareness needs object permanence, not just point tracking.

Zoe Ai collab manager1 year ago

Dm

Related Videos

Introducing FLARE #CVPR2026 2025 FLARE is a feed-forward model that simultaneously estimates high-quality camera poses, 3D geometry, and appearance from sparse uncalibrated images. 1/4

Introducing FLARE #CVPR2026 2025 FLARE is a feed-forward model that simultaneously estimates high-quality camera poses, 3D geometry, and appearance from sparse uncalibrated images. 1/4

Gordon Wetzstein

29,572 views • 1 year ago

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs

Zhiwen(Aaron) Fan

14,895 views • 1 year ago

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

"YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting" TL;DR: a unified 3D Gaussian splatting model that reconstructs high-quality scene geometry and camera poses from unposed/uncalibrated images in a single forward pass.

Alexandre Morgand

15,035 views • 6 months ago

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

AK

280,570 views • 3 years ago

Apple drops a new 3D model generator, LiTo > Learns geometry + appearance together in a unified 3D latent space > Turns RGB-D images into a compact surface light field > Recreates specular highlights under complex lighting > Generates 3D objects from a single image with input-consistent materials Big step toward more realistic 3D generation

Apple drops a new 3D model generator, LiTo > Learns geometry + appearance together in a unified 3D latent space > Turns RGB-D images into a compact surface light field > Recreates specular highlights under complex lighting > Generates 3D objects from a single image with input-consistent materials Big step toward more realistic 3D generation

⚡AI Search⚡

13,033 views • 3 months ago

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

World Labs

15,870 views • 10 months ago

i bought a iphone just to add tracking to my model 💀 2d live pixel texture face on a 3d body made in the #GodotEngine

i bought a iphone just to add tracking to my model 💀 2d live pixel texture face on a 3d body made in the #GodotEngine

byte 🕹️ nme64 | DEBUT: TBA

50,954 views • 2 years ago

Christian Rupprecht explains their interpretability research in 3D computer vision, testing if (and where in the model) multi-view transformers like VGGT, DepthAnything 3, and DUSt3R use point/patch correspondences to make sense of 3D scene geometry.

Christian Rupprecht explains their interpretability research in 3D computer vision, testing if (and where in the model) multi-view transformers like VGGT, DepthAnything 3, and DUSt3R use point/patch correspondences to make sense of 3D scene geometry.

Chris Offner

74,628 views • 5 months ago

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

AK

194,605 views • 2 years ago

Excited to share SLAMFormer-∞: an Infinite SLAM Transformer for unbounded frontend and backend processing. From monocular RGB alone, it jointly estimates camera trajectories + dense geometry—and scales to a 17 km urban drive. 🧵

Excited to share SLAMFormer-∞: an Infinite SLAM Transformer for unbounded frontend and backend processing. From monocular RGB alone, it jointly estimates camera trajectories + dense geometry—and scales to a 17 km urban drive. 🧵

Hang Zhao

17,276 views • 25 days ago

Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.

Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.

World Labs

10,613,761 views • 7 days ago

Introducing 👀Stereo4D👀 A method for mining 4D from internet stereo videos. It enables large-scale, high-quality, dynamic, *metric* 3D reconstructions, with camera poses and long-term 3D motion trajectories. We used Stereo4D to make a dataset of over 100k real-world 4D scenes.

Introducing 👀Stereo4D👀 A method for mining 4D from internet stereo videos. It enables large-scale, high-quality, dynamic, metric 3D reconstructions, with camera poses and long-term 3D motion trajectories. We used Stereo4D to make a dataset of over 100k real-world 4D scenes.

Linyi Jin

94,645 views • 1 year ago

🔥Your Videos Just Became 4D Worlds (1/2) #OmniX predicts 3D trajectories for every pixel across any viewpoint and time - all in one forward pass. Our key insight: 3D motion is low-rank, so a small set of dynamic tokens is enough to track everything. Code is here👇 #ECCV'26

🔥Your Videos Just Became 4D Worlds (1/2) #OmniX predicts 3D trajectories for every pixel across any viewpoint and time - all in one forward pass. Our key insight: 3D motion is low-rank, so a small set of dynamic tokens is enough to track everything. Code is here👇 #ECCV'26

Tengfei Wang

31,667 views • 1 month ago

byebye expensive motion tracking equipment 👋 ai makes motion capturing so easy now! nvidia presented GENMO last week, a new model that can generate and estimate human motion from text, audio, video, and 3D keyframes

byebye expensive motion tracking equipment 👋 ai makes motion capturing so easy now! nvidia presented GENMO last week, a new model that can generate and estimate human motion from text, audio, video, and 3D keyframes

Dreaming Tulpa 🥓👑

73,693 views • 1 year ago

🪬 #Live3D #Live2D #Live2DWIP 🪬 Testing finger articulation... Left: randomized auto motion Right: real-time camera tracking I wonder how many Vtubers would consider adding this kind of pseudo-3D hand to their Live2D model 🧐?

🪬 #Live3D #Live2D #Live2DWIP 🪬 Testing finger articulation... Left: randomized auto motion Right: real-time camera tracking I wonder how many Vtubers would consider adding this kind of pseudo-3D hand to their Live2D model 🧐?

📐Hephaestus📏Live2D匠人魂

36,652 views • 7 months ago

Most generative models predict pixels. Predicting a 3D scene instead has many benefits: the scene won’t change if you look away and come back, and it obeys the basic physical rules of 3D geometry. The simplest way to visualize the 3D scene is a depth map, where each pixel is colored by its distance to the camera. 4/n

Most generative models predict pixels. Predicting a 3D scene instead has many benefits: the scene won’t change if you look away and come back, and it obeys the basic physical rules of 3D geometry. The simplest way to visualize the 3D scene is a depth map, where each pixel is colored by its distance to the camera. 4/n

World Labs

16,605 views • 1 year ago

Introducing Texel Splatting: Perspective-Stable 3D Pixel Art open source paper+code Most 3D pixel art techniques (e.g. t3ssel8r, ProPixelizer) snap pixels to a screen grid, which only works with an orthographic camera Texel splatting solves this for perspective cameras: first,

Introducing Texel Splatting: Perspective-Stable 3D Pixel Art open source paper+code Most 3D pixel art techniques (e.g. t3ssel8r, ProPixelizer) snap pixels to a screen grid, which only works with an orthographic camera Texel splatting solves this for perspective cameras: first,

dylan

112,567 views • 5 months ago

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)

Hanwen Jiang

70,830 views • 1 year ago