Loading video...

Video Failed to Load

Go Home

📢 New Paper PointSt3R: Point Tracking through 3D Grounded Correspondence Can point tracking be re-formulated as pairwise frame correspondence solely? We fine-tuning MASt3R with dynamic correspondences and a visibility loss and achieve competitive point tracking results 1/3

dimadamen's profile picture

Dima Damen @CVPR

11,293 subscribers

10,488 views • 8 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Excellent new fine-grained tracking from DeepMind: TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement arxiv: project: tldr: TapNet for localization then PIPs-style refinement; outperforms everything!
0:31

Excellent new fine-grained tracking from DeepMind: TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement arxiv: project: tldr: TapNet for localization then PIPs-style refinement; outperforms everything!

Adam W. Harley

203,971 views • 3 years ago

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.
0:33

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,533 views • 3 years ago

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.
0:27

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

218,966 views • 1 year ago

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇
0:32

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,233 views • 3 months ago

Introducing #DJIRS5, a lightweight commercial stabilizer built to lead the scene. Featuring touchscreen subject selection with the new Enhanced Intelligent Tracking Module, enabling precise tracking of people, vehicles, pets, and custom subjects. All-axis fine-tuning knobs deliver smooth, accurate balancing, while the Electronic Briefcase Handle offers direct, intuitive control for more creative angles. Faster charging and extended battery life keep creativity moving, turning ambition into cinematic results. Key features of the DJI RS 5: ✅ Enhanced Intelligent Tracking and Composition ✅ Briefcase Handle With Gimbal & Camera Control ✅ 1-Hour Full Charge, 15% Extended Runtime ✅ Fine-Tuning Knobs and Teflon™ Enhanced Balancing ✅ 2nd-Gen Automated Axis Locks ✅ New Z-Axis Indicator Learn more: Buy now: *The Enhanced Intelligent Tracking Module and Electronic Briefcase Handle are available for separate purchase or as part of a compatible combo. *The DJI RS BG33 Battery Grip is not available for purchase in India. #DJI #DJIRonin #DJIRS5 #RS5 #CameraGimbal #CameraStabilizer #Videography
0:59

Introducing #DJIRS5, a lightweight commercial stabilizer built to lead the scene. Featuring touchscreen subject selection with the new Enhanced Intelligent Tracking Module, enabling precise tracking of people, vehicles, pets, and custom subjects. All-axis fine-tuning knobs deliver smooth, accurate balancing, while the Electronic Briefcase Handle offers direct, intuitive control for more creative angles. Faster charging and extended battery life keep creativity moving, turning ambition into cinematic results. Key features of the DJI RS 5: ✅ Enhanced Intelligent Tracking and Composition ✅ Briefcase Handle With Gimbal & Camera Control ✅ 1-Hour Full Charge, 15% Extended Runtime ✅ Fine-Tuning Knobs and Teflon™ Enhanced Balancing ✅ 2nd-Gen Automated Axis Locks ✅ New Z-Axis Indicator Learn more: Buy now: *The Enhanced Intelligent Tracking Module and Electronic Briefcase Handle are available for separate purchase or as part of a compatible combo. *The DJI RS BG33 Battery Grip is not available for purchase in India. #DJI #DJIRonin #DJIRS5 #RS5 #CameraGimbal #CameraStabilizer #Videography

DJI

62,155 views • 5 months ago

Introducing Vtuber's 3D stream gear I'm currently using a 15-point tracking gear called REBOCAP. As you can see in the video, the 3D model moves quite smoothly. Moreover, it can be purchased quite cheaply for about 200 dollars. It seems that many people are sad about the lack of information on 3D streams, so I thought I would share this information. Please share it with many people.
2:22

Introducing Vtuber's 3D stream gear I'm currently using a 15-point tracking gear called REBOCAP. As you can see in the video, the 3D model moves quite smoothly. Moreover, it can be purchased quite cheaply for about 200 dollars. It seems that many people are sad about the lack of information on 3D streams, so I thought I would share this information. Please share it with many people.

🍊Oren-chan/加賀オレン🍊＠JPVtuber　2020/07/05debut

270,877 views • 1 year ago

VGGT: Visual Geometry Grounded Transformer TL;DR: Is DUSt3R facing a formidable new rival? Contributions: (1) We introduce VGGT, a large feed-forward transformer that can, given one, a few, or even hundreds of images of a scene, predict all its key 3D attributes - including camera intrinsics and extrinsics, point maps, depth maps, and 3D point tracks - in seconds. (2) We demonstrate that VGGT’s predictions are directly usable, being highly competitive and usually better than those of state-of-the-art methods that use slow post-processing optimization techniques. (3) We also show that when further combined with BA post-processing, VGGT achieves state-of-the-art results across the board, even when compared to methods that specialize in a subset of 3D tasks, often improving quality substantially.
0:28

VGGT: Visual Geometry Grounded Transformer TL;DR: Is DUSt3R facing a formidable new rival? Contributions: (1) We introduce VGGT, a large feed-forward transformer that can, given one, a few, or even hundreds of images of a scene, predict all its key 3D attributes - including camera intrinsics and extrinsics, point maps, depth maps, and 3D point tracks - in seconds. (2) We demonstrate that VGGT’s predictions are directly usable, being highly competitive and usually better than those of state-of-the-art methods that use slow post-processing optimization techniques. (3) We also show that when further combined with BA post-processing, VGGT achieves state-of-the-art results across the board, even when compared to methods that specialize in a subset of 3D tasks, often improving quality substantially.

MrNeRF

29,461 views • 1 year ago

Claude Code v2.1.16 adds new task management system with full dependency tracking I demoed this through CC Mirror a few weeks ago and it was extremely powerful Watch the full video - this is going to change the way you work forever Each task can be assigned to sub agents in parallel and they can easily pick up tasks as they become unblocked
1:57

Claude Code v2.1.16 adds new task management system with full dependency tracking I demoed this through CC Mirror a few weeks ago and it was extremely powerful Watch the full video - this is going to change the way you work forever Each task can be assigned to sub agents in parallel and they can easily pick up tasks as they become unblocked

Numman Ali

51,261 views • 6 months ago

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!
0:29

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Bilawal Sidhu

703,816 views • 1 year ago

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion
11:23

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion

Bilawal Sidhu

26,863 views • 2 months ago

Tariffs just crashed crypto. Again. I'm sure many are watching the hourly chart, but I'm watching new Philly Fed data that just came out showing a 21-point swing into expansion territory. That's a leading indicator for the PMI cycle I've been tracking. QT ended. Liquidity injections are starting. The macro setup hasn't changed. Be careful to get distracted. Can we dip more? Yes. Is this a bear market? I don't think so. I'm zoomed out and I'm holding.
15:39

Tariffs just crashed crypto. Again. I'm sure many are watching the hourly chart, but I'm watching new Philly Fed data that just came out showing a 21-point swing into expansion territory. That's a leading indicator for the PMI cycle I've been tracking. QT ended. Liquidity injections are starting. The macro setup hasn't changed. Be careful to get distracted. Can we dip more? Yes. Is this a bear market? I don't think so. I'm zoomed out and I'm holding.

Dan Gambardello

164,295 views • 6 months ago

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:
0:50

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,590 views • 10 months ago

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:
0:47

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:

Tencent Hy

20,792 views • 4 months ago

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.
0:26

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 views • 2 years ago

🎋MOCOPI vs VIVE TRACKERS🎋 I got a crazy crazy upgrade on full body tracking thanks to the support of my bambuds, so I made a silly dance comparison to compare the tracking between my old FBT and my new one!! ✨ I danced Bibbidiba by Suisei! (ft. bae as my guide lol) I had to use a different model for the Vive comparison because my Toffee model doesn't work well with the trackers! PROS AND CONS 🔽 1. Mocopi is SO MUCH EASIER to set up! It's very very beginner friendly as well and a great asset to get if you're just starting on doing 3D stuff or want to try it out! Vive trackers are lowkey a pain to set up each time! 2. Vive trackers are EXPENSIIIVEEEE!!!!! The tracking is so so so good but only invest in it if you plan to do lots of 3D content! Mocopi is a much affordable option! 3. Related to point 2, but you get what you pay for when it comes to tracking. Vive may be more expensive, but the tracking quality and accuracy makes it worth it. 4. This is only for people that don't have the Valve Headset, but I use a Quest 3 and I have to do EXTRA SETUP to calibrate the trackers with an external program to use in VRChat! (Mocopi also needs an external program (SlimeVR) but it's much easier to set up imo). 5. Since calibration is paired to your headset with the vive trackers, if your headset goes to sleep or something the calibration will be LOST and you will have to recalibrate again which may take 1 to 2 minutes. Mocopi doesn't have this issue and recalibrating is only one button. 6. Mocopi is an IMU so no base stations required but because of this the trackers need to be recalibrated A LOT of times. This may be annoying if you're streaming and you constantly have to press the button to recalibrate. Vive trackers may take longer but you VERY RARELY have to recalibrate if your headset doesn't go to sleep. 7. Vive trackers take a lot of space on your setup. Each tracker needs to be connected to the PC so if you have a lot you may need to buy an USD hub! 8. You need to have a good room space to use vive trackers properly because the base stations need to have a clear view of your trackers or else they can lose tracking! Mocopi is an IMU, so no amount of obstruction will make the trackers lose tracking. 9. The amount of poses and movement that you can do with vive trackers is NIGHT AND DAY compared to the mocopi. A lot of poses or rapid movements will make the mocopis mess up their tracking and you will have to recalibrate. Vive trackers will not need any recalibration. Basically if you're a vtuber that only wants to dabble into 3D content as a fun little thing I think mocopis are very much a good choice! But if you'd like to do 3D stuff that require more intensive tracking like dancing, vive trackers are excellent and very much worth the price!
0:40

🎋MOCOPI vs VIVE TRACKERS🎋 I got a crazy crazy upgrade on full body tracking thanks to the support of my bambuds, so I made a silly dance comparison to compare the tracking between my old FBT and my new one!! ✨ I danced Bibbidiba by Suisei! (ft. bae as my guide lol) I had to use a different model for the Vive comparison because my Toffee model doesn't work well with the trackers! PROS AND CONS 🔽 1. Mocopi is SO MUCH EASIER to set up! It's very very beginner friendly as well and a great asset to get if you're just starting on doing 3D stuff or want to try it out! Vive trackers are lowkey a pain to set up each time! 2. Vive trackers are EXPENSIIIVEEEE!!!!! The tracking is so so so good but only invest in it if you plan to do lots of 3D content! Mocopi is a much affordable option! 3. Related to point 2, but you get what you pay for when it comes to tracking. Vive may be more expensive, but the tracking quality and accuracy makes it worth it. 4. This is only for people that don't have the Valve Headset, but I use a Quest 3 and I have to do EXTRA SETUP to calibrate the trackers with an external program to use in VRChat! (Mocopi also needs an external program (SlimeVR) but it's much easier to set up imo). 5. Since calibration is paired to your headset with the vive trackers, if your headset goes to sleep or something the calibration will be LOST and you will have to recalibrate again which may take 1 to 2 minutes. Mocopi doesn't have this issue and recalibrating is only one button. 6. Mocopi is an IMU so no base stations required but because of this the trackers need to be recalibrated A LOT of times. This may be annoying if you're streaming and you constantly have to press the button to recalibrate. Vive trackers may take longer but you VERY RARELY have to recalibrate if your headset doesn't go to sleep. 7. Vive trackers take a lot of space on your setup. Each tracker needs to be connected to the PC so if you have a lot you may need to buy an USD hub! 8. You need to have a good room space to use vive trackers properly because the base stations need to have a clear view of your trackers or else they can lose tracking! Mocopi is an IMU, so no amount of obstruction will make the trackers lose tracking. 9. The amount of poses and movement that you can do with vive trackers is NIGHT AND DAY compared to the mocopi. A lot of poses or rapid movements will make the mocopis mess up their tracking and you will have to recalibrate. Vive trackers will not need any recalibration. Basically if you're a vtuber that only wants to dabble into 3D content as a fun little thing I think mocopis are very much a good choice! But if you'd like to do 3D stuff that require more intensive tracking like dancing, vive trackers are excellent and very much worth the price!

Toffee 🐾🎋

80,586 views • 1 year ago

1) Wemby takes Lively off the bounce, skying over him 2) Wemby rejects Lively easily making him look small 3) Wemby dazzles with the handle and hits a stepback setting up a 4 point play How hyperbolic can we be? Look at the Spurs bench. This is absurd.
0:41

1) Wemby takes Lively off the bounce, skying over him 2) Wemby rejects Lively easily making him look small 3) Wemby dazzles with the handle and hits a stepback setting up a 4 point play How hyperbolic can we be? Look at the Spurs bench. This is absurd.

Hot Hand Theory

35,933 views • 9 months ago

FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.
0:21

FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.

AK

45,459 views • 2 years ago

Russian satellites have approached ICEYE spacecraft, which are used to assist the Ukrainian Armed Forces. According to orbit tracking services, Russian "Kosmos" series spacecraft (2609-2614) have altered their trajectories and moved into the same orbital plane as the Finnish ICEYE satellites, which are used for radar imaging. It is reported that such maneuvers may allow for tracking the satellites and potentially interfering with them, including through the use of electronic warfare means. Russia now has a new type satellite that has ability to decommission in orbit multiple satellites known as the " SHREDDER!" STARLINK may soon come to its own end with a new type of microwave weapon ... WAR may well be in the stars sooner than we think.....
0:28

Russian satellites have approached ICEYE spacecraft, which are used to assist the Ukrainian Armed Forces. According to orbit tracking services, Russian "Kosmos" series spacecraft (2609-2614) have altered their trajectories and moved into the same orbital plane as the Finnish ICEYE satellites, which are used for radar imaging. It is reported that such maneuvers may allow for tracking the satellites and potentially interfering with them, including through the use of electronic warfare means. Russia now has a new type satellite that has ability to decommission in orbit multiple satellites known as the " SHREDDER!" STARLINK may soon come to its own end with a new type of microwave weapon ... WAR may well be in the stars sooner than we think.....

𝐃𝐚𝐯𝐢𝐝 𝐙 🇷🇺🇮🇪

63,272 views • 2 days ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.
0:50

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,708 views • 3 years ago