Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

📢 New Paper PointSt3R: Point Tracking through 3D Grounded Correspondence Can point tracking be re-formulated as pairwise frame correspondence solely? We fine-tuning MASt3R with dynamic correspondences and a visibility loss and achieve competitive point tracking results 1/3

Dima Damen ✈️ #ECCV2026

11,588 subscribers

10,498 Aufrufe • vor 10 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von Dima Damen ✈️ #ECCV2026

Dima Damen ✈️ #ECCV2026vor 10 Monaten

Through balancing static and dynamic correspondences, the model can maintain MASt3R's power in tracking static parts of the scene but also track dynamic points successfully. We use *no* temporal knowledge - only pairwise matching!

Profilbild von Dima Damen ✈️ #ECCV2026

Dima Damen ✈️ #ECCV2026vor 10 Monaten

Trained visibility head shows impressive performance in identifying camera motion and tracking dynamic objects jointly... Code and Models are out Work led by Rhodri Guerrier w @AdamWHarley 3/3

Profilbild von ζ Pedram ζ

ζ Pedram ζvor 10 Monaten

Ooh, kind of a new pixel flow

Profilbild von Himanshu Kumar

Himanshu Kumarvor 10 Monaten

Ah, Dima, that's a clever idea! Re-formulating point tracking is quite innovative; I'm eager to see how it plays out.

Ähnliche Videos

Excellent new fine-grained tracking from DeepMind: TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement arxiv: project: tldr: TapNet for localization then PIPs-style refinement; outperforms everything!

Excellent new fine-grained tracking from DeepMind: TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement arxiv: project: tldr: TapNet for localization then PIPs-style refinement; outperforms everything!

Adam W. Harley

203,971 Aufrufe • vor 3 Jahren

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,567 Aufrufe • vor 3 Jahren

MonST3R can estimate 3D shapes from videos over time, creating a dynamic point cloud and tracking camera positions! This method improves video depth estimation and separates moving from still objects more effectively than previous techniques. Links ⬇️

MonST3R can estimate 3D shapes from videos over time, creating a dynamic point cloud and tracking camera positions! This method improves video depth estimation and separates moving from still objects more effectively than previous techniques. Links ⬇️

Dreaming Tulpa 🥓👑

13,484 Aufrufe • vor 1 Jahr

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

219,050 Aufrufe • vor 1 Jahr

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,315 Aufrufe • vor 5 Monaten

Introducing #DJIRS5, a lightweight commercial stabilizer built to lead the scene. Featuring touchscreen subject selection with the new Enhanced Intelligent Tracking Module, enabling precise tracking of people, vehicles, pets, and custom subjects. All-axis fine-tuning knobs deliver smooth, accurate balancing, while the Electronic Briefcase Handle offers direct, intuitive control for more creative angles. Faster charging and extended battery life keep creativity moving, turning ambition into cinematic results. Key features of the DJI RS 5: ✅ Enhanced Intelligent Tracking and Composition ✅ Briefcase Handle With Gimbal & Camera Control ✅ 1-Hour Full Charge, 15% Extended Runtime ✅ Fine-Tuning Knobs and Teflon™ Enhanced Balancing ✅ 2nd-Gen Automated Axis Locks ✅ New Z-Axis Indicator Learn more: Buy now: *The Enhanced Intelligent Tracking Module and Electronic Briefcase Handle are available for separate purchase or as part of a compatible combo. *The DJI RS BG33 Battery Grip is not available for purchase in India. #DJI #DJIRonin #DJIRS5 #RS5 #CameraGimbal #CameraStabilizer #Videography

Introducing #DJIRS5, a lightweight commercial stabilizer built to lead the scene. Featuring touchscreen subject selection with the new Enhanced Intelligent Tracking Module, enabling precise tracking of people, vehicles, pets, and custom subjects. All-axis fine-tuning knobs deliver smooth, accurate balancing, while the Electronic Briefcase Handle offers direct, intuitive control for more creative angles. Faster charging and extended battery life keep creativity moving, turning ambition into cinematic results. Key features of the DJI RS 5: ✅ Enhanced Intelligent Tracking and Composition ✅ Briefcase Handle With Gimbal & Camera Control ✅ 1-Hour Full Charge, 15% Extended Runtime ✅ Fine-Tuning Knobs and Teflon™ Enhanced Balancing ✅ 2nd-Gen Automated Axis Locks ✅ New Z-Axis Indicator Learn more: Buy now: The Enhanced Intelligent Tracking Module and Electronic Briefcase Handle are available for separate purchase or as part of a compatible combo. The DJI RS BG33 Battery Grip is not available for purchase in India. #DJI #DJIRonin #DJIRS5 #RS5 #CameraGimbal #CameraStabilizer #Videography

DJI

62,155 Aufrufe • vor 7 Monaten

Introducing Vtuber's 3D stream gear I'm currently using a 15-point tracking gear called REBOCAP. As you can see in the video, the 3D model moves quite smoothly. Moreover, it can be purchased quite cheaply for about 200 dollars. It seems that many people are sad about the lack of information on 3D streams, so I thought I would share this information. Please share it with many people.

Introducing Vtuber's 3D stream gear I'm currently using a 15-point tracking gear called REBOCAP. As you can see in the video, the 3D model moves quite smoothly. Moreover, it can be purchased quite cheaply for about 200 dollars. It seems that many people are sad about the lack of information on 3D streams, so I thought I would share this information. Please share it with many people.

🍊Oren-chan/加賀オレン🍊＠JPVtuber　2020/07/05debut

270,877 Aufrufe • vor 2 Jahren

Claude Code v2.1.16 adds new task management system with full dependency tracking I demoed this through CC Mirror a few weeks ago and it was extremely powerful Watch the full video - this is going to change the way you work forever Each task can be assigned to sub agents in parallel and they can easily pick up tasks as they become unblocked

Claude Code v2.1.16 adds new task management system with full dependency tracking I demoed this through CC Mirror a few weeks ago and it was extremely powerful Watch the full video - this is going to change the way you work forever Each task can be assigned to sub agents in parallel and they can easily pick up tasks as they become unblocked

Numman Ali

51,261 Aufrufe • vor 8 Monaten

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Bilawal Sidhu

704,318 Aufrufe • vor 1 Jahr

We open-source Action Images — a new representation that translates 7-DoF robot actions into interpretable images. Video models are emerging as powerful robotic foundation models, but a key challenge remains: how can we seamlessly integrate robot policies into video models? Instead of representing actions as low-dimensional control tokens, Action Images provide a pixel-grounded action representation, reframing policy learning as a visual tracking problem! By unifying observations and actions in the same video space, Action Images enable a unified robotics world model that supports video-action joint generation, action-conditioned video generation, and action labeling! Code: Paper:

We open-source Action Images — a new representation that translates 7-DoF robot actions into interpretable images. Video models are emerging as powerful robotic foundation models, but a key challenge remains: how can we seamlessly integrate robot policies into video models? Instead of representing actions as low-dimensional control tokens, Action Images provide a pixel-grounded action representation, reframing policy learning as a visual tracking problem! By unifying observations and actions in the same video space, Action Images enable a unified robotics world model that supports video-action joint generation, action-conditioned video generation, and action labeling! Code: Paper:

Chuang Gan

20,639 Aufrufe • vor 3 Monaten

Tariffs just crashed crypto. Again. I'm sure many are watching the hourly chart, but I'm watching new Philly Fed data that just came out showing a 21-point swing into expansion territory. That's a leading indicator for the PMI cycle I've been tracking. QT ended. Liquidity injections are starting. The macro setup hasn't changed. Be careful to get distracted. Can we dip more? Yes. Is this a bear market? I don't think so. I'm zoomed out and I'm holding.

Tariffs just crashed crypto. Again. I'm sure many are watching the hourly chart, but I'm watching new Philly Fed data that just came out showing a 21-point swing into expansion territory. That's a leading indicator for the PMI cycle I've been tracking. QT ended. Liquidity injections are starting. The macro setup hasn't changed. Be careful to get distracted. Can we dip more? Yes. Is this a bear market? I don't think so. I'm zoomed out and I'm holding.

Dan Gambardello

164,367 Aufrufe • vor 8 Monaten

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion

Bilawal Sidhu

26,863 Aufrufe • vor 4 Monaten

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,869 Aufrufe • vor 1 Jahr

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:

Tencent Hy

20,792 Aufrufe • vor 6 Monaten

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 Aufrufe • vor 3 Jahren

✩ — #เตนิว : #TayNew 📝 Here we go again, similar situation as always. So hungry but no one wanna decide a place "Anything is fine." "What about this?" "No." so let's close the case with rock paper scissors. 🫍 So you represent that restaurant, and you represent for this restaurant. Neng: Wait, which one I want to eat at? 🫍 It has nothing to do with that, you just represent it. Neng: Represent, okay, let's go. 🐻‍❄ Represent the restaurant. You're for this restaurant, me for that restaurant. 🫍 Okay 1, 2... 🐻‍❄ I have to win- If I win... 🫍 No, let the fate decide. 🐻‍❄ Yeah, I mean if I win we go to that one right? 🫍 Yes, if you win, we go there. If you win, we go this. 1, 2... 🐻‍❄ What about you? 🫍 No. I'm the judge. 🐻‍❄ You damn... 🫍 I don't want to have a conflict. 🐻‍❄ Okay [close eyes] 1, 2, 3. [new wins] Are we over yet? 🫍 55555555555555 —— 📝 What's the point of rock paper scissors? 🫍 And we're in... Ginger Farm~ (none in the choices before) 🐻‍❄ What's the point of rock paper scissors? Neng: Why this restaurant? Where's this coming from? 🐻‍❄ What's the point of rock paper scissors? Neng: What's the point of rock paper scissors? 🫍 [😆]

✩ — #เตนิว : #TayNew 📝 Here we go again, similar situation as always. So hungry but no one wanna decide a place "Anything is fine." "What about this?" "No." so let's close the case with rock paper scissors. 🫍 So you represent that restaurant, and you represent for this restaurant. Neng: Wait, which one I want to eat at? 🫍 It has nothing to do with that, you just represent it. Neng: Represent, okay, let's go. 🐻‍❄ Represent the restaurant. You're for this restaurant, me for that restaurant. 🫍 Okay 1, 2... 🐻‍❄ I have to win- If I win... 🫍 No, let the fate decide. 🐻‍❄ Yeah, I mean if I win we go to that one right? 🫍 Yes, if you win, we go there. If you win, we go this. 1, 2... 🐻‍❄ What about you? 🫍 No. I'm the judge. 🐻‍❄ You damn... 🫍 I don't want to have a conflict. 🐻‍❄ Okay [close eyes] 1, 2, 3. [new wins] Are we over yet? 🫍 55555555555555 —— 📝 What's the point of rock paper scissors? 🫍 And we're in... Ginger Farm~ (none in the choices before) 🐻‍❄ What's the point of rock paper scissors? Neng: Why this restaurant? Where's this coming from? 🐻‍❄ What's the point of rock paper scissors? Neng: What's the point of rock paper scissors? 🫍 [😆]

deeコ

16,253 Aufrufe • vor 13 Tagen

Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors paper page: Most 3D generation research focuses on up-projecting 2D foundation models into the 3D space, either by minimizing 2D Score Distillation Sampling (SDS) loss or fine-tuning on multi-view datasets. Without explicit 3D priors, these methods often lead to geometric anomalies and multi-view inconsistency. Recently, researchers have attempted to improve the genuineness of 3D objects by directly training on 3D datasets, albeit at the cost of low-quality texture generation due to the limited texture diversity in 3D datasets. To harness the advantages of both approaches, we propose Bidirectional Diffusion(BiDiff), a unified framework that incorporates both a 3D and a 2D diffusion process, to preserve both 3D fidelity and 2D texture richness, respectively. Moreover, as a simple combination may yield inconsistent generation results, we further bridge them with novel bidirectional guidance. In addition, our method can be used as an initialization of optimization-based models to further improve the quality of 3D model and efficiency of optimization, reducing the generation process from 3.4 hours to 20 minutes. Experimental results have shown that our model achieves high-quality, diverse, and scalable 3D generation.

Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors paper page: Most 3D generation research focuses on up-projecting 2D foundation models into the 3D space, either by minimizing 2D Score Distillation Sampling (SDS) loss or fine-tuning on multi-view datasets. Without explicit 3D priors, these methods often lead to geometric anomalies and multi-view inconsistency. Recently, researchers have attempted to improve the genuineness of 3D objects by directly training on 3D datasets, albeit at the cost of low-quality texture generation due to the limited texture diversity in 3D datasets. To harness the advantages of both approaches, we propose Bidirectional Diffusion(BiDiff), a unified framework that incorporates both a 3D and a 2D diffusion process, to preserve both 3D fidelity and 2D texture richness, respectively. Moreover, as a simple combination may yield inconsistent generation results, we further bridge them with novel bidirectional guidance. In addition, our method can be used as an initialization of optimization-based models to further improve the quality of 3D model and efficiency of optimization, reducing the generation process from 3.4 hours to 20 minutes. Experimental results have shown that our model achieves high-quality, diverse, and scalable 3D generation.

AK

38,853 Aufrufe • vor 2 Jahren