Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

💥 Think more real data is needed for scene reconstruction? Think again! Meet MegaSynth: scaling up feed-forward 3D scene reconstruction with synthesized scenes. In 3 days, it generates 700K scenes for training—70x larger than real data! ✨ The secret? Reconstruction is mostly non-semantic! No need to rely heavily on...

hanwenjiang1's profile picture

Hanwen Jiang

1,989 subscribers

26,969 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

🔍 How does MegaSynth work? MegaSynth focuses on basic geometric structures, using augmented non-semantic shape primitives combined with randomized lighting and materials. It enhances data scalability, control, diversity, and provides accurate metadata for training models. (2/4)

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

📈 MegaSynth delivers results! Training with MegaSynth consistently improves performance across models, testing scenarios, and training settings. It enhances handling of complex lighting, materials, thin structures, and cluttered scenes—highlighting the power of synthesized data! (3/4)

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

🤯 Surprising insight. Training with zero real data performs comparably, confirming that multi-view reconstruction is largely non-semantic and low-level—aligning with observations from optimization-based methods like NeRF and COLMAP. (4/4)

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

Joint work with @zexiangxu , @DesaiXie , @chenziwee , @Haian_Jin , @fujun_luan , Zhixin Shu, @KaiZhang9546 , @Sai__Bi , Xin Sun, Jiuxiang Gu, @qixing_huang , @geopavlakos and @HaoTan5

Profilbild von Jianyuan Wang
Jianyuan Wangvor 1 Jahr

Awesome! Do you happen to have an estimated timeline for the release of the data?

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

Thanks for your interest. It should be very quick, probably in January

Profilbild von Dusan Svilarkovic
Dusan Svilarkovicvor 1 Jahr

What license are you using for datasets ?

Profilbild von Jonathan Clark
Jonathan Clarkvor 1 Jahr

Looks really interesting! Would it be able to handle object centric reconstruction too?

Profilbild von Hanwen Jiang
Hanwen Jiangvor 1 Jahr

Yes it works. Object-centric is easier 😁

Profilbild von Thuan Hoang Nguyen
Thuan Hoang Nguyenvor 1 Jahr

How about real+synthetic combined ? Can it boost performance further ?

Ähnliche Videos

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:
0:50

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,563 Aufrufe • vor 9 Monaten

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,346 Aufrufe • vor 1 Jahr

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:
2:13

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:

Siyuan Huang

12,612 Aufrufe • vor 2 Monaten

Feed-forward 3D Gaussians from Visual Geometry Group (VGG) strike again! Flash3D has now been accepted to 3DV 2025: it is a method for feed-forward single-view 3D scene reconstruction. Project page: Code: Arxiv: A 🧵👇
0:27

Feed-forward 3D Gaussians from Visual Geometry Group (VGG) strike again! Flash3D has now been accepted to 3DV 2025: it is a method for feed-forward single-view 3D scene reconstruction. Project page: Code: Arxiv: A 🧵👇

Stan Szymanowicz

26,970 Aufrufe • vor 1 Jahr

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.
0:48

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

779,640 Aufrufe • vor 1 Monat

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction
0:23

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction

Zhiwen(Aaron) Fan

43,871 Aufrufe • vor 1 Jahr

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation Contributions: • We propose a novel approach to inject SfM priors into diffusion-based depth estimation, enabling highly accurate and multi-view consistent depth predictions for each viewpoint. • Based on the proposed depth estimator, we design a new multi-view 3D geometry reconstruction framework and process some synthetic datasets to facilitate training. • We evaluate our method on diverse real-world scene data, including objects, indoor environments, streetscapes, and aerial scenes, demonstrating the superior performance and generalization capability of our approach.
0:30

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation Contributions: • We propose a novel approach to inject SfM priors into diffusion-based depth estimation, enabling highly accurate and multi-view consistent depth predictions for each viewpoint. • Based on the proposed depth estimator, we design a new multi-view 3D geometry reconstruction framework and process some synthetic datasets to facilitate training. • We evaluate our method on diverse real-world scene data, including objects, indoor environments, streetscapes, and aerial scenes, demonstrating the superior performance and generalization capability of our approach.

MrNeRF

25,651 Aufrufe • vor 1 Jahr

Scale AI's Alexandr Wang says "the data wall and the wall on progress that we've hit right now is certainly real" and synthetic data isn't living up to its promise, with more human-generated data needed
2:06

Scale AI's Alexandr Wang says "the data wall and the wall on progress that we've hit right now is certainly real" and synthetic data isn't living up to its promise, with more human-generated data needed

Tsarathustra

578,870 Aufrufe • vor 1 Jahr

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗
0:47

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,840 Aufrufe • vor 2 Monaten

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.
1:38

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Robots Digest 🤖

10,362 Aufrufe • vor 4 Monaten

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.
0:31

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.

MrNeRF

53,292 Aufrufe • vor 1 Jahr

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.
0:31

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.

Shubham Tulsiani

16,406 Aufrufe • vor 4 Monaten

Our product team walks through the full journey: Capture → Reconstruct → Localize Scaniverse creates high-fidelity 3D inputs from the real world—built for reconstruction, localization, and semantic understanding. Watch the full YouTube playlist: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI
1:26

Our product team walks through the full journey: Capture → Reconstruct → Localize Scaniverse creates high-fidelity 3D inputs from the real world—built for reconstruction, localization, and semantic understanding. Watch the full YouTube playlist: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI

Niantic Spatial 🌎

10,949 Aufrufe • vor 2 Monaten

📢 GenRecon Code Release 📢 Few images in → complete, high-fidelity 3D scene out! GenRecon builds a generative prior on full scenes, resulting in unprecedented 3D reconstruction quality. 🔗 🌐 📄
0:20

📢 GenRecon Code Release 📢 Few images in → complete, high-fidelity 3D scene out! GenRecon builds a generative prior on full scenes, resulting in unprecedented 3D reconstruction quality. 🔗 🌐 📄

Matthias Niessner

38,772 Aufrufe • vor 8 Tagen

NEWS: Nvidia shows how its new “Cosmos” system uses cars to generate synthetic real world driving scenarios to train self-driving cars. It can create drivable 3D environments. Driving scenario variations can be generated from replayed drive logs or AI traffic generators. Next, a neural reconstruction engine uses autonomous vehicle sensor logs to create high fidelity 4D simulation environments. It replays previous drives in 3D and generates scenario variations to amplify training data. Nvidia’s AI data factory scales hundreds of drives into billions of effective miles.
2:41

NEWS: Nvidia shows how its new “Cosmos” system uses cars to generate synthetic real world driving scenarios to train self-driving cars. It can create drivable 3D environments. Driving scenario variations can be generated from replayed drive logs or AI traffic generators. Next, a neural reconstruction engine uses autonomous vehicle sensor logs to create high fidelity 4D simulation environments. It replays previous drives in 3D and generates scenario variations to amplify training data. Nvidia’s AI data factory scales hundreds of drives into billions of effective miles.

Sawyer Merritt

918,239 Aufrufe • vor 1 Jahr

WaterSplatting: Fast Underwater 3D Scene Reconstruction Using Gaussian Splatting Project: Method ⬇️ 1 I 2
1:07

WaterSplatting: Fast Underwater 3D Scene Reconstruction Using Gaussian Splatting Project: Method ⬇️ 1 I 2

MrNeRF

12,175 Aufrufe • vor 1 Jahr

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.
0:36

Large-scale 3D Scene Generation (all scenes are real-time rendered)!! Physically-grounded generative data without hallucinations is the missing link for robot learning and testing at scale. We introduce a method that directly generates large-scale 3D driving scenes with accurate geometry, allowing for causal view synthesis and generation with object permanence and explicit 3D geometry. This also allows for extreme trajectory extrapolation without failure! We also show that we can build fully data-driven simulators for end-to-end learning with this approach. Project: with the amazing team of Julian Ost, Amogh Joshi , Andrea Ramazzina, Maximilian Bömer, Mario Bijelic.

Felix Heide

27,763 Aufrufe • vor 10 Monaten

🚀 Introducing Salesforce's game-changer: Data Cloud Vector Database + Einstein Copilot Semantic Search! ✨ Harness both structured & unstructured data for smarter AI responses. No more costly training! Real-time data-driven insights with RAG, complete with citations to curb AI hallucinations. Trust in enterprise AI just got a major upgrade, thanks to Salesforce! ❤️
4:37

🚀 Introducing Salesforce's game-changer: Data Cloud Vector Database + Einstein Copilot Semantic Search! ✨ Harness both structured & unstructured data for smarter AI responses. No more costly training! Real-time data-driven insights with RAG, complete with citations to curb AI hallucinations. Trust in enterprise AI just got a major upgrade, thanks to Salesforce! ❤️

Marc Benioff

113,095 Aufrufe • vor 2 Jahren

If the Fed is data dependent and the data is bad, make-up cuts are needed. President Donald J. Trump showed the system was broken, and now it’s time to fix it with real data. E.J. Antoni, Ph.D. is the right leader for BLS with integrity, data-driven focus, and a relentless commitment to pursuing the truth.
1:00

If the Fed is data dependent and the data is bad, make-up cuts are needed. President Donald J. Trump showed the system was broken, and now it’s time to fix it with real data. E.J. Antoni, Ph.D. is the right leader for BLS with integrity, data-driven focus, and a relentless commitment to pursuing the truth.

Treasury Secretary Scott Bessent

524,351 Aufrufe • vor 10 Monaten