正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

💥 Think more real data is needed for scene reconstruction? Think again! Meet MegaSynth: scaling up feed-forward 3D scene reconstruction with synthesized scenes. In 3 days, it generates 700K scenes for training—70x larger than real data! ✨ The secret? Reconstruction is mostly non-semantic! No need to rely heavily on... show more

Hanwen Jiang

2,309 subscribers

27,343 次观看 • 1 年前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

10 条评论

Hanwen Jiang 的头像

Hanwen Jiang1 年前

🔍 How does MegaSynth work? MegaSynth focuses on basic geometric structures, using augmented non-semantic shape primitives combined with randomized lighting and materials. It enhances data scalability, control, diversity, and provides accurate metadata for training models. (2/4)

Hanwen Jiang 的头像

Hanwen Jiang1 年前

📈 MegaSynth delivers results! Training with MegaSynth consistently improves performance across models, testing scenarios, and training settings. It enhances handling of complex lighting, materials, thin structures, and cluttered scenes—highlighting the power of synthesized data! (3/4)

Hanwen Jiang 的头像

Hanwen Jiang1 年前

🤯 Surprising insight. Training with zero real data performs comparably, confirming that multi-view reconstruction is largely non-semantic and low-level—aligning with observations from optimization-based methods like NeRF and COLMAP. (4/4)

Hanwen Jiang 的头像

Hanwen Jiang1 年前

Joint work with @zexiangxu , @DesaiXie , @chenziwee , @Haian_Jin , @fujun_luan , Zhixin Shu, @KaiZhang9546 , @Sai__Bi , Xin Sun, Jiuxiang Gu, @qixing_huang , @geopavlakos and @HaoTan5

Jianyuan Wang 的头像

Jianyuan Wang1 年前

Awesome! Do you happen to have an estimated timeline for the release of the data?

Hanwen Jiang 的头像

Hanwen Jiang1 年前

Thanks for your interest. It should be very quick, probably in January

Dusan Svilarkovic 的头像

Dusan Svilarkovic1 年前

What license are you using for datasets ?

Jonathan Clark 的头像

Jonathan Clark1 年前

Looks really interesting! Would it be able to handle object centric reconstruction too?

Hanwen Jiang 的头像

Hanwen Jiang1 年前

Yes it works. Object-centric is easier 😁

Thuan Hoang Nguyen 的头像

Thuan Hoang Nguyen1 年前

How about real+synthetic combined ? Can it boost performance further ?

相关视频

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,960 次观看 • 1 年前

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 次观看 • 1 年前

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:

Scaling 3D scene data is a long-standing challenge in scene understanding, spatial reasoning, and robotics. Since scanning, reconstruction, and labeling are so labor-intensive, data scarcity has remained a major bottleneck. 🛑 To solve this, we propose SceneVerse++: Lifting Unlabeled Internet-level Data for 3D Scene Understanding (CVPR 2026). By reconstructing internet videos and annotating 3D scenes automatically, we’ve created a massive real-world dataset for end-to-end understanding. 🌐📐 SceneVerse++ makes it easy to scale "in-the-wild" 3D scenes toward more capable spatial reasoning systems. This significantly promotes progress in 3D VQA, visual navigation, and broader tasks in Embodied AI and Robotics. 🤖🦾 We are fully open-sourced! Check out the paper, code, and data here: 🌐 Project: 📄 Paper: 📊 Dataset: Code:

Siyuan Huang

12,857 次观看 • 6 个月前

Feed-forward 3D Gaussians from Visual Geometry Group (VGG) strike again! Flash3D has now been accepted to 3DV 2025: it is a method for feed-forward single-view 3D scene reconstruction. Project page: Code: Arxiv: A 🧵👇

Feed-forward 3D Gaussians from Visual Geometry Group (VGG) strike again! Flash3D has now been accepted to 3DV 2025: it is a method for feed-forward single-view 3D scene reconstruction. Project page: Code: Arxiv: A 🧵👇

Stan Szymanowicz

26,994 次观看 • 1 年前

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

784,854 次观看 • 5 个月前

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction

Zhiwen(Aaron) Fan

44,051 次观看 • 2 年前

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

As robotics teams scale, collecting real-world data can quickly become slow and expensive. 📉 Lightwheel is changing that with a simulation-first platform built on our technology, helping customers generate a 100:1 simulated-to-real data ratio for training autonomous systems. Together, we’re turning synthetic and real data into more capable, more reliable robots. Read the full success story 🔗

NVIDIA Robotics

10,928 次观看 • 7 个月前

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Robots Digest 🤖

10,425 次观看 • 8 个月前

BREAKING: Elon Musk says SpaceX and Tesla data, could make Grok excellent at real-world engineering. “We've got a real opportunity with the SpaceX training data and maybe with the Tesla training data to make AI extremely capable in real-world engineering. No one has yet made AI excellent at real-world engineering. And I think that's an opportunity for Grok using the training data from SpaceX and Tesla.”

BREAKING: Elon Musk says SpaceX and Tesla data, could make Grok excellent at real-world engineering. “We've got a real opportunity with the SpaceX training data and maybe with the Tesla training data to make AI extremely capable in real-world engineering. No one has yet made AI excellent at real-world engineering. And I think that's an opportunity for Grok using the training data from SpaceX and Tesla.”

DogeDesigner

66,090 次观看 • 1 个月前

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.

MrNeRF

53,292 次观看 • 1 年前

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.

[1/N] Current visual geometry prediction models primarily rely on labeled 3D data. Our CVPR26 paper, Flow3r, allows additionally leveraging unlabeled videos (using flow supervision) for scalable visual geometry learning, enabling accurate multi-view 3D reconstruction in-the-wild.

Shubham Tulsiani

17,487 次观看 • 8 个月前

NEWS: Nvidia shows how its new “Cosmos” system uses cars to generate synthetic real world driving scenarios to train self-driving cars. It can create drivable 3D environments. Driving scenario variations can be generated from replayed drive logs or AI traffic generators. Next, a neural reconstruction engine uses autonomous vehicle sensor logs to create high fidelity 4D simulation environments. It replays previous drives in 3D and generates scenario variations to amplify training data. Nvidia’s AI data factory scales hundreds of drives into billions of effective miles.

NEWS: Nvidia shows how its new “Cosmos” system uses cars to generate synthetic real world driving scenarios to train self-driving cars. It can create drivable 3D environments. Driving scenario variations can be generated from replayed drive logs or AI traffic generators. Next, a neural reconstruction engine uses autonomous vehicle sensor logs to create high fidelity 4D simulation environments. It replays previous drives in 3D and generates scenario variations to amplify training data. Nvidia’s AI data factory scales hundreds of drives into billions of effective miles.

Sawyer Merritt

918,467 次观看 • 1 年前

LEGO-Anything Coding Agents for 3D Scene Reconstruction paper:

LEGO-Anything Coding Agents for 3D Scene Reconstruction paper:

AK

100,450 次观看 • 1 个月前

📢 GenRecon Code Release 📢 Few images in → complete, high-fidelity 3D scene out! GenRecon builds a generative prior on full scenes, resulting in unprecedented 3D reconstruction quality. 🔗 🌐 📄

📢 GenRecon Code Release 📢 Few images in → complete, high-fidelity 3D scene out! GenRecon builds a generative prior on full scenes, resulting in unprecedented 3D reconstruction quality. 🔗 🌐 📄

Matthias Niessner

40,586 次观看 • 4 个月前

Our product team walks through the full journey: Capture → Reconstruct → Localize Scaniverse creates high-fidelity 3D inputs from the real world—built for reconstruction, localization, and semantic understanding. Watch the full YouTube playlist: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI

Our product team walks through the full journey: Capture → Reconstruct → Localize Scaniverse creates high-fidelity 3D inputs from the real world—built for reconstruction, localization, and semantic understanding. Watch the full YouTube playlist: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI

Niantic Spatial 🌎

10,949 次观看 • 6 个月前

3D scenes, built locally. Describe what you want in plain language. Edge0 can generate a structured 3D scene, then modify and refine it on your phone or PC. No project data leaves the device.

3D scenes, built locally. Describe what you want in plain language. Edge0 can generate a structured 3D scene, then modify and refine it on your phone or PC. No project data leaves the device.

Samuel Zeng

21,642 次观看 • 2 个月前

introducing by Raindrop test your agent on every PR, against real scenarios, with one-click. we handle all the set up: sandboxing, fixtures, real data, synthetic data, evals and more.

introducing by Raindrop test your agent on every PR, against real scenarios, with one-click. we handle all the set up: sandboxing, fixtures, real data, synthetic data, evals and more.

ben hylak

80,154 次观看 • 5 天前