正在加载视频...

视频加载失败

💥 Think more real data is needed for scene reconstruction? Think again! Meet MegaSynth: scaling up feed-forward 3D scene reconstruction with synthesized scenes. In 3 days, it generates 700K scenes for training—70x larger than real data! ✨ The secret? Reconstruction is mostly non-semantic! No need to rely heavily on...

26,969 次观看 • 1 年前 •via X (Twitter)

10 条评论

Hanwen Jiang 的头像
Hanwen Jiang1 年前

🔍 How does MegaSynth work? MegaSynth focuses on basic geometric structures, using augmented non-semantic shape primitives combined with randomized lighting and materials. It enhances data scalability, control, diversity, and provides accurate metadata for training models. (2/4)

Hanwen Jiang 的头像
Hanwen Jiang1 年前

📈 MegaSynth delivers results! Training with MegaSynth consistently improves performance across models, testing scenarios, and training settings. It enhances handling of complex lighting, materials, thin structures, and cluttered scenes—highlighting the power of synthesized data! (3/4)

Hanwen Jiang 的头像
Hanwen Jiang1 年前

🤯 Surprising insight. Training with zero real data performs comparably, confirming that multi-view reconstruction is largely non-semantic and low-level—aligning with observations from optimization-based methods like NeRF and COLMAP. (4/4)

Hanwen Jiang 的头像
Hanwen Jiang1 年前

Joint work with @zexiangxu , @DesaiXie , @chenziwee , @Haian_Jin , @fujun_luan , Zhixin Shu, @KaiZhang9546 , @Sai__Bi , Xin Sun, Jiuxiang Gu, @qixing_huang , @geopavlakos and @HaoTan5

Jianyuan Wang 的头像
Jianyuan Wang1 年前

Awesome! Do you happen to have an estimated timeline for the release of the data?

Hanwen Jiang 的头像
Hanwen Jiang1 年前

Thanks for your interest. It should be very quick, probably in January

Dusan Svilarkovic 的头像
Dusan Svilarkovic1 年前

What license are you using for datasets ?

Jonathan Clark 的头像
Jonathan Clark1 年前

Looks really interesting! Would it be able to handle object centric reconstruction too?

Hanwen Jiang 的头像
Hanwen Jiang1 年前

Yes it works. Object-centric is easier 😁

Thuan Hoang Nguyen 的头像
Thuan Hoang Nguyen1 年前

How about real+synthetic combined ? Can it boost performance further ?

相关视频

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,199 次观看 • 2 年前