Загрузка видео...

Не удалось загрузить видео

На главную

💥 Think more real data is needed for scene reconstruction? Think again! Meet MegaSynth: scaling up feed-forward 3D scene reconstruction with synthesized scenes. In 3 days, it generates 700K scenes for training—70x larger than real data! ✨ The secret? Reconstruction is mostly non-semantic! No need to rely heavily on...

26,969 просмотров • 1 год назад •via X (Twitter)

Комментарии: 10

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

🔍 How does MegaSynth work? MegaSynth focuses on basic geometric structures, using augmented non-semantic shape primitives combined with randomized lighting and materials. It enhances data scalability, control, diversity, and provides accurate metadata for training models. (2/4)

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

📈 MegaSynth delivers results! Training with MegaSynth consistently improves performance across models, testing scenarios, and training settings. It enhances handling of complex lighting, materials, thin structures, and cluttered scenes—highlighting the power of synthesized data! (3/4)

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

🤯 Surprising insight. Training with zero real data performs comparably, confirming that multi-view reconstruction is largely non-semantic and low-level—aligning with observations from optimization-based methods like NeRF and COLMAP. (4/4)

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

Joint work with @zexiangxu , @DesaiXie , @chenziwee , @Haian_Jin , @fujun_luan , Zhixin Shu, @KaiZhang9546 , @Sai__Bi , Xin Sun, Jiuxiang Gu, @qixing_huang , @geopavlakos and @HaoTan5

Фото профиля Jianyuan Wang
Jianyuan Wang1 год назад

Awesome! Do you happen to have an estimated timeline for the release of the data?

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

Thanks for your interest. It should be very quick, probably in January

Фото профиля Dusan Svilarkovic
Dusan Svilarkovic1 год назад

What license are you using for datasets ?

Фото профиля Jonathan Clark
Jonathan Clark1 год назад

Looks really interesting! Would it be able to handle object centric reconstruction too?

Фото профиля Hanwen Jiang
Hanwen Jiang1 год назад

Yes it works. Object-centric is easier 😁

Фото профиля Thuan Hoang Nguyen
Thuan Hoang Nguyen1 год назад

How about real+synthetic combined ? Can it boost performance further ?

Похожие видео

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,199 просмотров • 2 лет назад