Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

The adding of the panettone got me. Panetonnes all year round please in NeRFs and the real world!

Thanks a lot for featuring our work @_akhaliq! Here is the project page for more details and results:

I bet this works as good as most google AI products we’ve seen demo(ed)

Existing methods for 3D scene editing are mostly effective for style and appearance changes or removing objects. But generating new objects is a challenge for them. InseRF addresses this by combining advances in NeRFs with advances in generative AI and also shows potential for future improvements in generative 2D and 3D models.

@adyaman

If real, that's the future of video. Starting from a blank scene, adding assets, then actors and animating them while being able to move the camera where you want.

what the

What is the big deal my bro

"Add a panettone on the tray" is an odd prompt if you're already drawing a bounding box over the tray. The figure in the paper makes it look like the actual prompt would just be "a panettone".

Nice, but perhaps they need to use the existing scene to do a little bit of global illumination and environment mapping from the scene to the inserted object?