Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing Generative Omnimatte: A method for decomposing a video into complete layers, including objects and their associated effects (e.g., shadows, reflections). It enables many cool applications, such as video stylization, compositions, moment retiming, and object removal.

jbhuang0604's profile picture

Jia-Bin Huang

78,607 subscribers

81,713 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Jia-Bin Huang
Jia-Bin Huangvor 1 Jahr

Watch the explainer video to learn more! I experimented with new editing styles for this video and had a lot of fun making it. I hope you like it.

Profilbild von Jia-Bin Huang
Jia-Bin Huangvor 1 Jahr

Check out the project website for many more examples and comparisons.

Profilbild von Jia-Bin Huang
Jia-Bin Huangvor 1 Jahr

Brought to you by the amazing @YaoChihLee and friends. Check out Yao-Chih's post for a more detailed breakdown of this exciting work.

Profilbild von Dan Benyamin (Æ)
Dan Benyamin (Æ)vor 1 Jahr

So cool! Will you be releasing the code?

Profilbild von FunTresting Things :)
FunTresting Things :)vor 1 Jahr

Cool. Is that open source?

Profilbild von Walker Hawker 🦎🦅🔜 IWAG
Walker Hawker 🦎🦅🔜 IWAGvor 1 Jahr

Rotoscoping made easier

Profilbild von Divyansh Tripathi
Divyansh Tripathivor 1 Jahr

AD and marketing industry seems to be its biggest consumer

Profilbild von Designer J
Designer Jvor 1 Jahr

This is such a huge game changer for video editing.

Profilbild von Alexandre Morgand
Alexandre Morgandvor 1 Jahr

Beautiful and awesome video. Hopefully I can give it a spin one day 👌

Profilbild von Andu 🇺🇦🇪🇺
Andu 🇺🇦🇪🇺vor 1 Jahr

Cool idea!

Ähnliche Videos

🚀🚨🤖Adobe revealed the integration of four innovative generative AI features: Object Addition, Object Removal, Generative Extend, and Text Video Supporting an AI open ecosystem for third-party plugins, such as OpenAI Pika and Runway Highlighting commitment to responsible AI, focusing on ethically guided training for commercial applications. Video: Adobe Creative Cloud
4:40

🚀🚨🤖Adobe revealed the integration of four innovative generative AI features: Object Addition, Object Removal, Generative Extend, and Text Video Supporting an AI open ecosystem for third-party plugins, such as OpenAI Pika and Runway Highlighting commitment to responsible AI, focusing on ethically guided training for commercial applications. Video: Adobe Creative Cloud

Teodora P L

13,644 Aufrufe • vor 2 Jahren

"ROSE: Remove Objects with Side Effects in Videos" TL;DR: Diffusion transformer; five common cases: shadows, reflections, light, translucency and mirror as video side effects to remove
0:45

"ROSE: Remove Objects with Side Effects in Videos" TL;DR: Diffusion transformer; five common cases: shadows, reflections, light, translucency and mirror as video side effects to remove

Alexandre Morgand

85,858 Aufrufe • vor 9 Monaten

🚀 Introducing DiffusionRenderer, a neural rendering engine powered by video diffusion models. 🎥 Estimates high-quality geometry and materials from videos, synthesizes photorealistic light transport, enables relighting and material editing with realistic shadows and reflections
2:14

🚀 Introducing DiffusionRenderer, a neural rendering engine powered by video diffusion models. 🎥 Estimates high-quality geometry and materials from videos, synthesizes photorealistic light transport, enables relighting and material editing with realistic shadows and reflections

Zian Wang

125,321 Aufrufe • vor 1 Jahr

A paper in Nature Communications describes a detachable robotic hand that can crawl and grab objects. The design enables tasks such as retrieving objects beyond normal reach and performing multi-object handling.
1:24

A paper in Nature Communications describes a detachable robotic hand that can crawl and grab objects. The design enables tasks such as retrieving objects beyond normal reach and performing multi-object handling.

Nature Portfolio

28,358 Aufrufe • vor 4 Monaten

Introducing 3D Projections & Noise Effects - Wrap layers into 3D objects - Perspective and lighting - Noise Glass + Distortion Available on Spline Hana now.
0:36

Introducing 3D Projections & Noise Effects - Wrap layers into 3D objects - Perspective and lighting - Noise Glass + Distortion Available on Spline Hana now.

Spline

63,222 Aufrufe • vor 10 Monaten

Track Anything: Segment Anything Meets Videos Track-Anything is a flexible and interactive tool for video object tracking and segmentation suitable for: - Video object tracking and segmentation with shot changes. - Visualized development and data annnotation for video object tracking and segmentation. - Object-centric downstream video tasks, such as video inpainting and editing. abs: github:
0:22

Track Anything: Segment Anything Meets Videos Track-Anything is a flexible and interactive tool for video object tracking and segmentation suitable for: - Video object tracking and segmentation with shot changes. - Visualized development and data annnotation for video object tracking and segmentation. - Object-centric downstream video tasks, such as video inpainting and editing. abs: github:

AK

578,577 Aufrufe • vor 3 Jahren

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:
1:00

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 Aufrufe • vor 1 Jahr

We're thrilled to announce breakthrough #GenerativeAI features powered by the new #AdobeFirefly video model. Take a sneak peek at Object Addition, Object Removal and Generative Extend. All coming soon to #PremierePro! 💥
0:59

We're thrilled to announce breakthrough #GenerativeAI features powered by the new #AdobeFirefly video model. Take a sneak peek at Object Addition, Object Removal and Generative Extend. All coming soon to #PremierePro! 💥

Adobe

624,528 Aufrufe • vor 2 Jahren

OmnimatteZero. Training-free video editing. Pulls foreground objects (shadows/reflections) and reconstructs backgrounds in real-time. - LTX-Video/Wan2.1, ~24fps on an A100. Finally, clean background without manual rotoscoping.
0:38

OmnimatteZero. Training-free video editing. Pulls foreground objects (shadows/reflections) and reconstructs backgrounds in real-time. - LTX-Video/Wan2.1, ~24fps on an A100. Finally, clean background without manual rotoscoping.

Wildminder

38,917 Aufrufe • vor 4 Monaten

Introducing Runway Aleph, a new way to edit, transform and generate video. Aleph is a state-of-the-art in-context video model, setting a new frontier for multi-task visual generation, with the ability to perform a wide range of edits on an input video such as adding, removing and transforming objects, getting new angles of a scene and modifying style and lighting, among many other tasks.
1:06

Introducing Runway Aleph, a new way to edit, transform and generate video. Aleph is a state-of-the-art in-context video model, setting a new frontier for multi-task visual generation, with the ability to perform a wide range of edits on an input video such as adding, removing and transforming objects, getting new angles of a scene and modifying style and lighting, among many other tasks.

Runway

646,259 Aufrufe • vor 10 Monaten

ObjectClear can remove objects from images while also getting rid of shadows and reflections
0:27

ObjectClear can remove objects from images while also getting rid of shadows and reflections

Dreaming Tulpa 🥓👑

26,692 Aufrufe • vor 1 Jahr

Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:
0:27

Spatial AI is increasingly important, and the newest papers from #NVIDIAResearch, 3DGRT and 3DGUT, represent significant advancements in enabling researchers and developers to explore and innovate with 3D Gaussian Splatting techniques. 💎 3DGRT (Gaussian Ray Tracing) ➡️ - Enables secondary lighting effects such as refractions, shadows, depth of field, and mirrors. - Provides a foundation for further research in ray tracing-based rendering of Gaussian representations. - Achieves real-time rendering performance on modern GPUs. ✨ 3DGUT (Gaussain Unscented Transform) ➡️ - Support for distorted camera models and time-dependent effects like rolling shutter. - Enables phenomena such as reflections and refractions while retaining real-time efficiency. 📥 Code:

NVIDIA AI Developer

27,724 Aufrufe • vor 1 Jahr

Introducing UWPSpy, an inspection tool for UWP applications that enables to view and manipulate UI elements and their properties in real time! Download: Source code:
1:40

Introducing UWPSpy, an inspection tool for UWP applications that enables to view and manipulate UI elements and their properties in real time! Download: Source code:

Michael Maltsev

23,018 Aufrufe • vor 2 Jahren

Introducing 𝐒𝐀𝐑𝟑𝐃, which tokenizes 3D objects into multiscale tokens and generates 3D objects by autoregressive next-scale prediction. 𝐒𝐀𝐑𝟑𝐃 enables fast 3D generation and comprehensive 3D understanding. arXiv: Project:
0:20

Introducing 𝐒𝐀𝐑𝟑𝐃, which tokenizes 3D objects into multiscale tokens and generates 3D objects by autoregressive next-scale prediction. 𝐒𝐀𝐑𝟑𝐃 enables fast 3D generation and comprehensive 3D understanding. arXiv: Project:

Xingang Pan

28,248 Aufrufe • vor 1 Jahr

adobe is cooking a new inpainting method that understands context extremely well! ELI5: it can insert stuff with the correct perspective and add/remove connected reflections/shadows 👀
0:55

adobe is cooking a new inpainting method that understands context extremely well! ELI5: it can insert stuff with the correct perspective and add/remove connected reflections/shadows 👀

Dreaming Tulpa 🥓👑

58,637 Aufrufe • vor 1 Jahr

Introducing TAPIR & RoboTAP, our latest research from Google DeepMind. It focuses on spatial intelligence via point tracking, outlining how it enables applications from robotics to video generation to augmented reality, and more!
0:20

Introducing TAPIR & RoboTAP, our latest research from Google DeepMind. It focuses on spatial intelligence via point tracking, outlining how it enables applications from robotics to video generation to augmented reality, and more!

Carl Doersch

47,833 Aufrufe • vor 2 Jahren

and just for fun. heres one of the effects layers for this video that i thought looked really cool. basically i set the shutter in blender to 4 (max is usually 1) and it looked really fucked up.
0:20

and just for fun. heres one of the effects layers for this video that i thought looked really cool. basically i set the shutter in blender to 4 (max is usually 1) and it looked really fucked up.

Sparky1j

24,439 Aufrufe • vor 11 Monaten

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:
0:46

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,363 Aufrufe • vor 1 Jahr

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without *any* robot teleop data collection 1/n
0:49

Humans grasp objects with a purpose! Web2Grasp enables such functional grasping for dexterous robot hands via hand-object reconstruction from web images - without *any* robot teleop data collection 1/n

Homanga Bharadhwaj @ CVPR

28,408 Aufrufe • vor 1 Jahr

The Gods Must Be Crazy (1980) is such a bizarrely clever setup, a low contact tribe finding a Coca-Cola bottle and slowly viewing it as this object bringing jealousy, conflict, and disorder into their world.
1:05

The Gods Must Be Crazy (1980) is such a bizarrely clever setup, a low contact tribe finding a Coca-Cola bottle and slowly viewing it as this object bringing jealousy, conflict, and disorder into their world.

cinesthetic.

4,757,500 Aufrufe • vor 1 Monat