Загрузка видео...

Не удалось загрузить видео

На главную

#ECCV24 We introduce Controllable Human-Object Interaction Synthesis (CHOIS) to generate interactions given text in a 3D scene. w alex, Roozbeh Mottaghi Jiajun Wu Xavier Puig Karen. code: project page:

jiaman01's profile picture

Jiaman Li

1,295 subscribers

13,185 просмотров • 1 год назад •via X (Twitter)

ОбразованиеНовости и политикаНаука и технологии#ECCV24
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]
0:15

Checkout ObjectForesight! We tackle the problem of predicting future 3D object trajectories given a past context, purely from human videos of everyday interactions! [1/n]

Homanga Bharadhwaj @ CVPR

16,365 просмотров • 5 месяцев назад

Excited to present GenZI at #CVPR2024! Lei Li introduces GenZI, the first zero-shot approach to creating realistic 3D human-scene interactions by leveraging interaction priors from large VLMs. Code and data on our website!
0:12

Excited to present GenZI at #CVPR2024! Lei Li introduces GenZI, the first zero-shot approach to creating realistic 3D human-scene interactions by leveraging interaction priors from large VLMs. Code and data on our website!

Angela Dai

11,910 просмотров • 2 лет назад

If you’re not in freefall, you’re likely contacting something. Yet 3D human-object interaction (HOI) reconstruction remains underexplored. PICO (#CVPR2025) recovers humans 🏃‍♂️, objects 🏓, and their interactions 👉🍎- all in 3D, from just a single internet image. 1/11
0:12

If you’re not in freefall, you’re likely contacting something. Yet 3D human-object interaction (HOI) reconstruction remains underexplored. PICO (#CVPR2025) recovers humans 🏃‍♂️, objects 🏓, and their interactions 👉🍎- all in 3D, from just a single internet image. 1/11

Shashank Tripathi

24,087 просмотров • 1 год назад

TokenHSI is out on Hugging Face Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization
0:17

TokenHSI is out on Hugging Face Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization

AK

57,563 просмотров • 1 год назад

🛠Made a tool for creators That's Generate 3D text in secs..
0:12

🛠Made a tool for creators That's Generate 3D text in secs..

Eve3D | UEFN

72,498 просмотров • 11 месяцев назад

WeatherEdit: Controllable Weather Editing with 4D Gaussian Field Contributions: 1. Based on our analysis of weather editing characteristics, we introduce WeatherEdit, a comprehensive and efficient framework for realistic and controllable weather generation. Compared with existing methods that focus on either background editing or static weather effects, a progressive 2D-to-4D transformation process in WeatherEdit enhances adaptability across a wider range of scenarios. 2. We introduce an all-in-one adapter to enable a diffusion model for multi-weather (snowy, rainy, and fog) synthesis, along with a Temporal-View attention to ensure consistent editing across multi-frame and multi-view. 3. We design a 4D Gaussian field for weather particle modeling, enabling plausible simulation of raindrops, snowflakes, and fog with controllable severity. 4. We demonstrate WeatherEdit’s effectiveness in generating realistic, consistent, and controllable weather effects in 3D driving scenes, showcasing its applicability to real-world scenarios.
0:13

WeatherEdit: Controllable Weather Editing with 4D Gaussian Field Contributions: 1. Based on our analysis of weather editing characteristics, we introduce WeatherEdit, a comprehensive and efficient framework for realistic and controllable weather generation. Compared with existing methods that focus on either background editing or static weather effects, a progressive 2D-to-4D transformation process in WeatherEdit enhances adaptability across a wider range of scenarios. 2. We introduce an all-in-one adapter to enable a diffusion model for multi-weather (snowy, rainy, and fog) synthesis, along with a Temporal-View attention to ensure consistent editing across multi-frame and multi-view. 3. We design a 4D Gaussian field for weather particle modeling, enabling plausible simulation of raindrops, snowflakes, and fog with controllable severity. 4. We demonstrate WeatherEdit’s effectiveness in generating realistic, consistent, and controllable weather effects in 3D driving scenes, showcasing its applicability to real-world scenarios.

MrNeRF

10,607 просмотров • 11 месяцев назад

DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion abs: project page: github:
0:16

DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion abs: project page: github:

AK

152,818 просмотров • 3 лет назад

🎶Time to start NeRFing audio. Introducing NVAS3D for novel acoustic synthesis in 3D scenes. Watch with AirPods. arxiv: code:
0:17

🎶Time to start NeRFing audio. Introducing NVAS3D for novel acoustic synthesis in 3D scenes. Watch with AirPods. arxiv: code:

Anurag Ranjan

24,702 просмотров • 2 лет назад

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,017 просмотров • 1 год назад

We introduce TurboEdit -- simple text-based image editing in 1/2 sec! We leverage few-step diffusion models, mapping real images into noise with an encoder. Please see our #ECCV2024 paper: Work w/ Zongze Wu, Nick Kolkin, Jon Brandt, Eli Shechtman 1/
0:15

We introduce TurboEdit -- simple text-based image editing in 1/2 sec! We leverage few-step diffusion models, mapping real images into noise with an encoder. Please see our #ECCV2024 paper: Work w/ Zongze Wu, Nick Kolkin, Jon Brandt, Eli Shechtman 1/

Richard Zhang

39,306 просмотров • 1 год назад

UPDATE: Spline's 3D Code API for SwiftUI is GA! Create custom 3D interactions for your iOS/macOS projects. - Trigger events and transitions - Change object properties - Update variables - and more! Try it now.
0:10

UPDATE: Spline's 3D Code API for SwiftUI is GA! Create custom 3D interactions for your iOS/macOS projects. - Trigger events and transitions - Change object properties - Update variables - and more! Try it now.

Spline

20,004 просмотров • 10 месяцев назад

LLMs, trained only on text, might already know more about other modalities than we realized; we just need to find ways elicit it. project page: w/ Phillip Isola and Brian Cheung
0:11

LLMs, trained only on text, might already know more about other modalities than we realized; we just need to find ways elicit it. project page: w/ Phillip Isola and Brian Cheung

Sophie Wang

67,822 просмотров • 8 месяцев назад

📢We introduce “RefFusion”, a novel inpainting method for scenes reconstructed using 3D Gaussian Splatting. 🔗 TLDR: we personalize an image diffusion model to a given reference image and distill its knowledge to 3D through score distillation sampling.
0:11

📢We introduce “RefFusion”, a novel inpainting method for scenes reconstructed using 3D Gaussian Splatting. 🔗 TLDR: we personalize an image diffusion model to a given reference image and distill its knowledge to 3D through score distillation sampling.

Ashkan Mirzaei

34,683 просмотров • 2 лет назад

🎉We are exploring the #Mira project~ - Built a long video dataset #MiraData with structured captions. - Trained #MiraDiT to explore the consistency in long video generation. Hope it will be a supplement to existing text-to-video methods. Project Page:
0:10

🎉We are exploring the #Mira project~ - Built a long video dataset #MiraData with structured captions. - Trained #MiraDiT to explore the consistency in long video generation. Hope it will be a supplement to existing text-to-video methods. Project Page:

Xintao Wang

27,698 просмотров • 2 лет назад

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]
0:16

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]

Xuanchi Ren

26,586 просмотров • 1 год назад

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 просмотров • 1 год назад

We are presenting **SAD** (Segment Any RGBD): SAD is able to perform 3D segmentation (segment out any 3D object) with RGBD inputs (or rendered depth images only). - Code: - Demo Hugging Face:
0:11

We are presenting **SAD** (Segment Any RGBD): SAD is able to perform 3D segmentation (segment out any 3D object) with RGBD inputs (or rendered depth images only). - Code: - Demo Hugging Face:

Ziwei Liu

195,633 просмотров • 3 лет назад

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)
0:10

Humans are social creatures and physical interaction with other humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)

Lea Wilken/Müller

76,528 просмотров • 2 лет назад

RGB Human Videos -> Dexterous Manipulation Policies No teleoperation. No wearables. No robot interaction. The 🔑: Reliable 3D hand-object trajectory reconstruction and efficient action augmentation Fun collaboration led by Hongyi Chen
0:15

RGB Human Videos -> Dexterous Manipulation Policies No teleoperation. No wearables. No robot interaction. The 🔑: Reliable 3D hand-object trajectory reconstruction and efficient action augmentation Fun collaboration led by Hongyi Chen

Homanga Bharadhwaj

10,775 просмотров • 4 месяцев назад

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):
0:16

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):

Xingang Pan

11,473 просмотров • 6 месяцев назад