Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We just released HaMeR, our latest work for Hand Mesh Recovery! We reconstruct hands in 3D from a single image with big improvements in accuracy and robustness. Code and models are available. Also colab and Hugging Face demo!

geopavlakos's profile picture

Georgios Pavlakos

2,437 subscribers

38,336 görüntüleme • 2 yıl önce •via X (Twitter)

Bilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

Carlos Barreto profil fotoğrafı
Carlos Barreto2 yıl önce

Looks like one more amazing work from you 👏

Georgia Gkioxari profil fotoğrafı
Georgia Gkioxari2 yıl önce

Haaaaaaands! 🫶👐

Georgios Pavlakos profil fotoğrafı
Georgios Pavlakos2 yıl önce

Yeees! Actually, instead of emojis, I will be using 3D hands now!

Carlos Barreto profil fotoğrafı
Carlos Barreto2 yıl önce

This is so popular that the link to download the model is disabled temporarily 😅

John Mierzwa 🏞 profil fotoğrafı
John Mierzwa 🏞2 yıl önce

This looks amazing, great work!

Akshay Mundra profil fotoğrafı
Akshay Mundra2 yıl önce

I'm curious about the run time of the method. Can it be used for real time applications?

Georgios Pavlakos profil fotoğrafı
Georgios Pavlakos2 yıl önce

A single forward pass of the HaMeR network given a crop of a hand requires ~32ms on an RTX 3090.

INIYSA profil fotoğrafı
INIYSA2 yıl önce

This is crazy good

AI Bites | YouTube Channel profil fotoğrafı
AI Bites | YouTube Channel2 yıl önce

Yes this is awesome. Just featured your work on our account yesterday 🙂

Andrzej Białecki profil fotoğrafı
Andrzej Białecki2 yıl önce

Just a question is it possible then to extract that 3D model of a calibrated hand as a rig? And is it then possible to provide it with a single measurement to get precise hand size?

Georgios Pavlakos profil fotoğrafı
Georgios Pavlakos2 yıl önce

Our method actually predicts model parameters for the MANO hand parametric model, so you have access to the joint angles, not just the mesh surface. If you have some metric measurement, you can rescale the output accordingly.

Benzer Videolar

Check out HSMR, Yan Xia's Yan XIA latest work. We reconstruct humans using a biomechanically accurate skeleton. Code and HuggingFace demo are live. Give it a try! Webpage: Code: Demo:
2:00

Check out HSMR, Yan Xia's Yan XIA latest work. We reconstruct humans using a biomechanically accurate skeleton. Code and HuggingFace demo are live. Give it a try! Webpage: Code: Demo:

Georgios Pavlakos

12,110 görüntüleme • 1 yıl önce

Alibaba just released LHM on Hugging Face Large Animatable Human Reconstruction Model from a Single Image in Seconds
1:31

Alibaba just released LHM on Hugging Face Large Animatable Human Reconstruction Model from a Single Image in Seconds

AK

170,372 görüntüleme • 1 yıl önce

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4
0:37

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Gordon Wetzstein

19,210 görüntüleme • 2 yıl önce

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:
1:02

We are thrilled to release the next leap in art-grade 3D generative models. Our single-click model pipeline gives unprecedented mesh outputs, with mesh parts-based topology. It is available now for all Cube tiers to start for free. ☑️ Our multi-stage hierarchical AI models produce a fully assembled 3D mesh with adaptive poly-counts, providing the clean, separated topology you need. ✅ A parts-based approach enables high-resolution meshes. 🌐 Quad and Triangle mesh support. Access now:

Common Sense Machines

420,323 görüntüleme • 1 yıl önce

Our new work PSHuman reconstructs a detailed 3D human mesh from a single-view image in ~1min. Codes have already been released! Welcome to try it! Project page: Codes: Paper:
0:39

Our new work PSHuman reconstructs a detailed 3D human mesh from a single-view image in ~1min. Codes have already been released! Welcome to try it! Project page: Codes: Paper:

Yuan Liu

53,252 görüntüleme • 1 yıl önce

We just released our latest single, "Immortals Die". - - And everywhere else!
0:24

We just released our latest single, "Immortals Die". - - And everywhere else!

Floodfall

32,087 görüntüleme • 7 ay önce

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.
0:40

I prompted Omma to build a #threejs app with real dynamic lighting on a 3D mesh - generated from a single image, using Depth Anything v2 + Transformers from Hugging Face ⚡ Depth estimation in the browser 🫧 3D mesh from the depth map 💡 Dynamic lighting reacting to the geometry No manual code. Pure AI prompting. The future of 3D on the web feels wide open.

Joseph Azar

14,041 görüntüleme • 2 ay önce

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito
0:32

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito

Matthias Niessner

62,159 görüntüleme • 1 yıl önce

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:
1:05

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

AK

87,679 görüntüleme • 3 yıl önce

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.
0:22

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 görüntüleme • 2 yıl önce

Are you ready for the journey? We are happy to announce that we've released the demo version of our game, and we invite every degen to join! Participate in our galactic race and win prizes we have prepared just for you! Demo Download -
0:40

Are you ready for the journey? We are happy to announce that we've released the demo version of our game, and we invite every degen to join! Participate in our galactic race and win prizes we have prepared just for you! Demo Download -

Arcana Continuum

12,618 görüntüleme • 3 yıl önce

[1/2] We’ve released the code for #pix2pixturbo and #CycleGANTurbo. These conditional GANs are able to adapt a text-to-image model such as SD-Turbo for both paired and unpaired image translation with a single step (0.11 sec on A100 and 0.29 sec on A6000). Try our code and the Gradio demo. Paper: Code: Demo: This is a joint work with Gaurav Parmar (the leading author), Taesung Park, and Srinivasa Narasimhan. This work shows that a pre-trained one-step model can be easily adapted to conditional GANs frameworks for downstream image editing and synthesis tasks. #Edges2Cats
0:26

[1/2] We’ve released the code for #pix2pixturbo and #CycleGANTurbo. These conditional GANs are able to adapt a text-to-image model such as SD-Turbo for both paired and unpaired image translation with a single step (0.11 sec on A100 and 0.29 sec on A6000). Try our code and the Gradio demo. Paper: Code: Demo: This is a joint work with Gaurav Parmar (the leading author), Taesung Park, and Srinivasa Narasimhan. This work shows that a pre-trained one-step model can be easily adapted to conditional GANs frameworks for downstream image editing and synthesis tasks. #Edges2Cats

Jun-Yan Zhu

36,488 görüntüleme • 2 yıl önce

📢WorldAgents: 3D worlds only from 2D image models - without any training! We propose an agentic approach with a Director (VLM) to plan the scene, a Generator (Flux or NanoBanana) for new views, and a Verifier (VLM) for selection / 3D consistency. -> High-fidelity 3D worlds from a single text prompt. What's remarkable: our agents find consistent views from 2D image models to obtain 3D-consistent worlds; this shows that image models contain world priors - agents just need to find them! Great work by Ziya Erkoç Angela Dai
0:25

📢WorldAgents: 3D worlds only from 2D image models - without any training! We propose an agentic approach with a Director (VLM) to plan the scene, a Generator (Flux or NanoBanana) for new views, and a Verifier (VLM) for selection / 3D consistency. -> High-fidelity 3D worlds from a single text prompt. What's remarkable: our agents find consistent views from 2D image models to obtain 3D-consistent worlds; this shows that image models contain world priors - agents just need to find them! Great work by Ziya Erkoç Angela Dai

Matthias Niessner

18,923 görüntüleme • 3 ay önce

We've just posted a new dev blog! In this community update, we cover a bunch of upcoming critter improvements like critter growth and powerful alpha critters. We also share sneak peeks at the work-in-progress ambient audio system and our progress on other immersive features. Check out our full blog for our latest news here!:
10:30

We've just posted a new dev blog! In this community update, we cover a bunch of upcoming critter improvements like critter growth and powerful alpha critters. We also share sneak peeks at the work-in-progress ambient audio system and our progress on other immersive features. Check out our full blog for our latest news here!:

Path of Titans

16,471 görüntüleme • 1 yıl önce

VAST AI releases Triplane Meets Gaussian Splatting on Hugging Face Fast and Generalizable Single-View 3D Reconstruction with Transformers demo: TGS enables fast reconstruction from single-view image. It builds the 3D representation upon a hybrid Triplane-Gaussian representation by evaluating a transformer-based framework, from which 3D Gaussians would be decoded
0:23

VAST AI releases Triplane Meets Gaussian Splatting on Hugging Face Fast and Generalizable Single-View 3D Reconstruction with Transformers demo: TGS enables fast reconstruction from single-view image. It builds the 3D representation upon a hybrid Triplane-Gaussian representation by evaluating a transformer-based framework, from which 3D Gaussians would be decoded

AK

113,226 görüntüleme • 2 yıl önce

Introducing Moonlake's 3D Agent. Our agent acts like a technical artist that can build and reconstruct articulated assets and large-scale editable scenes with hundreds of objects from a single image and can improve its generations continuously. Learn more in the thread below.
0:30

Introducing Moonlake's 3D Agent. Our agent acts like a technical artist that can build and reconstruct articulated assets and large-scale editable scenes with hundreds of objects from a single image and can improve its generations continuously. Learn more in the thread below.

Moonlake

1,161,139 görüntüleme • 2 ay önce

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!
0:23

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!

Felix Heide

19,105 görüntüleme • 2 yıl önce

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein
0:30

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein

Matthias Niessner

18,827 görüntüleme • 8 ay önce

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.
0:34

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.

AK

305,643 görüntüleme • 3 yıl önce

🚀 Big news for the Colab community! 🚀 In addition to being available in VSCode, Colab is also available for use in Antigravity, Cursor, and Windsurf via the Open VSX Registry! Get started with your favorite AI Code editors today! 🧑‍💻👇
0:27

🚀 Big news for the Colab community! 🚀 In addition to being available in VSCode, Colab is also available for use in Antigravity, Cursor, and Windsurf via the Open VSX Registry! Get started with your favorite AI Code editors today! 🧑‍💻👇

Colaboratory

64,876 görüntüleme • 7 ay önce