Loading video...

Video Failed to Load

Go Home

Early R&D using our SP-6M dataset. Exploring image-to-3D reconstruction from single images, including heavily modified inputs (lighting, hair, etc). Still a work in progress.

53,370 views • 5 months ago •via X (Twitter)

56 Comments

Toby VirtualFilmer's profile picture
Toby VirtualFilmer5 months ago

Oh my god. That… is… insanity. Currently all the ai image to 3d human services fail horribly 😂 but this could be the answer!

3D Scanstore's profile picture
3D Scanstore5 months ago

Yep I know :) data and diversity are king when it comes to this kind of thing. We have over 7600 individual scans with 11 expressions each so around 80k scans across a hugely diverse group of people.

Vadim B's profile picture
Vadim B5 months ago

Very cool, can't wait, this is actually an amazing use of all your scanned data

3D Scanstore's profile picture
3D Scanstore5 months ago

Cheers Vadim!

MR3D-Dev's profile picture
MR3D-Dev5 months ago

holy cow. Is this real?

3D Scanstore's profile picture
3D Scanstore5 months ago

Yep! It's all we've been doing for the past 18 months. :)

Jscott's profile picture
Jscott5 months ago

Fucking fantastic dude. I dont even want to know what your compute bill looks like 😅

3D Scanstore's profile picture
3D Scanstore5 months ago

Ha ha wait till I show you what we're using to train. We will show that soon.

Luciano Daluz's profile picture
Luciano Daluz5 months ago

It is too good to be true.

3D Scanstore's profile picture
3D Scanstore5 months ago

Ha ha 🤣

Pierre Schiller's profile picture
Pierre Schiller5 months ago

This is fantastic! Are you training your own model? What about topology?

3D Scanstore's profile picture
3D Scanstore5 months ago

Yep we're training our own model from scratch with all of our own data. Topology is production ready consistant morphable geometry with 8k textures. This is the dataset we created for it.

Pierre Schiller's profile picture
Pierre Schiller5 months ago

Instant sub. Way to go!!👍

racki's profile picture
racki5 months ago

where can I buy the alpha?

3D Scanstore's profile picture
3D Scanstore5 months ago

It's way to early for this I'm afraid.

Anthony R. Risen's profile picture
Anthony R. Risen5 months ago

This is cool are you using flame smplx models with your textures overlayed? That’s pretty sweet!

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you! no this isn't flame this is our own topology / model.

r2's profile picture
r25 months ago

well this is

TheAlchemyRoom's profile picture
TheAlchemyRoom5 months ago

Woah 😭

Progen's profile picture
Progen5 months ago

Is the image-to-3d construction available to try yet?🤔

3D Scanstore's profile picture
3D Scanstore5 months ago

No I'm afraid not. We have a lot of work still to do :)

Progen's profile picture
Progen5 months ago

any timeline on date of release?

3D Scanstore's profile picture
3D Scanstore5 months ago

No I'm afraid not. It's very early in development.

Divesh Naidoo's profile picture
Divesh Naidoo5 months ago

Amazing!

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you!

Juan's profile picture
Juan5 months ago

How ! 🔥🔥🔥😭

3D Scanstore's profile picture
3D Scanstore5 months ago

A lot of work :)

HighLegion's profile picture
HighLegion5 months ago

My first question - when is the tool coming out and what would it be priced at? (genuinely want to know on behalf of 3d artists around the globe)

3D Scanstore's profile picture
3D Scanstore5 months ago

Im afraid thats not something were announcing at the moment.

Leo Phiêu Diêu's profile picture
Leo Phiêu Diêu4 months ago

Can you train more AI based on images of people/AI who are smiling, showing their teeth shape and dimples? Thank you. @Ten_24

Thomas Halpin's profile picture
Thomas Halpin5 months ago

Do these work with MetaHuman?

3D Scanstore's profile picture
3D Scanstore5 months ago

It will do yes but thats not its intended use :) more on that later in development.

Risichad 🦾's profile picture
Risichad 🦾5 months ago

RIP Metahuman

3D Scanstore's profile picture
3D Scanstore5 months ago

ha ha not quite :) but thanks for the confidence boost :)

Yousef | Developer | e/acc's profile picture
Yousef | Developer | e/acc5 months ago

Does it produce a mesh?

3D Scanstore's profile picture
3D Scanstore5 months ago

yes a fully retopologised production ready mesh

Joshua Johnson's profile picture
Joshua Johnson5 months ago

Yes, more pls. 🔥🔥🔥

Key Thirteenth's profile picture
Key Thirteenth5 months ago

Wow!

3D Scanstore's profile picture
3D Scanstore5 months ago

Cheers!

Macroblock's profile picture
Macroblock5 months ago

So impressive! Does this build it with the standard topology you've established for all your scans?

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you and yes indeed it does :)

Macroblock's profile picture
Macroblock5 months ago

That's great, really looking forward to trying it out!

Womp's profile picture
Womp5 months ago

🔥🔥🔥🔥🔥🔥🔥

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you! 🤩

Bad Uncle's profile picture
Bad Uncle5 months ago

What????!!!

hx1984's profile picture
hx19845 months ago

amazing, great job bro

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you!

peculiar ohwodiasa's profile picture
peculiar ohwodiasa5 months ago

WIll there be a ten 24 to Blender pipeline ?

Sky's profile picture
Sky5 months ago

wtf this is crazy

3D Scanstore's profile picture
3D Scanstore5 months ago

Yeah it's been an insane project. Just scanning this many people has been a full time job for 20 people for the last 18 months.

Sky's profile picture
Sky5 months ago

I know you said elsewhere this is still early stages, but are you able to feed it any image and have it create a model? Or only specifically curated ones? I’m dying to see what it would do with a photo of me hahaha

3D Scanstore's profile picture
3D Scanstore5 months ago

Yes the goal is any image :)

A.Robot's profile picture
A.Robot5 months ago

Making good progress! Still some issues with noses, and some random tattoos (LOL), but good first steps

3D Scanstore's profile picture
3D Scanstore5 months ago

Thank you! Yes we're learning as we go. It would be a lot easier if we built on top of another model but we're committed to using all our own data. Totally clean, not scraped, and GDPR compliant. There's a lot still to do and we have so much more to add to this. We expect to see a 50 - 100 fold increase in quality over the next 12 months :)

Roderick Weise's profile picture
Roderick Weise5 months ago

@100PercentRobot Where can I invest in the company and the tech? This is definitely going to explode for sure.

3D Scanstore's profile picture
3D Scanstore5 months ago

@100PercentRobot Cheers Roberick! I'm afraid there are no investment opportunities at the moment.

Related Videos

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,818 views • 1 year ago