Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Early R&D using our SP-6M dataset. Exploring image-to-3D reconstruction from single images, including heavily modified inputs (lighting, hair, etc). Still a work in progress.

53,370 Aufrufe • vor 5 Monaten •via X (Twitter)

56 Kommentare

Profilbild von Toby VirtualFilmer
Toby VirtualFilmervor 5 Monaten

Oh my god. That… is… insanity. Currently all the ai image to 3d human services fail horribly 😂 but this could be the answer!

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Yep I know :) data and diversity are king when it comes to this kind of thing. We have over 7600 individual scans with 11 expressions each so around 80k scans across a hugely diverse group of people.

Profilbild von Vadim B
Vadim Bvor 5 Monaten

Very cool, can't wait, this is actually an amazing use of all your scanned data

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Cheers Vadim!

Profilbild von MR3D-Dev
MR3D-Devvor 5 Monaten

holy cow. Is this real?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Yep! It's all we've been doing for the past 18 months. :)

Profilbild von Jscott
Jscottvor 5 Monaten

Fucking fantastic dude. I dont even want to know what your compute bill looks like 😅

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Ha ha wait till I show you what we're using to train. We will show that soon.

Profilbild von Luciano Daluz
Luciano Daluzvor 5 Monaten

It is too good to be true.

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Ha ha 🤣

Profilbild von Pierre Schiller
Pierre Schillervor 5 Monaten

This is fantastic! Are you training your own model? What about topology?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Yep we're training our own model from scratch with all of our own data. Topology is production ready consistant morphable geometry with 8k textures. This is the dataset we created for it.

Profilbild von Pierre Schiller
Pierre Schillervor 5 Monaten

Instant sub. Way to go!!👍

Profilbild von racki
rackivor 5 Monaten

where can I buy the alpha?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

It's way to early for this I'm afraid.

Profilbild von Anthony R. Risen
Anthony R. Risenvor 5 Monaten

This is cool are you using flame smplx models with your textures overlayed? That’s pretty sweet!

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you! no this isn't flame this is our own topology / model.

Profilbild von r2
r2vor 5 Monaten

well this is

Profilbild von TheAlchemyRoom
TheAlchemyRoomvor 5 Monaten

Woah 😭

Profilbild von Progen
Progenvor 5 Monaten

Is the image-to-3d construction available to try yet?🤔

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

No I'm afraid not. We have a lot of work still to do :)

Profilbild von Progen
Progenvor 5 Monaten

any timeline on date of release?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

No I'm afraid not. It's very early in development.

Profilbild von Divesh Naidoo
Divesh Naidoovor 5 Monaten

Amazing!

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you!

Profilbild von Juan
Juanvor 5 Monaten

How ! 🔥🔥🔥😭

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

A lot of work :)

Profilbild von HighLegion
HighLegionvor 5 Monaten

My first question - when is the tool coming out and what would it be priced at? (genuinely want to know on behalf of 3d artists around the globe)

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Im afraid thats not something were announcing at the moment.

Profilbild von Leo Phiêu Diêu
Leo Phiêu Diêuvor 4 Monaten

Can you train more AI based on images of people/AI who are smiling, showing their teeth shape and dimples? Thank you. @Ten_24

Profilbild von Thomas Halpin
Thomas Halpinvor 5 Monaten

Do these work with MetaHuman?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

It will do yes but thats not its intended use :) more on that later in development.

Profilbild von Risichad 🦾
Risichad 🦾vor 5 Monaten

RIP Metahuman

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

ha ha not quite :) but thanks for the confidence boost :)

Profilbild von Yousef | Developer | e/acc
Yousef | Developer | e/accvor 5 Monaten

Does it produce a mesh?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

yes a fully retopologised production ready mesh

Profilbild von Joshua Johnson
Joshua Johnsonvor 5 Monaten

Yes, more pls. 🔥🔥🔥

Profilbild von Key Thirteenth
Key Thirteenthvor 5 Monaten

Wow!

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Cheers!

Profilbild von Macroblock
Macroblockvor 5 Monaten

So impressive! Does this build it with the standard topology you've established for all your scans?

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you and yes indeed it does :)

Profilbild von Macroblock
Macroblockvor 5 Monaten

That's great, really looking forward to trying it out!

Profilbild von Womp
Wompvor 5 Monaten

🔥🔥🔥🔥🔥🔥🔥

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you! 🤩

Profilbild von Bad Uncle
Bad Unclevor 5 Monaten

What????!!!

Profilbild von hx1984
hx1984vor 5 Monaten

amazing, great job bro

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you!

Profilbild von peculiar ohwodiasa
peculiar ohwodiasavor 5 Monaten

WIll there be a ten 24 to Blender pipeline ?

Profilbild von Sky
Skyvor 5 Monaten

wtf this is crazy

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Yeah it's been an insane project. Just scanning this many people has been a full time job for 20 people for the last 18 months.

Profilbild von Sky
Skyvor 5 Monaten

I know you said elsewhere this is still early stages, but are you able to feed it any image and have it create a model? Or only specifically curated ones? I’m dying to see what it would do with a photo of me hahaha

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Yes the goal is any image :)

Profilbild von A.Robot
A.Robotvor 5 Monaten

Making good progress! Still some issues with noses, and some random tattoos (LOL), but good first steps

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

Thank you! Yes we're learning as we go. It would be a lot easier if we built on top of another model but we're committed to using all our own data. Totally clean, not scraped, and GDPR compliant. There's a lot still to do and we have so much more to add to this. We expect to see a 50 - 100 fold increase in quality over the next 12 months :)

Profilbild von Roderick Weise
Roderick Weisevor 5 Monaten

@100PercentRobot Where can I invest in the company and the tech? This is definitely going to explode for sure.

Profilbild von 3D Scanstore
3D Scanstorevor 5 Monaten

@100PercentRobot Cheers Roberick! I'm afraid there are no investment opportunities at the moment.

Ähnliche Videos

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,818 Aufrufe • vor 1 Jahr