Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi

Gerard Pons-Moll

6,091 subscribers

43,472 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Gerard Pons-Moll

Gerard Pons-Mollvor 2 Jahren

The key idea is to train a @neural_fields of localizer networks. The user can choose a continuous point in canonical space, and from this we predict the weights of a convolutional neural network to predict that point.

Profilbild von Gerard Pons-Moll

Gerard Pons-Mollvor 2 Jahren

Benefits are flexibility at test time, independence of formats, and imposing structure in weights. Nearby joints will have similar localizer networks.

Profilbild von Gerard Pons-Moll

Gerard Pons-Mollvor 2 Jahren

In addition to this, we introduce a fast differentiable inverse kinematics solver to obtain SMPL models from random points.

Profilbild von Gerard Pons-Moll

Gerard Pons-Mollvor 2 Jahren

Ah, and the model is real time! I'm beyond excited about this work by @Istvan_Sarandi !

Profilbild von Dan Casas

Dan Casasvor 2 Jahren

@Istvan_Sarandi Wow, looks impressive -- congratulations!

Profilbild von Naureen Mahmood

Naureen Mahmoodvor 2 Jahren

@Istvan_Sarandi So so good!!

Profilbild von Fabien Baradel

Fabien Baradelvor 2 Jahren

@Istvan_Sarandi Nice method and great results!

Profilbild von Jen-Chun Lin

Jen-Chun Linvor 2 Jahren

@Istvan_Sarandi Amazing !

Profilbild von Yong-Lu Li

Yong-Lu Livor 2 Jahren

@Istvan_Sarandi Super cool!

Profilbild von Happy Fruitee

Happy Fruiteevor 2 Jahren

@Istvan_Sarandi Coooooool！

Ähnliche Videos

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,105 Aufrufe • vor 2 Jahren

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,406 Aufrufe • vor 2 Jahren

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

Chelsea Finn

23,121 Aufrufe • vor 1 Monat

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

István Sárándi

10,162 Aufrufe • vor 1 Jahr

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

Andi Marafioti

49,888 Aufrufe • vor 8 Monaten

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

Bit Rivals AI

29,792 Aufrufe • vor 1 Jahr

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

el.cine

174,453 Aufrufe • vor 7 Monaten

Elon Musk demonstrates a revolutionary way to interface with complex 3D models allowing engineers to rotate, zoom, and translate wireframes in real-time with their hands

Elon Musk demonstrates a revolutionary way to interface with complex 3D models allowing engineers to rotate, zoom, and translate wireframes in real-time with their hands

X Freeze

59,510 Aufrufe • vor 4 Monaten

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

Akshay 🚀

46,821 Aufrufe • vor 1 Jahr

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Lightspeed

144,561 Aufrufe • vor 24 Tagen

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

Solana

71,207 Aufrufe • vor 8 Monaten

🔥Image -> 3D -> 3D Print 🤖Now you can easily obtain your favorite figurine models using #Rodin Gen-1 and 3D printing at any time. #3dprint #GenerativeAI #3D #CG #game

🔥Image -> 3D -> 3D Print 🤖Now you can easily obtain your favorite figurine models using #Rodin Gen-1 and 3D printing at any time. #3dprint #GenerativeAI #3D #CG #game

Hyper3D by Deemos

24,732 Aufrufe • vor 2 Jahren

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

DogeDesigner

1,590,829 Aufrufe • vor 1 Jahr

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

István Sárándi

40,556 Aufrufe • vor 2 Jahren

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 Aufrufe • vor 2 Jahren

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 Aufrufe • vor 1 Monat

They did use 3D models and we can fly through the scenes and check it all out! #threejs Shopify Using the Needle three.js inspector

They did use 3D models and we can fly through the scenes and check it all out! #threejs Shopify Using the Needle three.js inspector

🌵 Marcel Wiessler

157,284 Aufrufe • vor 7 Monaten

An AVATAR program developed with UE5 that supports connecting to AI motion capture for real-time, high-quality UE animation output. It features a highly flexible character customization system and rich clothing and accessory assets. You can upload images to generate 3D models and quickly obtain high-quality 3D model assets.

CYANPUPPETS

11,542 Aufrufe • vor 5 Monaten