Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi

Gerard Pons-Moll

6,091 subscribers

43,472 görüntüleme • 1 yıl önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

10 Yorum

Gerard Pons-Moll profil fotoğrafı

Gerard Pons-Moll1 yıl önce

The key idea is to train a @neural_fields of localizer networks. The user can choose a continuous point in canonical space, and from this we predict the weights of a convolutional neural network to predict that point.

Gerard Pons-Moll profil fotoğrafı

Gerard Pons-Moll1 yıl önce

Benefits are flexibility at test time, independence of formats, and imposing structure in weights. Nearby joints will have similar localizer networks.

Gerard Pons-Moll profil fotoğrafı

Gerard Pons-Moll1 yıl önce

In addition to this, we introduce a fast differentiable inverse kinematics solver to obtain SMPL models from random points.

Gerard Pons-Moll profil fotoğrafı

Gerard Pons-Moll1 yıl önce

Ah, and the model is real time! I'm beyond excited about this work by @Istvan_Sarandi !

Dan Casas profil fotoğrafı

Dan Casas1 yıl önce

@Istvan_Sarandi Wow, looks impressive -- congratulations!

Naureen Mahmood profil fotoğrafı

Naureen Mahmood1 yıl önce

@Istvan_Sarandi So so good!!

Fabien Baradel profil fotoğrafı

Fabien Baradel1 yıl önce

@Istvan_Sarandi Nice method and great results!

Jen-Chun Lin profil fotoğrafı

Jen-Chun Lin1 yıl önce

@Istvan_Sarandi Amazing !

Yong-Lu Li profil fotoğrafı

Yong-Lu Li1 yıl önce

@Istvan_Sarandi Super cool!

Happy Fruitee profil fotoğrafı

Happy Fruitee1 yıl önce

@Istvan_Sarandi Coooooool！

Benzer Videolar

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,040 görüntüleme • 2 yıl önce

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,365 görüntüleme • 2 yıl önce

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

Chelsea Finn

23,121 görüntüleme • 11 gün önce

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

István Sárándi

10,162 görüntüleme • 1 yıl önce

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

Andi Marafioti

49,888 görüntüleme • 8 ay önce

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

Bit Rivals AI

29,792 görüntüleme • 1 yıl önce

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

el.cine

174,354 görüntüleme • 7 ay önce

Elon Musk demonstrates a revolutionary way to interface with complex 3D models allowing engineers to rotate, zoom, and translate wireframes in real-time with their hands

Elon Musk demonstrates a revolutionary way to interface with complex 3D models allowing engineers to rotate, zoom, and translate wireframes in real-time with their hands

X Freeze

59,510 görüntüleme • 4 ay önce

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

Akshay 🚀

46,821 görüntüleme • 1 yıl önce

🔥Image -> 3D -> 3D Print 🤖Now you can easily obtain your favorite figurine models using #Rodin Gen-1 and 3D printing at any time. #3dprint #GenerativeAI #3D #CG #game

🔥Image -> 3D -> 3D Print 🤖Now you can easily obtain your favorite figurine models using #Rodin Gen-1 and 3D printing at any time. #3dprint #GenerativeAI #3D #CG #game

Hyper3D by Deemos

24,706 görüntüleme • 2 yıl önce

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

Solana

71,207 görüntüleme • 8 ay önce

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

István Sárándi

40,556 görüntüleme • 1 yıl önce

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

DogeDesigner

1,590,809 görüntüleme • 1 yıl önce

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 görüntüleme • 19 gün önce

They did use 3D models and we can fly through the scenes and check it all out! #threejs Shopify Using the Needle three.js inspector

They did use 3D models and we can fly through the scenes and check it all out! #threejs Shopify Using the Needle three.js inspector

🌵 Marcel Wiessler

157,234 görüntüleme • 6 ay önce

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 görüntüleme • 2 yıl önce

An AVATAR program developed with UE5 that supports connecting to AI motion capture for real-time, high-quality UE animation output. It features a highly flexible character customization system and rich clothing and accessory assets. You can upload images to generate 3D models and quickly obtain high-quality 3D model assets.

CYANPUPPETS

11,542 görüntüleme • 4 ay önce

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

ollama

219,382 görüntüleme • 1 yıl önce