Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi

Gerard Pons-Moll

6,091 subscribers

43,472 просмотров • 2 лет назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля Gerard Pons-Moll

Gerard Pons-Moll2 лет назад

The key idea is to train a @neural_fields of localizer networks. The user can choose a continuous point in canonical space, and from this we predict the weights of a convolutional neural network to predict that point.

Фото профиля Gerard Pons-Moll

Gerard Pons-Moll2 лет назад

Benefits are flexibility at test time, independence of formats, and imposing structure in weights. Nearby joints will have similar localizer networks.

Фото профиля Gerard Pons-Moll

Gerard Pons-Moll2 лет назад

In addition to this, we introduce a fast differentiable inverse kinematics solver to obtain SMPL models from random points.

Фото профиля Gerard Pons-Moll

Gerard Pons-Moll2 лет назад

Ah, and the model is real time! I'm beyond excited about this work by @Istvan_Sarandi !

Фото профиля Dan Casas

Dan Casas2 лет назад

@Istvan_Sarandi Wow, looks impressive -- congratulations!

Фото профиля Naureen Mahmood

Naureen Mahmood2 лет назад

@Istvan_Sarandi So so good!!

Фото профиля Fabien Baradel

Fabien Baradel2 лет назад

@Istvan_Sarandi Nice method and great results!

Фото профиля Jen-Chun Lin

Jen-Chun Lin2 лет назад

@Istvan_Sarandi Amazing !

Фото профиля Yong-Lu Li

Yong-Lu Li2 лет назад

@Istvan_Sarandi Super cool!

Фото профиля Happy Fruitee

Happy Fruitee2 лет назад

@Istvan_Sarandi Coooooool！

Похожие видео

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,105 просмотров • 2 лет назад

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,406 просмотров • 2 лет назад

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

Chelsea Finn

23,121 просмотров • 1 месяц назад

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

István Sárándi

10,162 просмотров • 1 год назад

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

Andi Marafioti

49,888 просмотров • 8 месяцев назад

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

Bit Rivals AI

29,792 просмотров • 1 год назад

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

el.cine

174,453 просмотров • 8 месяцев назад

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

Akshay 🚀

46,821 просмотров • 1 год назад

🗣️ "It's a results business and we didn't get the results." Brendon McCullum has spoken for the first time since leaving his role as England Test coach.

🗣️ "It's a results business and we didn't get the results." Brendon McCullum has spoken for the first time since leaving his role as England Test coach.

Sky Sports News

241,060 просмотров • 3 дней назад

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Lightspeed

144,561 просмотров • 28 дней назад

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

Solana

71,207 просмотров • 9 месяцев назад

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

DogeDesigner

1,590,829 просмотров • 1 год назад

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

István Sárándi

40,556 просмотров • 2 лет назад

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 просмотров • 2 лет назад

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 просмотров • 1 месяц назад

An AVATAR program developed with UE5 that supports connecting to AI motion capture for real-time, high-quality UE animation output. It features a highly flexible character customization system and rich clothing and accessory assets. You can upload images to generate 3D models and quickly obtain high-quality 3D model assets.

CYANPUPPETS

11,542 просмотров • 5 месяцев назад

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

ollama

219,407 просмотров • 2 лет назад

The National Security Strategy and American system are coming alive in real time. Choices have to be made. Some choose cooperation. Others choose isolation. Regardless, all roads always ran through Iran.

The National Security Strategy and American system are coming alive in real time. Choices have to be made. Some choose cooperation. Others choose isolation. Regardless, all roads always ran through Iran.

PatriotDadEV2.0

20,765 просмотров • 3 месяцев назад