正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi

Gerard Pons-Moll

6,091 subscribers

43,472 次观看 • 2 年前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

10 条评论

Gerard Pons-Moll 的头像

Gerard Pons-Moll2 年前

The key idea is to train a @neural_fields of localizer networks. The user can choose a continuous point in canonical space, and from this we predict the weights of a convolutional neural network to predict that point.

Gerard Pons-Moll 的头像

Gerard Pons-Moll2 年前

Benefits are flexibility at test time, independence of formats, and imposing structure in weights. Nearby joints will have similar localizer networks.

Gerard Pons-Moll 的头像

Gerard Pons-Moll2 年前

In addition to this, we introduce a fast differentiable inverse kinematics solver to obtain SMPL models from random points.

Gerard Pons-Moll 的头像

Gerard Pons-Moll2 年前

Ah, and the model is real time! I'm beyond excited about this work by @Istvan_Sarandi !

Dan Casas 的头像

Dan Casas2 年前

@Istvan_Sarandi Wow, looks impressive -- congratulations!

Naureen Mahmood 的头像

Naureen Mahmood2 年前

@Istvan_Sarandi So so good!!

Fabien Baradel 的头像

Fabien Baradel2 年前

@Istvan_Sarandi Nice method and great results!

Jen-Chun Lin 的头像

Jen-Chun Lin2 年前

@Istvan_Sarandi Amazing !

Yong-Lu Li 的头像

Yong-Lu Li2 年前

@Istvan_Sarandi Super cool!

Happy Fruitee 的头像

Happy Fruitee2 年前

@Istvan_Sarandi Coooooool！

相关视频

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,105 次观看 • 2 年前

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,423 次观看 • 2 年前

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

Chelsea Finn

24,108 次观看 • 1 个月前

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

István Sárándi

10,162 次观看 • 1 年前

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

You can now train SOTA models without any storage!🌩️ We completely revamped the Hub’s backend to enable streaming at scale. We streamed TBs of data to 100s of H100s to train SOTA VLMs and saw serious speed-ups. But how?

Andi Marafioti

49,939 次观看 • 9 个月前

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

How are we transforming gaming with AI? We're using real-time game data to train AI models to help gamers and game developers alike. It all begins today.

Bit Rivals AI

29,792 次观看 • 1 年前

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

AI fully taking over 3D is inevitable you can now add 3D bg for 3D models inside 3d software using World Labs's Marble and.. view in real time and even use Octane's 3DGS to light up the model without adding lights this is impossible w/o AI

el.cine

174,533 次观看 • 8 个月前

We used RL to train models that create curated context from long documents for downstream use by agents. The models sometimes learn to invent their own abbreviations and shorthand. Optimizing with RL for downstream use produces very different artifacts from ordinary summaries: shorter, denser, creatively concise. We call these neural cheat-sheets.

We used RL to train models that create curated context from long documents for downstream use by agents. The models sometimes learn to invent their own abbreviations and shorthand. Optimizing with RL for downstream use produces very different artifacts from ordinary summaries: shorter, denser, creatively concise. We call these neural cheat-sheets.

Applied Compute

36,155 次观看 • 1 个月前

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

90% of AI products fail in the first month. It's not bad models but bad prompts. Adaline AI is a platform to create prompts, test across models and modalities, integrate tools, evaluate performance, and deploy with real-time monitoring. Find the best prompt for your LLMs...👇

Akshay 🚀

46,821 次观看 • 1 年前

With the stack fixed, can an AI agent choose data and experiment like a researcher? At Evolvent AI, we built RSIBench-Data to test whether agents can diagnose failures, design data, train checkpoints, and use feedback. 🧵 📄 💻

With the stack fixed, can an AI agent choose data and experiment like a researcher? At Evolvent AI, we built RSIBench-Data to test whether agents can diagnose failures, design data, train checkpoints, and use feedback. 🧵 📄 💻

Fanqing

43,532 次观看 • 4 天前

🗣️ "It's a results business and we didn't get the results." Brendon McCullum has spoken for the first time since leaving his role as England Test coach.

🗣️ "It's a results business and we didn't get the results." Brendon McCullum has spoken for the first time since leaving his role as England Test coach.

Sky Sports News

241,060 次观看 • 20 天前

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Lightspeed

144,826 次观看 • 1 个月前

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

.Max Kaplan says while we can use the internet to do anything at any time, we can’t use it for finance during nights and weekends Solana fixes this with a 24/7 global network

Solana

71,207 次观看 • 9 个月前

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

It's been a pleasure to work on this with Gerard Pons-Moll I think we found a really effective formulation for training large-scale strong pose and shape models. Here are some more qualitative results on tough, in-the-wild YouTube dance videos.

István Sárándi

40,582 次观看 • 2 年前

"It's now a collective bargaining issue. 92% of the players can say we prefer grass. Well, at least 50.1% have to prioritize grass when it's time to negotiate with the league. What are you willing to give up to get grass fields across the league? – Mike Florio (ProFootballTalk)

"It's now a collective bargaining issue. 92% of the players can say we prefer grass. Well, at least 50.1% have to prioritize grass when it's time to negotiate with the league. What are you willing to give up to get grass fields across the league? – Mike Florio (ProFootballTalk)

Dan Patrick Show

196,099 次观看 • 12 天前

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

🚨 ELON MUSK: "DOGE is trying to act broadly across all the departments, it's not just one department at a time. We are trying to make the government more efficient across the board."

DogeDesigner

1,590,834 次观看 • 1 年前

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Big thanks to AK for highlighting our work! LEO marks our pioneering step towards building an embodied generalist agent that can really comprehend the 3D world! 🚀Leveraging LLMs, we train LEO with real and synthetic 3D data across a diverse spectrum of tasks. It's thrilling to see LEO surpass current state-of-the-art SOTA methods in most benchmarked tasks, all under a single, unified model. 🔥 #Generalist_Agent

Siyuan Huang

22,710 次观看 • 2 年前

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wang

a16z

25,144 次观看 • 1 个月前