Loading video...

Video Failed to Load

Go Home

NLF is formulated in terms of an arbitrary keypoint estimation task, allowing to unify training data with different formats, with good pixel alignment. For SMPL output, we developed a super fast vertices-to-params fitter: And here's some more eye candy.

Istvan_Sarandi's profile picture

István Sárándi

1,178 subscribers

10,162 views • 1 year ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi
0:25

For 3D pose some use different keypoints, others SMPL and other models. It's a mess! With Neural Localizer Fields, we can choose the output at test time! allowing to train using any. Results are real time and SOTA across the board. István Sárándi

Gerard Pons-Moll

43,472 views • 2 years ago

Tomorrow I'll talk at the #ICRA workshop on "Enhancing human mobility" on recent 3D human estimation works from our group led by Gerard Pons-Moll. And I've released PyTorch training code for NLF, and more tools. Here's a preview of the even stronger upcoming DINOv2-ViTg-based NLF
1:08

Tomorrow I'll talk at the #ICRA workshop on "Enhancing human mobility" on recent 3D human estimation works from our group led by Gerard Pons-Moll. And I've released PyTorch training code for NLF, and more tools. Here's a preview of the even stronger upcoming DINOv2-ViTg-based NLF

István Sárándi

12,124 views • 1 year ago

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
2:15

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

433,383 views • 3 months ago

Some VINTAGE TypeScript magic. Here's how to create a type-safe version of Object.assign, with an arbitrary number of objects.
3:32

Some VINTAGE TypeScript magic. Here's how to create a type-safe version of Object.assign, with an arbitrary number of objects.

Matt Pocock

38,013 views • 1 year ago

first print of a transformation of S4 slicer output with an arbitrary .stl
0:35

first print of a transformation of S4 slicer output with an arbitrary .stl

Brendon 🤘

21,368 views • 1 year ago

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.
0:32

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

AK

305,633 views • 2 years ago

I have an unpopular opinion on ROKO, Ro is different & Ko is different. Ro has nothing to do with Ko, they are 2 different individual. In terms of stardom, in terms of longetivity, in terms of appeal, in terms of everything. Ko is a different gravy altogether. - K. Srinavas Rao
1:35

I have an unpopular opinion on ROKO, Ro is different & Ko is different. Ro has nothing to do with Ko, they are 2 different individual. In terms of stardom, in terms of longetivity, in terms of appeal, in terms of everything. Ko is a different gravy altogether. - K. Srinavas Rao

Rajiv

57,209 views • 5 months ago

Another super tiny update 😅 Squeezed out these two new shots before I move to a different country. I tried to attack the character animation with simple cushions this time but added some eye candy FX for contrast. And here it is! Cheers! 🍻
1:47

Another super tiny update 😅 Squeezed out these two new shots before I move to a different country. I tried to attack the character animation with simple cushions this time but added some eye candy FX for contrast. And here it is! Cheers! 🍻

Dan Mark Lazaga

240,017 views • 2 years ago

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.
44:56

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.

RoboPapers

89,011 views • 8 months ago

Introducing Project Stera by FPV Labs, an open data infra for embodied AI research. Project Stera includes Stera-10M, with 10M+ frames of long-horizon data with persistent state tracking, and an open-source pipeline that converts raw data into training-ready formats.
0:45

Introducing Project Stera by FPV Labs, an open data infra for embodied AI research. Project Stera includes Stera-10M, with 10M+ frames of long-horizon data with persistent state tracking, and an open-source pipeline that converts raw data into training-ready formats.

FPV Labs

14,981 views • 1 month ago

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,348 views • 1 year ago

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.
4:38

Code and data are now online for CameraHMR, our state-of-the-art parametric 3D human pose and shape (HPS) estimation method that will appear at hashtag#3DV2025. There are 4 key contributions that make it so accurate and robust: 1. To get accurate 3D shape and pose as well as good alignment to image features, you need to know the focal length of the camera. To solve this, we train HumanFOV to compute the field of view. 2. We introduce CameraHMR, which integrates HumanFOV into HMR2.0 to exploit the estimated focal length. 3. To get accurate pseudo ground truth (pGT) training data, we compute the focal length for images in 4DHumans dataset and modify SMPLify to take this into account. 4. But SMPLify only uses sparse 2D keypoints, which do not capture body shape. So we train a dense surface keypoint detector, DenseKP, on BEDLAM and run it on 4DHumans, resulting in improved body shape. The resulting method is CamSMPLify. We iterate training CameraHMR and running CamSMPLify on the training set initialized with CameraHMR. This results in much improved pGT for 4DHumans and a SOTA single-image HMR method.

Michael Black

21,696 views • 1 year ago

An old geezer flirted with this young lady and now she’s offended. Eye candy to an old grandpa, this is the patriarchy for you.
1:24

An old geezer flirted with this young lady and now she’s offended. Eye candy to an old grandpa, this is the patriarchy for you.

Dr. Jebra Faushay

89,368 views • 2 months ago

BeiDou is developed by China and is a global good. China will continue working with other nations in the spirit of openness and mutual benefit to deliver more advanced technologies for humanity.
0:49

BeiDou is developed by China and is a global good. China will continue working with other nations in the spirit of openness and mutual benefit to deliver more advanced technologies for humanity.

Mao Ning 毛宁

53,658 views • 1 year ago

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...
0:34

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...

beeple

155,748 views • 1 year ago

New in TanStack Router: more control over dynamic route params. Use params.parse to accept or skip a candidate, with priority for overlapping dynamic params.
0:30

New in TanStack Router: more control over dynamic route params. Use params.parse to accept or skip a candidate, with priority for overlapping dynamic params.

TANSTACK

118,165 views • 1 month ago

Model progress is no longer constrained by architecture, but by access to high-quality, human-generated data. Scraped internet data is finite, low-signal and increasingly synthetic. Kled AI pays real people to upload authentic, real world content task by task in their mobile app. We are super excited to back Avi Patel and Kled AI as they build a massive data marketplace. If you are in need of a great data partner, reach out! More from us here (we were not investors when we filmed):
23:44

Model progress is no longer constrained by architecture, but by access to high-quality, human-generated data. Scraped internet data is finite, low-signal and increasingly synthetic. Kled AI pays real people to upload authentic, real world content task by task in their mobile app. We are super excited to back Avi Patel and Kled AI as they build a massive data marketplace. If you are in need of a great data partner, reach out! More from us here (we were not investors when we filmed):

Nichole Wischoff

36,782 views • 3 months ago