正在加载视频...

视频加载失败

Introduce EgoInfinity: a web-scale (14.6 yrs, 142M clips) data engine that automatically lifts Youtube videos into 4D hand-object interaction, and retargets for robot learning Not only a dataset, but a modular, upgradable engine HF Space:

108,060 次观看 • 3 个月前 •via X (Twitter)

20 条评论

Vector Wang 的头像
Vector Wang3 个月前

The dataset is open to anyone to use and contribute. The pipeline is modular and upgradable. Come scale things up even without any hardwares! Project Page: Paper: Codes: Dataset:

Damian @ The Lighthouse 🤖💡🏠 的头像
Damian @ The Lighthouse 🤖💡🏠3 个月前

@antoinemarcel just shared this in our gc. Looks great. Passing this on to a handful of people now.

Jerry Chéng 的头像
Jerry Chéng3 个月前

Absolute game changer! Thanks for the open-sourcing

Vector Wang 的头像
Vector Wang3 个月前

Thanks Jerry! Let’s build it together!

Michael Cho - Rbt/Acc 的头像
Michael Cho - Rbt/Acc3 个月前

Mind blown! Tks for putting this together!!

Vector Wang 的头像
Vector Wang3 个月前

Just a start! As a modular platform, it still can be and needs to be upgraded and improved by us and everyone in the community! Just like XLeRobot, I am just offering a path for everyone to build upon.

ahad 的头像
ahad3 个月前

fire!!!

clankr 的头像
clankr3 个月前

The demos look fantastic. Creating an SE(3)-compliant coordinate space for every camera angle is the key. Here is our take:

Damian @ The Lighthouse 🤖💡🏠 的头像
Damian @ The Lighthouse 🤖💡🏠3 个月前

@nicks_robots

Tolu 的头像
Tolu3 个月前

❤️

Damian @ The Lighthouse 🤖💡🏠 的头像
Damian @ The Lighthouse 🤖💡🏠3 个月前

@astridwilde

Damian @ The Lighthouse 🤖💡🏠 的头像
Damian @ The Lighthouse 🤖💡🏠3 个月前

@grantg07

wasabee_dai 的头像
wasabee_dai3 个月前

Ming blowing, it's a game changer! Thank you for it

Mahid 的头像
Mahid3 个月前

NIce work! How long did you work on this?

🐱🐱💥 的头像
🐱🐱💥3 个月前

Nice work! Super excited to see what comes ahead!

bnw 的头像
bnw3 个月前

vaporware or fr? most such repos hide a critical component, are you fr fr?

Damian @ The Lighthouse 🤖💡🏠 的头像
Damian @ The Lighthouse 🤖💡🏠3 个月前

@krithikas12

Milton Calderon Donefer 的头像
Milton Calderon Donefer3 个月前

very interesting!

Kavin Raj 的头像
Kavin Raj3 个月前

amazing work!

Alen Gabs 的头像
Alen Gabs3 个月前

@tommiekerssies

相关视频

I don’t know if we live in a Matrix, but I know for sure that robots will spend most of their lives in simulation. Let machines train machines. I’m excited to introduce DexMimicGen, a massive-scale synthetic data generator that enables a humanoid robot to learn complex skills from only a handful of human demonstrations. Yes, as few as 5! DexMimicGen addresses the biggest pain point in robotics: where do we get data? Unlike with LLMs, where vast amounts of texts are readily available, you cannot simply download motor control signals from the internet. So researchers teleoperate the robots to collect motion data via XR headsets. They have to repeat the same skill over and over and over again, because neural nets are data hungry. This is a very slow and uncomfortable process. At NVIDIA, we believe the majority of high-quality tokens for robot foundation models will come from simulation. What DexMimicGen does is to trade GPU compute time for human time. It takes one motion trajectory from human, and multiplies into 1000s of new trajectories. A robot brain trained on this augmented dataset will generalize far better in the real world. Think of DexMimicGen as a learning signal amplifier. It maps a small dataset to a large (de facto infinite) dataset, using physics simulation in the loop. In this way, we free humans from babysitting the bots all day. The future of robot data is generative. The future of the entire robot learning pipeline will also be generative. 🧵

Jim Fan

165,246 次观看 • 1 年前

The Machine That Learns The Law Behind The Data A very very interesting US Patent US10963540B2 - Physics Informed Learning Machine describes a learning system that does not begin with data alone. It begins with a physical model, usually written as a differential equation (or PDE) dx/dt = f(x,t) A normal Machine Learning model sees scattered data and tries to fit it. A physics-informed learning machine starts with a law. Then it treats the data as evidence that updates what the model believes about the physical system. For this application, I use the patent idea on NASA C-MAPSS Turbofan engine data. The machine watches multivariate telemetry from a degrading engine and infers a hidden health state that is not measured directly. From that posterior belief, it estimates the engine’s remaining useful life. In the main 3D scene, the engine lifetime is turned into a tunnel. The spiral ribbons are real sensor channels evolving over cycle-time. The glowing core is the inferred health state. The surrounding cloud is uncertainty. The orange wall ahead is the predicted failure horizon. So the big picture is: sensor evidence comes in, posterior belief tightens, and the machine moves from uncertainty toward a concrete failure prediction. The inset posteriors make that explicit. The health posterior shows where the model believes the hidden engine condition sits at the current moment, and how sharply it believes it. The RUL posterior shows the same idea for remaining life... early on it is broad, later it shifts left and narrows as the machine becomes more certain about how close failure is. This idea is not limited to engines. The same idea can apply to data centers, CPUs, GPUs, cooling systems, power grids, robotics, batteries, and any machine that produces telemetry while obeying physical constraints. In an age where machine learning runs on massive hardware infrastructure, this kind of model matters: it can turn noisy sensor streams into early warnings before expensive systems fail.

Mathelirium

17,843 次观看 • 4 个月前