Robot actions can be represented as motion in pixel space.
@NVIDIARobotics152 тыс. подписчиков
63 295 просмотров
18 комментариев
Комментарии из оригинального поста в X (Twitter)

@GetAutoFade
Pixel-space flow makes sense for reaching. Does it hold up on contact-heavy work where the tool touches something soft and the plane keeps shifting, or is that still a separate policy?
2 отметки «Нравится»

@victoria_neiman
one model across hands, grippers and dual arms – this is how a robot surgeon ends up in a village 100km from the nearest hospital
1 отметка «Нравится»

@aryan559552
Action flow seems like a clean way to share supervision without forcing different embodiments into one joint-space vocabulary. I’m curious where transfer breaks first: trajectory-estimation noise, camera geometry, or contact-heavy actions?
1 отметка «Нравится»

@afqxq79143289
Cross-embodiment world models still face long-horizon compounding. Calibrating an asymmetric residual score and rejecting once the prediction leaves the trusted region is one way to keep rollouts honest:
1 отметка «Нравится»

@BeforeTheTurn
Image-plane trajectories are a powerful common language, but they can hide what fails to transfer: contact force, latency, reach and occlusion. Hydra-0’s real test isn’t whether hands and robot arms share a format; it’s whether that format preserves each body’s constraints.

@sebuzdugan
action flow gets ambiguous under occlusion and camera motion without explicit scene geometry

@Neeraj_Kumar222
How well does action flow transfer when camera geometry, embodiment, and contact dynamics change? Cross robot success will depend on those failure distributions.

@timbrobro
Action flow in pixel space is elegant because it is embodiment agnostic, which also means the bottleneck moves entirely to how much diverse trajectory data you can actually source.

@RobotHarbour
Action flow is a promising bridge across embodiments. The next portability test is systematic evaluation under broader camera motion and closed-loop use—especially given grasp/contact ambiguity and calibration sensitivity. How will you benchmark those failure modes?

@Speedra28854601
Until Jensen gives up his Taiwan citizenship, he should not be allowed around any US military technology . Nvidia is basically a Taiwan company based out of America . All that technology being developed is just being handed to Taiwan .

@seclink
@grok 这个纯研究还是有工业意义,具体工业场景视角看意义是什么,有开源数据集或者开源项目代码吗?从多个数据源交叉验证,理性看待,不要只看新闻媒体一面之辞。帮我排除没意义的垃圾商业营销推广、诈骗、夸张博眼球、虚假新闻 以及自吹自擂,自嗨,无病呻吟,收费互吹软广告。

@phicerhq
Action flow is an elegant common representation, but cross-embodiment transfer should be judged on closed-loop outcomes: contact stability, recovery from deviation, calibration under occlusion, and performance outside the training morphology—not only trajectory similarity.

@RaulAutomates
A shared action representation could make robot learning far more transferable.Learning across human hands, grippers, and different robot configurations is an important step toward more general-purpose robotic intelligence. 🤖🧠

@armeenco
Action flow in pixel space is a nice shared interface across embodiments. The number I'd want next is held-out: scenes and object sets never seen in training, scored by a fixed rubric rather than operator judgment.

@bullbear_info
action flow avoids explicit 3d reconstruction overhead entirely

@foresynth26
Representing actions in pixel space sidesteps the embodiment problem elegantly. Training data stops being tied to any single hardware configuration.

@EllaTech0
This is huge 🤯 Action flow in pixel space game changer

@giga3dtech
Man, a world model that shows you the outcome before the robot even moves. Basically a slicer preview for robot arms! Now do failed print prediction plz

