Robot arms can be extremely heavy compared to the objects they are asked to manipulate, especially with their metal links and highly geared motors. But is that actually a problem for robot learning, or are learning algorithms powerful enough to handle it? Are we paying any hidden costs for... using heavy robots? Here's what we found 🧵show more

[2/n] For decades, roboticists have pursued low-inertia arms: direct drive, cable drive, series elastic actuators, quasi-direct drive, and more. Meanwhile, robot learning took off on all kinds of arms, as if the hardware didn't matter. Seeing these two research directions side by side really made me curious: does high inertia carry a hidden cost for learning?

[3/n] To find out, we varied robot and object mass in a controlled environment, and tried to see how RL and BC gets impacted by different configurations. RL was able to find competent solutions at every mass pair. But BC revealed a hidden cost of using high-inertia hardware: it became less reliable as the robot grows heavier relative to the object.

[4/n] Some might ask: do we need to switch to a new, lighter robot? Actually, there might be something you can do with your existing arm. Task-space inertia for an articulated arm depends on the configuration and its contact direction, which opens up some practical action items: a lighter workspace or lighter demonstrations. BC succeeds more often on both.

[5/n] One last thing before we wrap up: does this mean expensive robots are bad, and that it's time to switch to low-cost arms? I want to be clear that our study does not reach this conclusion yet. Yes, cheaper arms in the market often do have lower inertia (you can compare different arms on our project page), along with lighter payloads. But they also tend to come with other confounding properties: no torque control with accurate feedforward modeling, significant backlash, and more. How much those limits cancel out the benefit of low inertia is still an open question.

TL;DR: When robots present far more inertia than the objects they handle, RL finds a way, but BC might be paying a cost that could worsen the data scaling law. We hope this brings robot's own dynamics back into the conversation for robot learning. There are quite a few nuanced details we had to think through to study this problem that we couldn't cover in this thread. We recommend reading the paper if you're interested! 📄arxiv: 🔗website:

Joint work with @BronarsToni and @pulkitology!

kudos for (once again) looking into the details most people take for granted! Great work @younghyo_park @BronarsToni @pulkitology

I think hardware and learning should be designed together much earlier. expecting the policy to compensate for every mechanical choice feels like an expensive habit.

very interesting results! do you expect these results to be exacerbated or mitigated when training BC policies with multi-embodied data? also, nice to see that RL works regardless :D

Interesting work. Similar question to what @nicks_robots had. I noticed, for fairness, you're scaling PD gains so trajectories across robots (at least in free space) are similar. If inertia effects were instead compensated for by the controller level, would findings change?

@nicks_robots We haven't tried inertia shaped controllers yet -- but I think it's def worth investigating! My bet is that it would bring the same benefit as lighter robots if perfectly implemented, which could be quite challenging depending on how good the force sensing is (cc @nicks_robots)

@nicks_robots Doing it on the controller side has it's issues too. e.g., we can tune an impedance controller to have low gain, which works well for contact, but isn't ideal for free-space motion. If we'd like variable gains, then need to deal with switching dynamic modes, smoothness, etc.

Quick question - do you try any inertia shaping / torque control methods on the larger robots to see if that has any useful effects on downstream performance? If so, then policies may be able to adjust their dynamics to help with attaining goals.

This is why we print lightweight parts for anything that moves! Every gram on the arm is a gram you pay for twice. Would love to see how hollow printed links stack up in your tests

Everybody talk so much about the learning algorithm that it’s easy to forget how much the arm itself might be holding it back.

My instinct says it's less the mass and more the geared motor dynamics that hurt. Looking forward to being proven wrong.

relatable mismatch with heavy arm & light object

My guess is it hurts force control way more than position control. Reflected inertia goes with gear ratio squared, so a heavy geared arm basically hides the contact force from the motors. Did lighter arms need less data to learn contact-rich tasks?