Introducing Adapt-1 Machina: Learning Continuous Control Sequences With Adapt-1 Machina, we explore how continuous sequences of machine controls can be learned through reinforcement learning, starting with coarse outcome feedback and without demonstrations or a critic.

We’ve documented the experiments, results and limitations in the full article, with execution traces and replays. Below, we walk through how Adapt-1 learned a movement, revised it and adapted it to different starting conditions.

We started with a YAM arm in MuJoCo. It had to pick up an object and place it in a receptacle, with both positions changing between layouts. We supplied pose and gripper controls through fixed inverse kinematics, but no movement demonstrations.

While learning the movement, Adapt-1 revised earlier attempts’ recorded commands. In one revision, it kept the first three commands and changed two values near the end. The parent links let us trace which attempt it reused, and the recorded commands confirmed the edits.

A useful sequence could still fail when we moved things around. We tested changes to its length and timing, then learned bounded corrections from the starting observation. Each proposed change went back through execution so we could see what it actually did.

With learning off, corrections increased completed placements from 208 to 232 on the same 384 unseen layouts. Adapt-1 could adapt a learned sequence to new layouts without further training, though the gains were uneven across layouts.

In another test AIM, Adapt-1 learned an aiming sequence, then learned to adjust it for different target setups. With learning turned off, those corrections raised hits from 11% to 87% across three independently trained policies on held-out test cases.

We also checked whether those adjustments used the current situation. Giving the same policies another case’s starting observation reduced hits to 8%. This supported that Adapt-1 had learned context-dependent corrections, rather than one fixed aiming sequence.

We also tested stopping a carriage and settling its load in Rail, learning controls for a supplied gate → jump → stop sequence in Harbor, and moving, aiming and firing between fresh observations in DOOM. The article explains each setup and its results.

Machina brings continuous-control sequence learning to Adapt-1: learning from executed attempts, revising commands through feedback and retaining them for reuse. Article: Try Adapt-1 Preview:

This supported that Adapt-1 had learned context-dependent corrections, rather than one fixed aiming sequence. 👀

weapons

Grei Grei

Learning continuous control directly from outcomes is a fascinating direction for robotics.

Huge

Great for the robotics unit!

Grei

Very interesting stuff Love the developments GREI

Solid. Learning full control sequences from coarse outcomes only (no demos, no critic) is the key part. Sequence revision + contextual corrections work — AIM jump from ~11% to 87% is the clearest signal. YAM gains are smaller but consistent.

Learning from coarse feedback with no

Can team do something for move rei from this dead spiral

grei

Grei

grei ⭐

this is interesting

Grei

gm

Great update!

you handed it IK and a color blob, searched an open-loop pose tape for 18,400 sim episodes and got 60% cube-in-bowl w/first success at 1,394. 2018 HER already solved cartesian+ik pick-and-place at 100% w/no token needed.

REI passed the “can I run DOOM on it” test. Amazing tech