First successes in training the robot policy for chess movements! 🧵

I’ve trained an ACT model on 100 samples of either the top or bottom rook moving ahead two squares (50-50 split). The policy shown is trained for 20k steps but I’ve trained up to 100k and will evaluate it in the same lighting conditions as the 20k policy tomorrow. You can find the dataset here:

Overall accuracy is 23.3%: 7 successes in 30 evaluations. Most often the gripper misses the rook as in this example.

In three cases the gripper failed to release the pawn at the right moment or at all (like in this example).

Wanna learn more? I keep a daily blog on my website where I document what I work on, learn and think about.

Let’s gooooo! Guess I can start looking for flights now to get my ass kicked by robo chess ;)

Awesome!!

Scan any documents, convert images into text, PDF files, etc. 👍

lets gooo

this is sick!

Is it trained on the same position you are picking on? I’ve trained ACT on 150 episodes for cube picking and it fails to generalize to other positions except the trained ones. Even on trained positions it’s like 50% accuracy or lesss

I collected data for two positions/movements that I jointly train on. I used the visual hints that you see in the video to indicate which piece it should be moving where. Accuracy is 23%. I haven’t tried other positions, but would have low expectations. I‘ll need much more data!