跳过导航
Going back to training single task models with a focus on improving success rates - here’s a longer…
0:00 / 1:31

Going back to training single task models with a focus on improving success rates - here’s a longer…

Ville🤖

@VilleKuosmanen3517 位关注者

在 X 上查看
135 个赞下载原帖

11,845 次观看

Going back to training single task models with a focus on improving success rates - here’s a longer, unedited real-time rollout of an ACT model trained with 40 episodes. Explanation in 🧵

11 条评论

评论来自 X (Twitter) 上的原帖

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    The robot freezing initially was due to a false positive from my safety system. If you can see a regular pattern of fast movements, this is when the action chunk border chances. I used longer action chunks which makes the robot less responsive but more “confident” in movements

    3 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    My previous data was recorded in 50Hz, while data for this model (and the inference) was in 20Hz. I also controlled the robot in a slower and more careful manner. In AI, garbage in -> garbage out so data quality really matters

    3 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    Why 20Hz and not 50Hz? I noticed my cheap cameras were lagging, especially the wrist camera during movements, so many frames would contain an outdated view of the scene. This did not eliminate the issues and buying better cameras is high on the priority list but 20Hz helps

    3 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    The wrist camera’s shakiness is a likely reason why the robot can pick capsules in front of the main camera but struggles with one’s further away, or ones that get occluded by the arm. Perhaps masking one of the cameras (or patches of them) at times during training would help?

    4 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    The data I collected is all from the same table and from 2 different coffee cups - I’ll probably add to the dataset by recording a few episodes from other places in my flat and a few other cups as well. Data diversity matters. A lot.

    2 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    I also want to augment my data from human corrections during inference, dAgger style. This will help the robot align itself back on track, as the original training episodes don’t have many mistakes.

    2 个赞

  • Dan Peña 的头像

    @danspena

    29 Years of proven track record and creating generational wealth! If you want to learn what it takes to becomes super successful then take the success test now!

    7,318 个赞

  • Shreyas Dixit 的头像

    @Shreyas__Dixit

    Question: Why ACT why not Diffusion Policy? I had tried ACT with around 100 samples and lot of epochs (I mean a lot) but it still failed to perform good.

    2 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    I might try diffusion (and VQ-BeT) as well to compare. But ACT is a good baseline and the model I have the most experience using. These experiments exist to see how different techniques can boost success rates, so you could start with any baseline model that's quick to train

    2 个赞

  • Julian Fried 的头像

    @JulianFried

    So strange to watch

    1 个赞

  • Ville Kuosmanen 的头像

    @VilleKuosmanen

    one day it will look indistinguishable from a human operator controlling the robot robot turing test 🤖

    2 个赞