simon's banner
simon's profile picture

simon

@nautsimon_ • 1,437 subscribers

Hardware Verification @Axiommathai, 🚗 @drivingbench | prev PyTorch/MTIA @meta, @uchicago

Shorts

Over the weekend, I put an unreleased checkpoint of π0.7 inside an excavator and placed 2nd at the Actor + Physical Intelligence 36hr hackathon. With about an hour of embodied data (3 cameras and joystick values, no joint angles), the policy could continuously move dirt with a single prompt. This was so straightforward to get working (pi is cooking) we spent the rest of the hackathon mounting a Nerf minigun on the roof. We added a people searching/tracking + firing feature to prevent people from getting too close to the dangerous machinery (ai safety feature). Here’s how we built it + more details (data collection, hardware hax, sim2real, video of me getting shot)🧵

Over the weekend, I put an unreleased checkpoint of π0.7 inside an excavator and placed 2nd at the Actor + Physical Intelligence 36hr hackathon. With about an hour of embodied data (3 cameras and joystick values, no joint angles), the policy could continuously move dirt with a single prompt. This was so straightforward to get working (pi is cooking) we spent the rest of the hackathon mounting a Nerf minigun on the roof. We added a people searching/tracking + firing feature to prevent people from getting too close to the dangerous machinery (ai safety feature). Here’s how we built it + more details (data collection, hardware hax, sim2real, video of me getting shot)🧵

51,268 views