Binh
@pham_blnh • 4,183 subscribers
breaking robots @livekit ↝ prev fellow @southpkcommons and @shipfr8
Shorts
Videos
I just made the most expressive real-time motion harness for the Reachy Mini. It was created by: > synthesizing a massive motion dataset using Astra (over 10000 samples) > fine-tuning Qwen 3.5 4B using such dataset > then training a flow matching transformer head to transform sparse motion plan (<1hz) into dense expressive motion (25Hz) The average inference time for a full motion is less than 200ms, enabling real-time interaction. Pollen Robotics and Hugging Face only recorded 85 emotions for the reachy mini. With my work, you can now prompt and animate any motion you want. Everything is open-source; write-up and links attached below. Here is a video of Reachy explaining itself through my harness.
Binh30,130 görüntüleme • 15 gün önce
Just become the world’s first to run an LLM locally on a business card lol. Why? For fun. Details: - llama2.c by Andrej Karpathy with modifications from Dave Bennett - 24tok/sec on esp32s3 n16r8 w/ a 260k param model - designed the card myself, whole inspiration comes from ouija boards i.e talking with ghosts. if i put llms on my business card, it ought to be the ouija form factor lol. - pcb and smt sponsored by NextPCB Dropping video on youtube + schematics this weekend
Binh386,139 görüntüleme • 1 yıl önce
you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at
Binh122,647 görüntüleme • 3 ay önce
I ported PhysX to Radeon and won the global AMD robotics hackathon Here is my winning video Details in thread 🧵
Binh49,741 görüntüleme • 2 ay önce
i bet you haven't seen a SO101 mounted on a wall like this before if you want to do the same, here is LeSlider: i built it cause i wanted something that can cover my whole desk for tasks like organizing and cleaning i originally wanted to have a belt system like what 3D printers have, but i was too lazy and used a pinion/track with another sts3215 so: > the extra motor shares the same bus as the rest of SO101 > you can have arbitrary length of track > really cheap and easy to assemble and control it turned out better than expected with this, i'm gonna train a model to pick random stuff up across my table and put it into a bin at the end of table (realistically using yolo to scan table, two policies, one for picking up objects, one for dropping)
Binh Pham98,324 görüntüleme • 5 ay önce
it took a lot of prep, but i managed to reproduce this instead of just having astra figure out everything (since i’m token poor), i told it to first do a real2sim of the environment, then move the robot around to check for alignment, then draw the trajectories will be validated first in sim, then transferred to real this saves tokens spent and makes astra draw faster it’s also a proof of how crazy the new model at both few-shot real2sim and robot control below is an attempt to draw a flower meadow using this method
Binh23,368 görüntüleme • 1 ay önce
today is the last day fable 5 is in the claude code plan, so i gave it something i've always wanted to build: polytopia rebuilt in pure C, RL agents trained from scratch, playable in the browser completely vibecoded. i didn't write a line. even the game assets were ai-generated using a pipeline fable created. 🧵 below
Binh Pham46,021 görüntüleme • 3 ay önce
for a long time i've wondered why open-loop policies and action chunks dominate modern robotics. it's counter-intuitive for me that an action model would work best when executing actions blindly. so last week, inspired by recent writings, i took a stab at understanding the problem space myself and simply did some testing. my latest mini experiment is here:
Binh11,350 görüntüleme • 1 ay önce
how i recorded training data for the web viz btw the project is very simple: > threejs handles the rendering and the cameras > mujoco handles the physics and robot states > a python websocket server handles receiving observations from the website and sending leader actions from an actual leader arm > the rest is just the usual lerobot stack
Binh Pham30,956 görüntüleme • 3 ay önce
hahaha we’re so close to getting sim2real to work y’all after 33 iterations and training runs, this is the closest i got to a successful zero-shot task gonna tackle this again with a fresh mind next week, got hit with the flu today but as you can see i was so happy with the rollout
Binh14,082 görüntüleme • 3 ay önce
day 2 of building a self-driving power wheel today i officially trained a self driving model from scratch and deploy it on the car by just simply brute forcing everything, I: > made a remote tele-op and remote data collection app built on LiveKit infra > feat: 60ms e2e latency between the car and inference compute (car and compute in vietnam with singapore sfu) > feat: data is collected on operator side, baking latency into observation space itself (I expect this made the model more robust against latency) > recorded 30 min of data at 30fps and converted the dataset to lerobot (you can check a sample here) > trained a simple ACT model (3 epoch, batch size 8) to drive the car around my house > deployed the model on the car with remote inference the video explains everything shortly reflection: > the model is ofc bad, idt behavior cloning would work at all for such complex task on such small sample size > it did work in some cases where the observation is well within distribution, even generalizes to back the car when it gets stuck up next: > will hack alpamayo (NVIDIA) or comma ’s e2e to somehow fit this > or train with a llm backbone or a locomotion prior to see if it generalizes
Binh Pham19,404 görüntüleme • 7 ay önce
meet bopi, my backpack pet built on top of LiveKit's esp32 sdk, this little kewtie pie can spawn occasionally and talk you out of boredom with cute faces unlike hardware product launches recently, you can buy and build your bopi today i also have a completely offline version where bopi can go into deep sleep and survive for a few days on your backpack source code and instructions below
Binh Pham17,312 görüntüleme • 6 ay önce
we won the embodied ai hackathon at last week this is how we did it 🧵
Binh10,669 görüntüleme • 4 ay önce
Daha fazla içerik yok.