Binh

@pham_blnh • 3,850 subscribers

breaking robots @livekit ↝ prev fellow @shipfr8 and @southpkcommons

Shorts

whoever made the robot catch the mouse cursor on Rerun cooked

whoever made the robot catch the mouse cursor on Rerun cooked

302,531 Aufrufe

since the video is sped up by 10x, the real time (1x speed) would look like this?

since the video is sped up by 10x, the real time (1x speed) would look like this?

57,787 Aufrufe

the so101 + leslider frame i've been playing with the past few weeks wasn't just a random rig i built for fun it's what we at LiveKit built to benchmark and debug our infra (s/o livekit-portal), and today we're open-sourcing it so everyone can reproduce and share coolio results at home here is the github: i know this isn’t the first frame designed for evals, but it’s ours, and it’s open, and it’s with a LeSlider! along with this, we’ve released simulations files (URDF, MJCF, and USD), as well as a sample RL env in MJLab for pick-and-place tasks we want people to have fun, being close to the frontier as much as possible, and we want people to know it shouldn’t take thousands of dollars in equipment to reproduce some of the most amazing results the academic world has to offer personally, I’m going to deploy sim2real and real2sim2real pipelines on this soon, for fun, so look out for that.

the so101 + leslider frame i've been playing with the past few weeks wasn't just a random rig i built for fun it's what we at LiveKit built to benchmark and debug our infra (s/o livekit-portal), and today we're open-sourcing it so everyone can reproduce and share coolio results at home here is the github: i know this isn’t the first frame designed for evals, but it’s ours, and it’s open, and it’s with a LeSlider! along with this, we’ve released simulations files (URDF, MJCF, and USD), as well as a sample RL env in MJLab for pick-and-place tasks we want people to have fun, being close to the frontier as much as possible, and we want people to know it shouldn’t take thousands of dollars in equipment to reproduce some of the most amazing results the academic world has to offer personally, I’m going to deploy sim2real and real2sim2real pipelines on this soon, for fun, so look out for that.

18,257 Aufrufe

There are currently 61 humanoid companies spanning 13 countries with a total of 88 humanoids Spent my free time creating an extensive display of humanoid companies to learn the current landscape Opinions: - Super saturated, every developed country fund and build humanoid robots, even developing countries like Vietnam has their own humanoid. - Software is the moat. Breakthrough in software will explode adaptation of robots in both consumer and industry. Observation: - Naming: Robots tends to be named in the following formula: Name + Generation (i.e. MagicBot Gen 1, Vanar Gen 1) or Name + Random Letter + Number (i.e. Jaka K1, Kinisi 01, Unitree G1). - Two diversion in use case: One side focus on utility (Figure, X1, etc.), the other focus on companionship (Enchanted Tools, Andromeda, etc.) Category imo: - Unique Design: Fourier, Enchanted Tools, Slightly Overdone Robotics, Pollen Robotics. - Unique Actuator: Clone Robotics - Best Capabilities: too soon to tell, too much online cherry picked demos instead of real life demonstration. - Most Natural Motion: Cartwheel Robotics, Pollen Robotics. - Highest Cost: Tesla. Hmu or comment below if you want the list.

There are currently 61 humanoid companies spanning 13 countries with a total of 88 humanoids Spent my free time creating an extensive display of humanoid companies to learn the current landscape Opinions: - Super saturated, every developed country fund and build humanoid robots, even developing countries like Vietnam has their own humanoid. - Software is the moat. Breakthrough in software will explode adaptation of robots in both consumer and industry. Observation: - Naming: Robots tends to be named in the following formula: Name + Generation (i.e. MagicBot Gen 1, Vanar Gen 1) or Name + Random Letter + Number (i.e. Jaka K1, Kinisi 01, Unitree G1). - Two diversion in use case: One side focus on utility (Figure, X1, etc.), the other focus on companionship (Enchanted Tools, Andromeda, etc.) Category imo: - Unique Design: Fourier, Enchanted Tools, Slightly Overdone Robotics, Pollen Robotics. - Unique Actuator: Clone Robotics - Best Capabilities: too soon to tell, too much online cherry picked demos instead of real life demonstration. - Most Natural Motion: Cartwheel Robotics, Pollen Robotics. - Highest Cost: Tesla. Hmu or comment below if you want the list.

65,669 Aufrufe

turned my Seeed Studio sensecap watcher into a dasai mochi roughly 50 animations atm to cover all sorts of emotions gonna create a LiveKit agent with this next

turned my Seeed Studio sensecap watcher into a dasai mochi roughly 50 animations atm to cover all sorts of emotions gonna create a LiveKit agent with this next

16,508 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

I ported PhysX to Radeon and won the global AMD robotics hackathon Here is my winning video Details in thread 🧵

I ported PhysX to Radeon and won the global AMD robotics hackathon Here is my winning video Details in thread 🧵

48,367 Aufrufe • vor 13 Tagen

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at

you don’t need a robot to see physical ai in action, you just need a browser. this week I finally tried something I've wanted to do for a while: design an SO-101 scene in three.js, connect it to a real leader arm, collect teleoperation data, train an ACT model, then port it to run in the browser. the result is the first ever demo of ACT that runs locally on the web. try it yourself now at

121,383 Aufrufe • vor 2 Monaten

Just become the world’s first to run an LLM locally on a business card lol. Why? For fun. Details: - llama2.c by Andrej Karpathy with modifications from Dave Bennett - 24tok/sec on esp32s3 n16r8 w/ a 260k param model - designed the card myself, whole inspiration comes from ouija boards i.e talking with ghosts. if i put llms on my business card, it ought to be the ouija form factor lol. - pcb and smt sponsored by NextPCB Dropping video on youtube + schematics this weekend

Just become the world’s first to run an LLM locally on a business card lol. Why? For fun. Details: - llama2.c by Andrej Karpathy with modifications from Dave Bennett - 24tok/sec on esp32s3 n16r8 w/ a 260k param model - designed the card myself, whole inspiration comes from ouija boards i.e talking with ghosts. if i put llms on my business card, it ought to be the ouija form factor lol. - pcb and smt sponsored by NextPCB Dropping video on youtube + schematics this weekend

385,996 Aufrufe • vor 1 Jahr

i bet you haven't seen a SO101 mounted on a wall like this before if you want to do the same, here is LeSlider: i built it cause i wanted something that can cover my whole desk for tasks like organizing and cleaning i originally wanted to have a belt system like what 3D printers have, but i was too lazy and used a pinion/track with another sts3215 so: > the extra motor shares the same bus as the rest of SO101 > you can have arbitrary length of track > really cheap and easy to assemble and control it turned out better than expected with this, i'm gonna train a model to pick random stuff up across my table and put it into a bin at the end of table (realistically using yolo to scan table, two policies, one for picking up objects, one for dropping)

i bet you haven't seen a SO101 mounted on a wall like this before if you want to do the same, here is LeSlider: i built it cause i wanted something that can cover my whole desk for tasks like organizing and cleaning i originally wanted to have a belt system like what 3D printers have, but i was too lazy and used a pinion/track with another sts3215 so: > the extra motor shares the same bus as the rest of SO101 > you can have arbitrary length of track > really cheap and easy to assemble and control it turned out better than expected with this, i'm gonna train a model to pick random stuff up across my table and put it into a bin at the end of table (realistically using yolo to scan table, two policies, one for picking up objects, one for dropping)

98,324 Aufrufe • vor 4 Monaten

today is the last day fable 5 is in the claude code plan, so i gave it something i've always wanted to build: polytopia rebuilt in pure C, RL agents trained from scratch, playable in the browser completely vibecoded. i didn't write a line. even the game assets were ai-generated using a pipeline fable created. 🧵 below

today is the last day fable 5 is in the claude code plan, so i gave it something i've always wanted to build: polytopia rebuilt in pure C, RL agents trained from scratch, playable in the browser completely vibecoded. i didn't write a line. even the game assets were ai-generated using a pipeline fable created. 🧵 below

46,021 Aufrufe • vor 1 Monat

how i recorded training data for the web viz btw the project is very simple: > threejs handles the rendering and the cameras > mujoco handles the physics and robot states > a python websocket server handles receiving observations from the website and sending leader actions from an actual leader arm > the rest is just the usual lerobot stack

how i recorded training data for the web viz btw the project is very simple: > threejs handles the rendering and the cameras > mujoco handles the physics and robot states > a python websocket server handles receiving observations from the website and sending leader actions from an actual leader arm > the rest is just the usual lerobot stack

30,956 Aufrufe • vor 2 Monaten

hahaha we’re so close to getting sim2real to work y’all after 33 iterations and training runs, this is the closest i got to a successful zero-shot task gonna tackle this again with a fresh mind next week, got hit with the flu today but as you can see i was so happy with the rollout

hahaha we’re so close to getting sim2real to work y’all after 33 iterations and training runs, this is the closest i got to a successful zero-shot task gonna tackle this again with a fresh mind next week, got hit with the flu today but as you can see i was so happy with the rollout

13,948 Aufrufe • vor 1 Monat

check out my claude knob Thariq

check out my claude knob Thariq

20,557 Aufrufe • vor 3 Monaten

day 2 of building a self-driving power wheel today i officially trained a self driving model from scratch and deploy it on the car by just simply brute forcing everything, I: > made a remote tele-op and remote data collection app built on LiveKit infra > feat: 60ms e2e latency between the car and inference compute (car and compute in vietnam with singapore sfu) > feat: data is collected on operator side, baking latency into observation space itself (I expect this made the model more robust against latency) > recorded 30 min of data at 30fps and converted the dataset to lerobot (you can check a sample here) > trained a simple ACT model (3 epoch, batch size 8) to drive the car around my house > deployed the model on the car with remote inference the video explains everything shortly reflection: > the model is ofc bad, idt behavior cloning would work at all for such complex task on such small sample size > it did work in some cases where the observation is well within distribution, even generalizes to back the car when it gets stuck up next: > will hack alpamayo (NVIDIA) or comma ’s e2e to somehow fit this > or train with a llm backbone or a locomotion prior to see if it generalizes

day 2 of building a self-driving power wheel today i officially trained a self driving model from scratch and deploy it on the car by just simply brute forcing everything, I: > made a remote tele-op and remote data collection app built on LiveKit infra > feat: 60ms e2e latency between the car and inference compute (car and compute in vietnam with singapore sfu) > feat: data is collected on operator side, baking latency into observation space itself (I expect this made the model more robust against latency) > recorded 30 min of data at 30fps and converted the dataset to lerobot (you can check a sample here) > trained a simple ACT model (3 epoch, batch size 8) to drive the car around my house > deployed the model on the car with remote inference the video explains everything shortly reflection: > the model is ofc bad, idt behavior cloning would work at all for such complex task on such small sample size > it did work in some cases where the observation is well within distribution, even generalizes to back the car when it gets stuck up next: > will hack alpamayo (NVIDIA) or comma ’s e2e to somehow fit this > or train with a llm backbone or a locomotion prior to see if it generalizes

19,404 Aufrufe • vor 5 Monaten

meet bopi, my backpack pet built on top of LiveKit's esp32 sdk, this little kewtie pie can spawn occasionally and talk you out of boredom with cute faces unlike hardware product launches recently, you can buy and build your bopi today i also have a completely offline version where bopi can go into deep sleep and survive for a few days on your backpack source code and instructions below

meet bopi, my backpack pet built on top of LiveKit's esp32 sdk, this little kewtie pie can spawn occasionally and talk you out of boredom with cute faces unlike hardware product launches recently, you can buy and build your bopi today i also have a completely offline version where bopi can go into deep sleep and survive for a few days on your backpack source code and instructions below

17,312 Aufrufe • vor 5 Monaten

we won the embodied ai hackathon at South Park Commons last week this is how we did it 🧵

we won the embodied ai hackathon at South Park Commons last week this is how we did it 🧵

10,501 Aufrufe • vor 3 Monaten

Keine weiteren Inhalte verfügbar