"run the model on the robot: cloud is too slow." been experimenting. Same VLA, same arms, same task. our custom cloud engine: 5.1x faster compute. 1.7x faster end-to-end round-trip (more in thread)

when experimenting with MolmoAct2 on bimanual i2rt YAM arms we got 18Hz to the arms on edge box spends 87% of every cycle just thinking kinda crazy

if we pull inference off the robot it gets lighter, cooler, more efficient and can run big server-side models that hold up in complex terrain only bottleneck is latency + live comms. that's the infra we need to build if we want robots in space and we're doing it.

this is great man! i tried solving the latency issue by temporal ensembling but the next chunk kept landing slower than I could roll the previous one out, so there was no overlap window left to average over lol interesting to see the 1.7x number (will be experimenting with a gpu cluster closer to home now)

yeah that’s the failure mode. for us, overlap goes to 0 the second round-trip beats your chunk horizon. cloud is great for that we have lower latency end-to-end with only 1-3ms std dev. local jitter was all over the place, and that variance is really what kills the overlap imo

@pabloberlangab I'm not sure about internet access in the real “cloud” when you're climbing Everest, but I suppose there can be network issues. If so, I don't think cloud computing is a great choice :)

starlink for the win video on how that works soon

@pabloberlangab Elon should repost this 😁

yes he should my tweets are bangers

@pabloberlangab I would bet all of my life savings that the Thor deployment is horribly unoptimized and latency could easily match cloud with better (specialized towards edge/vla) software.

@pabloberlangab cloud still adds network jitter in real deployments

yes you need good event orchestration when operating with starlink to account for obstructions and change in satellites more on that soon

Been thinking about this there is so much compute limitation on local robotics. Even if we are to increase the local modal there would be a wattage and battery limitation as well. Wouldn’t be surprised that there is a hybrid solution to this. Let’s say in a factory or a house there is a compute infrastructure(server of sort) to handle the heavy compute processing. Something I do wonder is the speed for vision processing. Will think about it more as I work on it.

@pabloberlangab Locally run models will always be better. As we are seeing now, the new models are shifting away from needing massive amounts of computational power

completely disagree, if you look at Claude for instance. moment you could run sonnet locally, fable arrived. you will always have better bigger models if you can tap in the cloud.

@pabloberlangab 1.7x faster cloud round-trip is surprising. Latency breakdown?

about 100ms for inference + 200ms for communication when doing cloud vs 500ms when doing edge, graphs available in second video in thread!

@pabloberlangab 👍 当本地推理耗时大于网络延时时，云端模型就具备理论优势了 一般本地算力都不会太高，所以我认为端云结合，端侧采集云端推理，是很好的方案

@pabloberlangab Cost difference?

cloud is pay as you go and available to everyone, edge you might have to spend upwards of $3000 just to get access to it

@pabloberlangab @batsuev_es

@pabloberlangab Hey chief..I can help to improve your market,let chat briefly..i dm you

@pabloberlangab The problem is not quite speed as much as it is at the intersection of fault tolerance, expeditionary mission support (what if my robot is under a tunnel? In that one room with bad WiFi?), privacy, and recurring costs

@pabloberlangab That sounds like something I gotta try 👀

for sure movements are sooo clean on cloud

@pabloberlangab Haha dooope I really wanna try a more advanced model but my old ass local GPU is soooo weak…