Speed running my home AI cluster running distributed inference across 2 MacBooks and 2 Mac Minis. exo intern displays a real-time network topology as devices discover each other over the local network. Code is open source 👇

The code is in the @exolabs_ GitHub repo here:

@mo_baioumy @exolabs_ To me this proves that compute will come back to local on edge particularly for sensitive data - the era of 100% cloud is cycling out and now we are going to have hybrid computing between edge and cloud.

@mo_baioumy @exolabs_ This thesis is why we started @exolabs_

The new live network topology view also shows which device is active in the inference in real-time.

@exolabs_ Man , no matter how many times i see this, this is still so fkin cool

@exolabs_ Did you notice it highlights which device is active (in red) in real-time? :)

Uses MLX, Apple’s open source ML library h/t @awnihannun

@exolabs_ is there an api possibility on it? like openai or anthropic compatible? this project is so cool. wow. could we connect my macs ipad and linux with 3 4090s to create a frankestein cluster. i can borrow my gf devices as well haha

@exolabs_ Already supports ChatGPT-compliant API endpoint:

@exolabs_ Are there plans on supporting Thunderbolt connection to increase the inference speed in between the Macs? I think that would be a game changer!

@exolabs_ Should work out of the box, made an issue here to double check: