THIS AI DEVELOPER RUNS 120B PARAMETER MODELS LOCALLY AND HIS MACBOOK PRO CAN’T EVEN OPEN THEM
leopardracer
Visit on X12K followers
85,838 views
THIS AI DEVELOPER RUNS 120B PARAMETER MODELS LOCALLY AND HIS MACBOOK PRO CAN’T EVEN OPEN THEM
he bought a DGX Spark, gave it the model, went back to his desk
by end of day the inference was done, tested, committed
his team is still waiting for the cloud GPU queue to free up
the only thing between them is 128GB of unified memory in a box smaller than your lunch
one Spark is a datacenter node, two Sparks is your own cluster
bookmark this and send it to whoever is still paying for cloud GPUs
Comments
Comments from the original post on X (Twitter)
No comments yet
