A few weeks ago, OpenAI's models started refusing normal work for my business. So I built 3 servers with 12 AMD R9700 GPUs and moved the workload in house. $32.2k total. Here's the whole story in 2 minutes. #LocalAI

Plan A was more Mac Studios. We already had six, but keeping up would have taken at least 41 more. That's $280k. Plan B was building our own. On our real workload, six Macs did 12 jobs/min. The 12 R9700s do 175. About 15x the work for about the same money

Each server runs on a regular 15-amp home circuit. The GPUs do the AI work, so the rest of each server runs our CI/CD. I get the argument that "the cloud is cheaper." But owning my business is the real goal, and nobody can change these models on me.

This is the way in my opinion. I don't know how CTOs and that right now are even able to do a budget with all the changes in cloud models that is out of their control and things can change over night with 0 notice.

Nice, love to see it.

I love the handle @WeLikeShooting

This is pretty awesome. Im curious about the harness, did you/your company did it or was it a third party? I see you are hitting 53 tok/s, whereas AMD publishes 51. You a have a pretty solid setup right there.

Working on a video about the harness and how to build a good one as I'm rewriting production again for like the 10th time haha. What do you want to know. What should I include in the video?

respect, that's a serious level of commitment 💪

Each server is solo? 4 cards per? What specs?

love it. i've got a similar system, though only 2 R9700s (went with a Ryzen vs Epyc/Threadripper).

Have you tried comparing the results of your model vs those of OpenAI and Anthropic? I guess what I want to ask is: are you not sacrificing a lot of accuracy?

Well my work is getting refused by OpenAI and candidly Qwen 3.8 27b on our own workflows collects about 2 more data points than OpenAI Luna given the exact same prompts.

@AAGresearch You’re the one managing this 🦑 or you have a team of devs 👨🏼‍💻👩‍💻working for you

@AAGresearch It is all me.

this is the way 🕳🐇

Everyone is gonna want to set this up for their businesses. At least I do. And maybe with water cooling so I can run a jacuzzi for free.