Ash Hart's banner

Ash Hart

@ashxhart • 3,085 subscribers

Doing my bit to bring local AI to everyone | https://t.co/BWp6YHfYSd | https://t.co/XfmTXso9Us | https://t.co/naxpf8lpU8 | Thoughts are my own.

Shorts

FB marketplace find £20 2009 MacBook in immaculate condition but the HDD was dead. Chucked a 256GB SSD in there along with Ubuntu 24. Thought I would use it as an additional red team box for experiments, siloed, running Pi and being driven by Deepseek V4 0731 Flash from the dual spark cluster. I should install Kali really. Let the fun begin 😎

FB marketplace find £20 2009 MacBook in immaculate condition but the HDD was dead. Chucked a 256GB SSD in there along with Ubuntu 24. Thought I would use it as an additional red team box for experiments, siloed, running Pi and being driven by Deepseek V4 0731 Flash from the dual spark cluster. I should install Kali really. Let the fun begin 😎

30,587 görüntüleme

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Deekseek V4 Flash 0731 driving Kali Linux via Docker to reverse engineer Apples RDMA protocol to allow my Spark and MBP to talk to each other. Wild times.

Deekseek V4 Flash 0731 driving Kali Linux via Docker to reverse engineer Apples RDMA protocol to allow my Spark and MBP to talk to each other. Wild times.

80,284 görüntüleme • 1 ay önce

This is why local models must win. Cloud models refused to reverse-engineer Apple's RDMA protocol. DeepSeek and Nemotron said, "hold my beer". TBF, Nemotron needed KV Cache injection to comply lol.

This is why local models must win. Cloud models refused to reverse-engineer Apple's RDMA protocol. DeepSeek and Nemotron said, "hold my beer". TBF, Nemotron needed KV Cache injection to comply lol.

63,065 görüntüleme • 1 ay önce

iPad connected over WiFi AP to the Spark running DeepSeek V4 Flash. Imagine rolling up to the coffee shop with this naughty little setup. Harness on the iPad time 👀😅 NVIDIA AI vLLM Open WebUI Apple

iPad connected over WiFi AP to the Spark running DeepSeek V4 Flash. Imagine rolling up to the coffee shop with this naughty little setup. Harness on the iPad time 👀😅 NVIDIA AI vLLM Open WebUI Apple

49,033 görüntüleme • 1 ay önce

Qwen 3.8 Next Flash 4bit MTP on my M3U Studio is absolutely flying now. My latest OMP run measured 97.1 tok/s overall decode at 23k context, with ~1,131 tok/s uncached PP. PR opened on oMLX for this.

Qwen 3.8 Next Flash 4bit MTP on my M3U Studio is absolutely flying now. My latest OMP run measured 97.1 tok/s overall decode at 23k context, with ~1,131 tok/s uncached PP. PR opened on oMLX for this.

21,917 görüntüleme • 23 gün önce

Been giving my M3 Ultra Studio some love and tinkering with MLX. Started the afternoon at 42 tok/s. Now: 73 tok/s, and there’s still more on the table. Running Qwen3.8-Flash-Next @ 4-bit locally. Now for the crazy bit: Artificial Analysis Intelligence Index v4.3: GPT-5.6 Sol 39 (Medium) GPT-5.6 Sol 42 (High) Qwen3.8-Flash-Next 42 GPT-5.6 Sol 44 (XHigh) GPT-5.6 Sol 47 (Max) Grok 4.5 39 (High) Claude Sonnet 5 38 (Max) *Qwen score currently estimated by Artificial Analysis. A 4-bit open-weights model running at 73 tok/s on a Mac under my desk... with an estimated intelligence score matching GPT-5.6 Sol at High reasoning. Local AI is getting ridiculous. 🔥 Plus, this model does not say no to a bit of red teaming. Once I am happy with it, I will do a pr for Jun Kim's oMLX :)

Been giving my M3 Ultra Studio some love and tinkering with MLX. Started the afternoon at 42 tok/s. Now: 73 tok/s, and there’s still more on the table. Running Qwen3.8-Flash-Next @ 4-bit locally. Now for the crazy bit: Artificial Analysis Intelligence Index v4.3: GPT-5.6 Sol 39 (Medium) GPT-5.6 Sol 42 (High) Qwen3.8-Flash-Next 42 GPT-5.6 Sol 44 (XHigh) GPT-5.6 Sol 47 (Max) Grok 4.5 39 (High) Claude Sonnet 5 38 (Max) *Qwen score currently estimated by Artificial Analysis. A 4-bit open-weights model running at 73 tok/s on a Mac under my desk... with an estimated intelligence score matching GPT-5.6 Sol at High reasoning. Local AI is getting ridiculous. 🔥 Plus, this model does not say no to a bit of red teaming. Once I am happy with it, I will do a pr for Jun Kim's oMLX :)

18,827 görüntüleme • 28 gün önce

Computer Use for your favourite CLI.

Computer Use for your favourite CLI.

16,587 görüntüleme • 27 gün önce

I put MCDMA through its paces today and tested Qwen 3.8 Flash Next on my dual Spark / Mac Studio cluster. My first test was disaggregated prefill across my two Sparks, then decode onto my Studio. PP 2,100 tok/s Decode 80 tok/s at 20k context, higher on short replies. That's faster than either machine manages on its own: the Studio alone prefills at about 1,200 tok/s, and the Sparks alone decode at 35 to 65. 🤯 Mac prefill has always been way behind the Sparks; now I get the best of both worlds with the hardware I currently have.

I put MCDMA through its paces today and tested Qwen 3.8 Flash Next on my dual Spark / Mac Studio cluster. My first test was disaggregated prefill across my two Sparks, then decode onto my Studio. PP 2,100 tok/s Decode 80 tok/s at 20k context, higher on short replies. That's faster than either machine manages on its own: the Studio alone prefills at about 1,200 tok/s, and the Sparks alone decode at 35 to 65. 🤯 Mac prefill has always been way behind the Sparks; now I get the best of both worlds with the hardware I currently have.

13,121 görüntüleme • 21 gün önce

Daha fazla içerik yok.