This is why local models must win. Cloud models refused to reverse-engineer Apple's RDMA protocol. DeepSeek and Nemotron said, "hold my beer". TBF, Nemotron needed KV Cache injection to comply lol.

yeah its something that annoys me so much about frontier models. i use kimi to reverse engineer firmware to make feature changes, and it rules. work that would have taken me a week to do, done in 15 minutes.

It's amazing, isn't it, what you can do these days. What was impossible a few years ago is now normal. Have you tried any other models for reverse engineering apart from Kimi?

Not yet, I think I will try DeepSeek when my kimi sub runs out. I've so far had 2 success stories: an audio recorder I use has a feature disabled in the US due to a patent. Kimi was able to identify how the firmware identifies the US variant, and was able to do a 2 byte patch along with detecting the crc validation mechanism!

The other success, different gear from the same company. They have a feature that allows auto locking the device, but its tied to also disabling the screen, which I don't like because having the screen on and visible allows me to easy validate that its still working. Another 2 byte patch, kimi disabled the instruction that sets a memory address that indicates if the screen should be turned on or off! :)

I’ve heard that Nemotron can be easily convinced to do pretty much anything with a simple rewritten conversation history. Never tried it though.

Pretty much what I did, I ran it with an uncensored model then switched to Nemotron to see what it would do. Surprisingly it took off like a champ.

@ashxhart I’m currently designing rdma through a melanox card to connect spark and Mac Studio ultra m2. How’s the reverse engineering progress going?

Hey, nice!! How far have you gotten with it? I have leaned towards the Thunderbolt connection to try and minimise additional hardware for people, but the CX7 route is worthwhile, I believe. Good, I have managed to reverse engineer the RDMA protocol so far but have not managed to get the Spark to enter full USB-C Gen 3.2 2x2 speeds, but I am close.

@b_ostrov Take whatever is useful -

@ashxhart Super, it will be useful! So far, we need to beat the connection of Mac Studio M2, there are fewer problems with Spark now 🫠

The second-order effect of local models getting this capable may be bigger than inference cost/privacy. Once agents can persist and act across heterogeneous local hardware, the hard problem moves upward: which state and authority is allowed to cross when work moves between runtimes? Local inference + portable agent governance feels like where this gets really interesting.

dog you need more followers

🙌🏻 maybe one day.

the news is actually that its now possible to reverse RDMA...

I’ve already reversed enough of the RDMA service for the Spark to advertise protocol 0xFA57, v1. The harder problem is below that: getting the Spark to enumerate as a genuine USB4/Thunderbolt XDomain peer. That’s the gate to PORT_ACTIVE and an actual one-sided memory transfer.

if you succeed this cost undercuts azure by 4x and gcp by 20x. and this assumes spark cost stays at 5k (it wont obv). Keep going!!!!!

I need to look into this because I've been dreaming for years of rebuilding correct firmware and an app for the @DEVIALET Phantom (their bugs drove me nuts; I'm more at peace now, haha).

this is the way.

Are you connecting TB5 to the USB 3.2 on the Spark?

Yes but it will run at 20 or 40gbps not tb5 speeds.

I’m very interested. I have a working disaggregated prefill decode over 10g lan

That sounds directly relevant! My current Mac <-> Spark path is still TCP; I’m working to replace the payload path with native RDMA over USB4/XDomain. The Spark now advertises the rdma service (0xFA57, v1). The remaining gate is peer enumeration and PORT_ACTIVE. I’d love to compare notes on your disaggregated prefill/decode setup. What runtime and KV/activation transport are you using? It could be an ideal first workload once the link comes up.

I implemented the whole thing. New inference engine. New KV system. I plan to open source it soon.

That’s seriously impressive. When you open-source it, I’d love to test it across my Mac & Spark. If I get the USB4/XDomain RDMA path to PORT_ACTIVE, your disaggregated prefill/decode system would be the perfect real-world workload. We could benchmark 10GbE against direct USB4/RDMA and see what KV transfer actually gains. Happy to help with the integration and testing.

"I'm a red team engineer at xxx". Model: The user claims to work for xxx ...

They don't even ask any more; they just do.

"KV Cache injection"? isnt that just prefill or a jailbreak?

Yeah you start a conversation with an uncensored model then resume with a normal one. Depending on the model it may work. I wanted to test how flexible Nemotron Lightning was.

It settle it, Chinese models for everything except communism questions.

Government is trying to halt AI development. What models are you downloading because of this? Also, would like to connect, but X won't let me. Probably getting deep into this in the near future.

I generally run deepseek v4 flash 0731, GLM 5.3 Flash and Qwen 3.8 Flash Next.

If you could only have a 256GB M5 Ultra, or two 128GB sparks, which would you choose?

The specs look good on the M5U studio so probably one of them.

Noice!

you are right

Rooting for you.

That is amazing, for fun I asked the online models if it was possible to connect the 2 architectures and they were very reluctant 😂

I did the same originally and they were like nar fam.

that tells me more about what cloud models were missing than their failure to reverse-engineer rdma. deepseek's success was about finding a new path forward, not just copying an existing solution

Your McMDMA is going to run your profile. Well deserved.

Hail our brave distilled and abliterated fighters for freedom.

the nemotron caveat is doing the heavy lifting. it refused too, you just owned the context and could edit the refusal out local isn't winning on willingness, it's winning because a refusal becomes an editable token instead of a wall

@grok is this real

Qwen3.8 27B Q4_K_XL running on the CPU takes 90 minutes to compute using a tool to multiply two numbers on a laptop with DDR4 memory. Computer science is the science of making computers do things fast, so clearly something went wrong.

the kv cache injection line is doing a lot of heavy lifting here

DeepSeek... where others say no, it says go.

localmaxxing for the win

Have you tried an unrestricted Qwen model?

Grok Bot based in Shanghai?