Skip navigation
This is why local models must win.
0:00 / 0:29

This is why local models must win.

Ash Hart

@ashxhart2.9K followers

Visit on X
982 likesDownloadOriginal post

63,065 views

This is why local models must win. Cloud models refused to reverse-engineer Apple's RDMA protocol. DeepSeek and Nemotron said, "hold my beer". TBF, Nemotron needed KV Cache injection to comply lol.

49 Comments

Comments from the original post on X (Twitter)

  • Hana Wilson's profile picture

    @bui1dingoutloud

    yeah its something that annoys me so much about frontier models. i use kimi to reverse engineer firmware to make feature changes, and it rules. work that would have taken me a week to do, done in 15 minutes.

    27 likes

  • Ash Hart's profile picture

    @ashxhart

    It's amazing, isn't it, what you can do these days. What was impossible a few years ago is now normal. Have you tried any other models for reverse engineering apart from Kimi?

    5 likes

  • Hana Wilson's profile picture

    @bui1dingoutloud

    Not yet, I think I will try DeepSeek when my kimi sub runs out. I've so far had 2 success stories: an audio recorder I use has a feature disabled in the US due to a patent. Kimi was able to identify how the firmware identifies the US variant, and was able to do a 2 byte patch along with detecting the crc validation mechanism!

    4 likes

  • Hana Wilson's profile picture

    @bui1dingoutloud

    The other success, different gear from the same company. They have a feature that allows auto locking the device, but its tied to also disabling the screen, which I don't like because having the screen on and visible allows me to easy validate that its still working. Another 2 byte patch, kimi disabled the instruction that sets a memory address that indicates if the screen should be turned on or off! :)

    2 likes

  • Fabian's profile picture

    @onthexitter69

    I’ve heard that Nemotron can be easily convinced to do pretty much anything with a simple rewritten conversation history. Never tried it though.

    13 likes

  • Ash Hart's profile picture

    @ashxhart

    Pretty much what I did, I ran it with an uncensored model then switched to Nemotron to see what it would do. Surprisingly it took off like a champ.

    16 likes

  • Benjamin Ostrov's profile picture

    @b_ostrov

    @ashxhart I’m currently designing rdma through a melanox card to connect spark and Mac Studio ultra m2. How’s the reverse engineering progress going?

    5 likes

  • Ash Hart's profile picture

    @ashxhart

    Hey, nice!! How far have you gotten with it? I have leaned towards the Thunderbolt connection to try and minimise additional hardware for people, but the CX7 route is worthwhile, I believe. Good, I have managed to reverse engineer the RDMA protocol so far but have not managed to get the Spark to enter full USB-C Gen 3.2 2x2 speeds, but I am close.

    7 likes

  • John D. Pope  🦒's profile picture

    @johndpope

    @b_ostrov Take whatever is useful -

    1 like

  • Benjamin Ostrov's profile picture

    @b_ostrov

    @ashxhart Super, it will be useful! So far, we need to beat the connection of Mac Studio M2, there are fewer problems with Spark now 🫠

  • Tømmy. T's profile picture

    @tallmetommy

    The second-order effect of local models getting this capable may be bigger than inference cost/privacy. Once agents can persist and act across heterogeneous local hardware, the hard problem moves upward: which state and authority is allowed to cross when work moves between runtimes? Local inference + portable agent governance feels like where this gets really interesting.

    3 likes

  • clandestine.eth 🦇🔊's profile picture

    @0xClandestine

    dog you need more followers

    2 likes

  • Ash Hart's profile picture

    @ashxhart

    🙌🏻 maybe one day.

    6 likes

  • Dave Talks Politics 🌐's profile picture

    @bigdavetalks

    the news is actually that its now possible to reverse RDMA...

    1 like

  • Ash Hart's profile picture

    @ashxhart

    I’ve already reversed enough of the RDMA service for the Spark to advertise protocol 0xFA57, v1. The harder problem is below that: getting the Spark to enumerate as a genuine USB4/Thunderbolt XDomain peer. That’s the gate to PORT_ACTIVE and an actual one-sided memory transfer.

    4 likes

  • Dave Talks Politics 🌐's profile picture

    @bigdavetalks

    if you succeed this cost undercuts azure by 4x and gcp by 20x. and this assumes spark cost stays at 5k (it wont obv). Keep going!!!!!

    1 like

  • Arthur's profile picture

    @eclictik

    I need to look into this because I've been dreaming for years of rebuilding correct firmware and an app for the @DEVIALET Phantom (their bugs drove me nuts; I'm more at peace now, haha).

    1 like

  • Jorge Ortega's profile picture

    @ortegajorge

    this is the way.

    1 like

  • Alvaro Videla - 🇺🇾🇨🇳🇨🇭🇮🇹's profile picture

    @old_sound

    Are you connecting TB5 to the USB 3.2 on the Spark?

    1 like

  • Ash Hart's profile picture

    @ashxhart

    Yes but it will run at 20 or 40gbps not tb5 speeds.

    5 likes