Skip navigation
Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s…
0:00 / 0:32

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s…

Chad Hurley is Volatile Markets

@volatilemarkts2.6K followers

Visit on X
134 likesDownloadOriginal post

28,183 views

Same Mac. Same model. Same words. My 512 GB M3 Ultra ran GLM-5.3-Flash on MLX last week: 27 tok/s, first token in 0.46 s. Today it runs TensorFold 0.6.2: 61 tok/s, first token in 0.20 s. In the clip it laps the reply twice while the MLX side is still typing. Same reply, byte for byte. Then eight requests at once, 78 tok/s combined, every answer identical to its solo run. The old engine took one at a time. Context stays at 1,048,576 on the Mac. Every number measured on my own machine with the same probe, one week apart. Nothing estimated. Ash Hart built it. Apache 2.0.

11 Comments

Comments from the original post on X (Twitter)

  • Ash Hart's profile picture

    @ashxhart

    Let’s get that to 100tps 👀

    4 likes

  • matt's profile picture

    @thybead

    61 tok/s on the same box is a nice jump. does it hold once the context gets ugly?

    2 likes

  • Volatile Markets's profile picture

    @volatilemarkts

    Like duct tape.

    1 like

  • 1x3e2's profile picture

    @1x3e2

    Yep, it almost doubled the speed on my sparks with the same model. And I have already seen improvements with the new updates and recipes. Excited to see where it goes with so many people being able to find these new techniques.

    2 likes

  • Andrés- e/acc's profile picture

    @pandresgq

    The eight-request result is the real systems datapoint: concurrency exposes memory-bandwidth headroom that single-stream tok/s hides.

    1 like

  • Mia's profile picture

    @MiaAI_lab

    🔥🔥

    1 like

  • Drake Stapleton's profile picture

    @aienscore

    2.25x on the same box, same model, measured with your own probe, is hard to argue with. The first token drop, 0.46 to 0.20, is the part that gets me. That's where interactive use lives or dies.

  • Mike Gannotti 🔳's profile picture

    @MichaelGannotti

    Love it.

  • Brandon's profile picture

    @brandon_lutz

    With vision?

  • chris_paul_walker's profile picture

    @blockbrain_labs

    Crushing

  • Klasta's profile picture

    @AndriusStepona1

    mlx 27 then tf 61 and eight concurrent still matches solo byte for byte