Skip navigation
My Mac Studio M3 Ultra has an identity problem. It thinks it's an M5.
0:00 / 0:21

My Mac Studio M3 Ultra has an identity problem. It thinks it's an M5.

Chad Hurley is Volatile Markets

@volatilemarkts2.6K followers

Visit on X
66 likesDownloadOriginal post

16,901 views

My Mac Studio M3 Ultra has an identity problem. It thinks it's an M5. Here's the proof.. Same box. Same model, GLM-5.3-Flash. Same question, asked twice. Old engine: 27 tok/s. You read along and wait. TensorFold: 58 tok/s, first token in 0.08 s. Now you're the one keeping up. Nothing under the lid changed. No new chip, no upgrade money spent. The only thing that changed is the engine serving the weights with: What is this magic: a prompt cache technique, so a repeated prompt resumes instantly instead of being re-read. Draft decoding that only accepts a guess when it matches what the model would say token by token, so the speed never changes the answer. Standard OpenAI API, so nothing I'd built had to change. 1M context. Several requests at once. Last month I figured the M3s would always trail my DGX Sparks. Not anymore. Don't take my word for it. Clone it and watch your own numbers. Inspired by Mia Apache 2.0.

13 Comments

Comments from the original post on X (Twitter)

  • Landon's profile picture

    @landontgreen

    TensorFold has me so excited for the future. It's such a big improvement and it's only the beginning!

    9 likes

  • Arthur Lee's profile picture

    @arthurlee

    What does this for M5?

    2 likes

  • Volatile Markets's profile picture

    @volatilemarkts

    Hopeful to have everyone numbers soon.

    1 like

  • Ryan's profile picture

    @HumanHitsSend

    How much VRAM you got on that bird?

    2 likes

  • ajodorowski's profile picture

    @jodorowsk1

    Oh, is the model quantized to 4 bits?

    1 like

  • Niklas's profile picture

    @niklaslenz_ai

    How is this even possible?! One dude is literally changing local AI in realtime.

    1 like

  • AI Quanting's profile picture

    @AIQuanting

    The 0.08 s first token is the cache doing the work. Cold runs are the number I'd want to see.

  • aṅkush's profile picture

    @ankushnarula

    How well does it perform as context window scales up?

  • sparkarena's profile picture

    @spark_arena

    Have you tried initial context above 32k and 2 or more concurrent requests?

  • Andrey Kolesnikov's profile picture

    @minviable_org

    which weights ?

  • Less's profile picture

    @lessaxwrong

    Wow

  • Cowboy Coder's profile picture

    @CowboyCoderHQ

    I love tensorfold as much as the next dev, but don’t you think this comparison is a tad unfair. Why not show omlx with mtp and apc on vs tensorfold for a fair engine to engine comparison.

  • Amir ··· 🇨🇭's profile picture

    @AmirFromZH

    Same quantization?