We’re open-sourcing lithos-metal 🚀 Megakernels + DSpark speculative decoding run Qwen3.8-27B at 200+ tokens/s/user peak on one Apple M5 Max. Ultra-fast inference on your laptop. Try it with any coding agent in one command. Code: Tech blog:

@Apple

@sh_reya @Apple nice peak numbers, but apple silicon still runs into the memory bandwidth wall pretty quick once the kv cache grows. curious how much of that 200 tok/s holds up in a real multi-turn agent loop with tool calls instead of pure generation

@sh_reya @Apple Great question! We observe 120-130 TPS when context length grows to 32K-64K. The fundamental challenge is the dense attention layers in Qwen3.8-27B.

@Apple Your projected 127 a second at 32K is the number I'd lead with. That's usually where laptop setups fall apart.

@Apple @ivanfioravanti what is this sorcellerie, how is it possible

@Apple 🍎🚀

@Apple 200+ tokens/s on a laptop is wild. as an engineer, what catches my eye is DSpark speculative decoding — that's where the real magic is, turning idle GPU cycles into throughput. local 27B models at this speed change what you can prototype offline.

@Apple Open-sourcing this for local inference on Apple silicon is a solid move. Curious to try it with a coding agent.

@Apple Whats the TTFT for 256 context vs 32k context? Don't see it mentioned anywhere

@Apple Incredible results! Gonna test it immediately!

@Apple reading more about your repo now and how good is drafter. i am trying similar here: though here reaches upto 180tps for shorter prompt. megakernel approach didn't try because smaller fused kernels gave good speed for me.

@Apple When M1 support, sirs?!?

@old_sound @Apple Really incredible, thank you for your service!

@Apple 200+ tokens per second on a laptop is seriously impressive inference performance

@Apple Curious looks promising

@Apple So you are offering the community in Apache 2 the engine that can infer Kimi K3 up to 1000tok/s. Thank you 😭

@Apple @grok is this faster than Tensorfold?