Skip navigation
Got continuous batching working with SSMs in mlx-lm.
0:00 / 0:18

Got continuous batching working with SSMs in mlx-lm.

Awni Hannun

@awnihannun45K followers

Visit on X

35,231 views

Got continuous batching working with SSMs in mlx-lm. Here's four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).

Comments

Comments from the original post on X (Twitter)

No comments yet