Steeve Morin's banner
Steeve Morin's profile picture

Steeve Morin

@steeve • 9,181 subscribers

Building @zml_ai (and we're hiring), ex @zenly, ex Exalead, ex @google. Skydiver and wingsuiter.

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.
4:33
steeve's profile picture

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

Steeve Morin

122,107 Aufrufe • vor 11 Monaten

another 5 days later, zml/llmd runs fully on Metal, serving 8 simultaneous requests at full bf16 zml/llmd is our LLM serving software that all the modern niceties (continuous batching etc..) ps: "full bf16", what a time to be alive
0:27
steeve's profile picture

another 5 days later, zml/llmd runs fully on Metal, serving 8 simultaneous requests at full bf16 zml/llmd is our LLM serving software that all the modern niceties (continuous batching etc..) ps: "full bf16", what a time to be alive

Steeve Morin

27,508 Aufrufe • vor 2 Monaten

And after 1 week of work, here is zml/llmd running transparently on TPU with full prefill/decode paged attention. No code change, single flag, as it should be.
0:28
steeve's profile picture

And after 1 week of work, here is zml/llmd running transparently on TPU with full prefill/decode paged attention. No code change, single flag, as it should be.

Steeve Morin

26,073 Aufrufe • vor 1 Jahr

Keine weiteren Inhalte verfügbar