Steeve Morin's banner

Steeve Morin

@steeve • 9,181 subscribers

Building @zml_ai (and we're hiring), ex @zenly, ex Exalead, ex @google. Skydiver and wingsuiter.

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

122,107 görüntüleme • 11 ay önce

another 5 days later, zml/llmd runs fully on Metal, serving 8 simultaneous requests at full bf16 zml/llmd is our LLM serving software that all the modern niceties (continuous batching etc..) ps: "full bf16", what a time to be alive

another 5 days later, zml/llmd runs fully on Metal, serving 8 simultaneous requests at full bf16 zml/llmd is our LLM serving software that all the modern niceties (continuous batching etc..) ps: "full bf16", what a time to be alive

27,540 görüntüleme • 2 ay önce

And after 1 week of work, here is zml/llmd running transparently on TPU with full prefill/decode paged attention. No code change, single flag, as it should be.

And after 1 week of work, here is zml/llmd running transparently on TPU with full prefill/decode paged attention. No code change, single flag, as it should be.

26,073 görüntüleme • 1 yıl önce

Daha fazla içerik yok.