presenting: n8loom, a library for generating trees-of-thought - the kind used in MCTS, variants of majority-voting, etc. - efficiently by splitting the kvcache into fragments at each node, and dynamically concatenating the results together when generating.

(and a kickass experimental GUI for creating this trees). the library lets you create, in-code, nodes that represent text processed or generated by the LLM, and then extended and 'grow' the loom with either preset text of more dynamic generation.

this lets you prefill assistant responses, inject entropix-style 'Wait', or generate a variety of branching pths and selecting the path that finds the correct answer to train on.

Find the library here:

Built on #mlx so apple silicon only atm - though the fundamental impl should be easy to port. works out-of-the box with any mlx model.

a known issue is how i could benchmark the tangible speedup here - theoretically, it should just be the lack of having to re-evaluate prompts, so the speedup should be obvious in theory. but what's theory doesn't always work out in practice - and i'd want to design a fair experiment that isn't too harsh to traditional generation - where you can prefix-cache SOME of the prompt fairly easily when generating...

's Avani Prabhakar reveals how teams can improve collaboration and productivity by adopting asynchronous communication. #Teamwork #ad

most cracked man at Nous fr fr

niiiiice very cool

Absolutely cracked! Will deff be testing this out 💯

This looks incredible

Fuckin love it