New Liquid research: STAR -- Evolutionary Synthesis of Tailored Architectures. At Liquid we design foundation models with two macro-objectives: maximize quality and efficiency. Balancing the two is challenging. To make progress towards this goal, we built a new algorithm — STAR. Read more about it here:

We first developed a new design theory for computational units of modern AI systems. We then used it to devise an efficient encoding into architecture genomes, and applied evolutionary algorithms to discover hundreds of new architecture designs.

One of the core innovations of STAR is encoding model architectures as hierarchical numeric sequences called STAR genomes, which we evolve using principles from evolutionary optimization. We compile genomes into architectures, evaluate them, and then select and recombine them.

Beyond optimizing architectures for specific objectives, STAR also provides an analysis tool to identify recurring architecture motifs emerging during evolution, driving the observed improvements

STAR’s capabilities have far-reaching implications: Thanks to the ability to optimize any mixture of metrics, combined with the versatility of our design space, we're witnessing continuous improvements in both the diversity and quality of synthesized designs.

The design space for model architectures is vast. With STAR we can optimize large populations of new architectures at scale for obtaining the highest-performing models that satisfy given computational requirements. Read the full paper here:

At Liquid, we build hardware-aware, best-in-class, and efficient AI systems at every scale, with a design theory for models grounded in dynamical systems, signal processing, and numerical linear algebra. If this resonates with you consider joining us:

@threadreaderapp unroll

Your thread is very popular today! #TopUnroll 🙏🏼@ewwfedcoin for 🥇unroll

Very cool

cc: @TonyZador @hardmaru