Navigation überspringen
Introducing SubQ - a major breakthrough in LLM intelligence.
0:00 / 1:24

Introducing SubQ - a major breakthrough in LLM intelligence.

Alexander Whedon

@alex_whedon26.177 Follower

Auf X besuchen
22.364 LikesHerunterladenOriginalbeitrag

13.499.800 Aufrufe

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

70 Kommentare

Kommentare aus dem Originalbeitrag auf X (Twitter)

  • Profilbild von Alexander Whedon

    @alex_whedon

    SubQ is available for early access today, alongside our coding agent, SubQ Code Get access today ↓

    1.798 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    We were a little slow on this, but we just got a technical blog post up with more details. Please take a look! We have a model card coming next week, and we are happy to take requests for any specific details there. I am happy to answer any questions here!

    680 Likes

  • Profilbild von Martin Shkreli

    @MartinShkreli

    congrats!! why only 150tok/sec if 52x faster, etc.?

    361 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    52x is for prefill speed. Decoding speedups are coming too! This is the first research announcement, with many to follow.

    426 Likes

  • Profilbild von Chris B. Ward (e/bored) 🇬🇧🇮🇪🤐

    @chrisbward

    @MartinShkreli absolute grift, didn't deliver

    1 Like

  • Profilbild von Alexander Whedon

    @alex_whedon

    @MartinShkreli Model card shared!

  • Profilbild von Vincent

    @vincexplorer

    Any papers? Seems too good to be true

    318 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    The model card is coming next week! We are releasing a technical blog post with more details later today.

    414 Likes

  • Profilbild von Vincent

    @vincexplorer

    Looking forward to reading it

    40 Likes

  • Profilbild von shirish

    @shiri_shh

    i still can't believe this.. 12M context window with 98% accuracy 💀 @alex_whedon @subquadratic

    301 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    @subquadratic And at a fraction of the cost...

    218 Likes

  • Profilbild von • nanou •

    @NanouuSymeon

    If this works as described, it basically changes how we build with LLMs. A lot of current pipelines exist just to work around context limits.

    142 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    And now that constraint goes away, what people choose to build is going to be very different.

    98 Likes

  • Profilbild von Linus ✦ Ekenstam

    @LinusEkenstam

    I remember when Gemini first made the claim of 1M in lab situations. We've come quite a bit from there. I'm extremely intrigued to see the real-world implications of this in bigger and more complex AI powered software. There is the zero shot environment, but there is also the qualitative repetitive work that is prone to drift and degradation over time. I really hope you will share case-studies, since I think this will be one of the most impressive ways to highlight the capabilities. Amazing work Alex and team. Kudos.

    116 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    This is the framing we'd love to test against. If you've got a workload that historically drifts, would genuinely want you to throw it at SubQ and see what happens.

    72 Likes

  • Profilbild von LADI ⟡⭐

    @Den_zel21

    If SubQ only attends to a small subset of tokens, how does it guarantee recall of critical long range dependencies in worst case inputs (e.g adversarial prompts or tasks where the important tokens aren’t locally obvious)

    108 Likes

  • Profilbild von Alexander Whedon

    @alex_whedon

    We dynamically select the token relationships that matter and train it against retrieval problems with distractor docs, etc.

    108 Likes

  • Profilbild von LADI ⟡⭐

    @Den_zel21

    What happens when the selector misses the one token that matters? Any fallback

    40 Likes

  • Profilbild von LADI ⟡⭐

    @Den_zel21

    I agree, it’s about frequency, not perfection.

    5 Likes

  • Profilbild von Veselin Stoyanov

    @st0yanov

    A few weeks ago I was in a discussion under an X post about LLMs, their adoption and cost. Most people argued it doesn't scale and brought up the usual economic concerns. My take was that it's only a matter of time before the next breakthrough. It's always been this way with tech. Congrats on this one 👏

    93 Likes