Loading video...

Video Failed to Load

Go Home

Atomic Chat has launched DFlash for macOS, Windows, and Linux. A new speculative decoding mode that runs local Qwen models 2.2x faster on llama.cpp, with byte-for-byte identical output. A separate small model can draft up to 15 tokens at once, and the full model only verifies them, making sure...

testingcatalog's profile picture

🚨 AI News | TestingCatalog

70,084 subscribers

20,346 views • 1 day ago •via X (Twitter)

Science & TechnologyNews & Politics
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

State of the new STABLE FUCKING RELEASE OF MacOS that my laptop is absolutely amazing ... maybe DHH has a point with all the Linux shit (in the background you can see my gaming computer that runs Linux and Windows just fine....)
0:40

State of the new STABLE FUCKING RELEASE OF MacOS that my laptop is absolutely amazing ... maybe DHH has a point with all the Linux shit (in the background you can see my gaming computer that runs Linux and Windows just fine....)

Fran Zekan

1,945,038 views • 9 months ago

Fiverr just launched a platform for freelancers to train a custom AI model, and "sell" the ability to generate work with it. They're pitching it as a way for creators to scale the number of projects they can handle at once. Models are trainable on text, image, and sound.
0:24

Fiverr just launched a platform for freelancers to train a custom AI model, and "sell" the ability to generate work with it. They're pitching it as a way for creators to scale the number of projects they can handle at once. Models are trainable on text, image, and sound.

Olivia Moore

117,853 views • 1 year ago

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.
1:03

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.

Qwen

21,333 views • 3 months ago

Today we're excited to introduce Multi-Model Sessions! — Load & prompt multiple local LLMs simultaneously — Use the API to build networks, pipelines of LLMs that can talk with each other — Use the new JSON Mode to constrain each model's output format LM Studio 0.2.17 is available for Mac, Windows, and Linux. Get it here:
0:21

Today we're excited to introduce Multi-Model Sessions! — Load & prompt multiple local LLMs simultaneously — Use the API to build networks, pipelines of LLMs that can talk with each other — Use the new JSON Mode to constrain each model's output format LM Studio 0.2.17 is available for Mac, Windows, and Linux. Get it here:

LM Studio

89,817 views • 2 years ago

QVAC SDK 0.13.0 is live, and this version brings a lot of exciting updates! Local AI now plugs into your coding agent, ships as a desktop app in one command, and runs even more models. Highlights: NEW INTEGRATIONS - OpenCode and coding agents: the new @qvac/ai-sdk-provider makes QVAC a local provider. Less setup, same-model requests queue cleanly, and managed mode starts and supervises qvac serve for you. - Broader OpenAI-compatible API, validated across supported flows so covered capabilities stay consistent and testable. - Turn your QVAC project into a real desktop app for Mac, Windows, or Linux with a single command. The new Electron plugin handles the packaging and keeps the app small by including only what it needs. NEW MODELS - New pi0.5 model support - run a vision-language "robot brain" on a single ordinary graphics card, at full accuracy. - Image-to-video, fully local, via the Wan2.1 model in the Diffusion addon. - New BCI add-on: brain-computer interface transcription, fully local. Decode recorded neural signals into text on-device via the Whisper.cpp-based BCI model. IMPROVEMENTS - Whisper GPU transcription on Android, auto-picking the best backend (OpenCL on Adreno 700+, Vulkan elsewhere), unified on one ggml engine. - Parakeet steadier on mobile, with real end-of-utterance detection for streaming. - Supertonic TTS now runs full GPU across Metal, Vulkan, and OpenCL, with native streaming.
1:39

QVAC SDK 0.13.0 is live, and this version brings a lot of exciting updates! Local AI now plugs into your coding agent, ships as a desktop app in one command, and runs even more models. Highlights: NEW INTEGRATIONS - OpenCode and coding agents: the new @qvac/ai-sdk-provider makes QVAC a local provider. Less setup, same-model requests queue cleanly, and managed mode starts and supervises qvac serve for you. - Broader OpenAI-compatible API, validated across supported flows so covered capabilities stay consistent and testable. - Turn your QVAC project into a real desktop app for Mac, Windows, or Linux with a single command. The new Electron plugin handles the packaging and keeps the app small by including only what it needs. NEW MODELS - New pi0.5 model support - run a vision-language "robot brain" on a single ordinary graphics card, at full accuracy. - Image-to-video, fully local, via the Wan2.1 model in the Diffusion addon. - New BCI add-on: brain-computer interface transcription, fully local. Decode recorded neural signals into text on-device via the Whisper.cpp-based BCI model. IMPROVEMENTS - Whisper GPU transcription on Android, auto-picking the best backend (OpenCL on Adreno 700+, Vulkan elsewhere), unified on one ggml engine. - Parakeet steadier on mobile, with real end-of-utterance detection for streaming. - Supertonic TTS now runs full GPU across Metal, Vulkan, and OpenCL, with native streaming.

QVAC

20,922,918 views • 29 days ago

Introducing Oasis: the first playable AI-generated game. We partnered with Decart to build a real-time, interactive world model that runs >10x faster on Sohu. We're open-sourcing the model architecture, weights, and research. Here's how it works (and a demo you can play!):
0:24

Introducing Oasis: the first playable AI-generated game. We partnered with Decart to build a real-time, interactive world model that runs >10x faster on Sohu. We're open-sourcing the model architecture, weights, and research. Here's how it works (and a demo you can play!):

Etched

24,675,513 views • 1 year ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

Geoffrey Hinton warns about the dangers of releasing AI Model weights releasing the weights of large AI models is dangerous, similar to making fissile material available for making bombs. once these models are released, bad actors can use them for harmful purposes but it's too late now and the cat is out of the bag.
0:55

Geoffrey Hinton warns about the dangers of releasing AI Model weights releasing the weights of large AI models is dangerous, similar to making fissile material available for making bombs. once these models are released, bad actors can use them for harmful purposes but it's too late now and the cat is out of the bag.

Haider.

23,588 views • 1 year ago

Jan v1 is now more reliable when handling searches. It's an open-source alternative to Perplexity Pro, and it now avoids infinite loops with Jan v1 2509. Performance updates: - Gains on reasoning & creativity benchmarks - Small drop on SimpleQA (91.1 -> 90.7), still competitive with Perplexity Pro The updated version is live on Jan Hub. You can run the model in Jan, llama.cpp, or vLLM. Use the model: - Model: - GGUF: Shoutout to the Qwen team for Qwen3-4B Thinking and Georgi Gerganov for llama.cpp.
0:36

Jan v1 is now more reliable when handling searches. It's an open-source alternative to Perplexity Pro, and it now avoids infinite loops with Jan v1 2509. Performance updates: - Gains on reasoning & creativity benchmarks - Small drop on SimpleQA (91.1 -> 90.7), still competitive with Perplexity Pro The updated version is live on Jan Hub. You can run the model in Jan, llama.cpp, or vLLM. Use the model: - Model: - GGUF: Shoutout to the Qwen team for Qwen3-4B Thinking and Georgi Gerganov for llama.cpp.

👋 Jan

17,759 views • 10 months ago

🚀 Introducing Spaghetti Models For Free A powerful way to track possible tropical storm paths and compare forecast scenarios. With this update, you can now: - View the latest and historical model runs - Filter models to focus on the guidance you trust - See and download intensity graphs to track trends over time Stay ahead of the storm with clearer insights than ever before. Check out the latest model run for Tropical Storm Fernand here: Available now for free on: iOS: Android: Browser: And to Plus/Pro subscribers using our Windows/Mac/Linux App
0:29

🚀 Introducing Spaghetti Models For Free A powerful way to track possible tropical storm paths and compare forecast scenarios. With this update, you can now: - View the latest and historical model runs - Filter models to focus on the guidance you trust - See and download intensity graphs to track trends over time Stay ahead of the storm with clearer insights than ever before. Check out the latest model run for Tropical Storm Fernand here: Available now for free on: iOS: Android: Browser: And to Plus/Pro subscribers using our Windows/Mac/Linux App

WeatherWise.app

36,159 views • 10 months ago

Alibaba just released a coding model that hits 82 percent on SWE-Bench Verified. That is the highest score ever published for an open-source model. The weights are free. The license is Apache 2.0. You can run it today. The model is Qwen 4 Coder 32B. Here is what 82 percent on SWE-Bench Verified actually means. SWE-Bench Verified tests whether an AI can autonomously resolve real bugs pulled from real production GitHub repositories. Not synthetic exercises. Real open-source projects that real teams depend on. A model gets a bug report, reads the code, writes a fix, and either passes the test suite or it does not. At 82 percent, Qwen 4 Coder 32B resolves 82 out of every 100 real production bugs it is given. Without a human guiding it. On code it has never seen before. For comparison: Qwen 4 Coder 32B: 82 percent SWE-Bench Verified. Open source. Apache 2.0. Claude Fable 5: 80.3 percent SWE-Bench Pro. $10 input / $50 output per million tokens. Currently suspended. GPT-5.6 Sol: Competitive on Terminal-Bench. $5 input / $30 output per million tokens. An open-weight model that you can download and run for free just beat both of them on the benchmark designed to measure real software engineering capability. Here is the architecture. Qwen 4 Coder 32B is a 32 billion parameter dense model. Not a Mixture-of-Experts. Every parameter is active on every request. This matters for inference: a dense 32B model runs on 22 gigabytes of VRAM, which fits on a single high-end consumer GPU or a MacBook Pro with 64GB of unified memory. The smaller variant, Qwen 4 Coder 4B, runs at approximately 135 tokens per second on an M5 Max and fits inside 8 gigabytes of RAM. For a model with usable coding capability, that is a new bar for what fits in a single laptop. The training methodology continued Alibaba's approach of reinforcement learning on verifiable coding tasks. The model gets rewarded when its code passes tests. It gets penalized when it fails. Over millions of training steps, the model learns to write code that actually runs rather than code that looks plausible. License: Apache 2.0. Full commercial use. No attribution requirement. No revenue threshold. No monthly active user ceiling. Weights: Hugging Face, available today. Runs on: vLLM, Ollama, SGLang, and any standard GGUF-compatible inference engine. Qwen 4 32B also runs at approximately 135 tokens per second on an M5 Max chip, setting a new bar for what a sub-8GB model can do on Apple Silicon. The open-source coding model just beat the best closed-source model in the world on the benchmark designed to test whether AI can actually do software engineering. The weights are free. The subscription is optional. Source: Autom8Labs AI Insight July 2026, State of Open Source LLMs June 2026, Kunal Ganglani blog June 2026.
0:25

Alibaba just released a coding model that hits 82 percent on SWE-Bench Verified. That is the highest score ever published for an open-source model. The weights are free. The license is Apache 2.0. You can run it today. The model is Qwen 4 Coder 32B. Here is what 82 percent on SWE-Bench Verified actually means. SWE-Bench Verified tests whether an AI can autonomously resolve real bugs pulled from real production GitHub repositories. Not synthetic exercises. Real open-source projects that real teams depend on. A model gets a bug report, reads the code, writes a fix, and either passes the test suite or it does not. At 82 percent, Qwen 4 Coder 32B resolves 82 out of every 100 real production bugs it is given. Without a human guiding it. On code it has never seen before. For comparison: Qwen 4 Coder 32B: 82 percent SWE-Bench Verified. Open source. Apache 2.0. Claude Fable 5: 80.3 percent SWE-Bench Pro. $10 input / $50 output per million tokens. Currently suspended. GPT-5.6 Sol: Competitive on Terminal-Bench. $5 input / $30 output per million tokens. An open-weight model that you can download and run for free just beat both of them on the benchmark designed to measure real software engineering capability. Here is the architecture. Qwen 4 Coder 32B is a 32 billion parameter dense model. Not a Mixture-of-Experts. Every parameter is active on every request. This matters for inference: a dense 32B model runs on 22 gigabytes of VRAM, which fits on a single high-end consumer GPU or a MacBook Pro with 64GB of unified memory. The smaller variant, Qwen 4 Coder 4B, runs at approximately 135 tokens per second on an M5 Max and fits inside 8 gigabytes of RAM. For a model with usable coding capability, that is a new bar for what fits in a single laptop. The training methodology continued Alibaba's approach of reinforcement learning on verifiable coding tasks. The model gets rewarded when its code passes tests. It gets penalized when it fails. Over millions of training steps, the model learns to write code that actually runs rather than code that looks plausible. License: Apache 2.0. Full commercial use. No attribution requirement. No revenue threshold. No monthly active user ceiling. Weights: Hugging Face, available today. Runs on: vLLM, Ollama, SGLang, and any standard GGUF-compatible inference engine. Qwen 4 32B also runs at approximately 135 tokens per second on an M5 Max chip, setting a new bar for what a sub-8GB model can do on Apple Silicon. The open-source coding model just beat the best closed-source model in the world on the benchmark designed to test whether AI can actually do software engineering. The weights are free. The subscription is optional. Source: Autom8Labs AI Insight July 2026, State of Open Source LLMs June 2026, Kunal Ganglani blog June 2026.

Harman

38,953 views • 6 days ago

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.
0:25

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,554,326 views • 4 months ago

QVAC SDK 0.12.0 is now live, bringing longer context, increased memory optimisation, new modalities, and broader ecosystem support directly to your device. Key Features and Updates: - TurboQuant KV-Cache Quantization: Fit much longer context in the same memory. TurboQuant, an algorithm from Google Research, compresses the KV cache by up to 5x, near-lossless. - Text-to-Video: Generate video from a text prompt, fully local, with the new wan2.1 model in the Diffusion addon - Apple Metal Performance for Flux2-klein: Diffusion on Apple Silicon now matches MLX performance, the native benchmark for Apple GPUs - Robot Control (new VLA addon): A GGML-based Vision-Language-Action addon brings fast, efficient robot control to edge devices - Coding Assistant / Harness Support: QVAC now works with OpenCode and OpenClaw as a local provider. A new @qvac/ai-sdk-provider package automates model registry and provider integration - Cross-Platform Voice: Text-to-speech and Parakeet transcription moved from ONNX to the GGML engine for better CPU and GPU support on macOS, iOS, Windows, Linux, and Android. Parakeet also adds long-term streaming diarization (tracking who spoke when on live audio) - Faster Lightweight Visual Classification: A new GGML-based Classification addon delivers millisecond-level classification, useful where a vision-language model (VLM) would be unnecessarily slow - Under the Hood: Fabric synced to llama.cpp v8828 (from v8189), plus GPU acceleration added to image-upscale models for faster results Full release notes:
1:29

QVAC SDK 0.12.0 is now live, bringing longer context, increased memory optimisation, new modalities, and broader ecosystem support directly to your device. Key Features and Updates: - TurboQuant KV-Cache Quantization: Fit much longer context in the same memory. TurboQuant, an algorithm from Google Research, compresses the KV cache by up to 5x, near-lossless. - Text-to-Video: Generate video from a text prompt, fully local, with the new wan2.1 model in the Diffusion addon - Apple Metal Performance for Flux2-klein: Diffusion on Apple Silicon now matches MLX performance, the native benchmark for Apple GPUs - Robot Control (new VLA addon): A GGML-based Vision-Language-Action addon brings fast, efficient robot control to edge devices - Coding Assistant / Harness Support: QVAC now works with OpenCode and OpenClaw as a local provider. A new @qvac/ai-sdk-provider package automates model registry and provider integration - Cross-Platform Voice: Text-to-speech and Parakeet transcription moved from ONNX to the GGML engine for better CPU and GPU support on macOS, iOS, Windows, Linux, and Android. Parakeet also adds long-term streaming diarization (tracking who spoke when on live audio) - Faster Lightweight Visual Classification: A new GGML-based Classification addon delivers millisecond-level classification, useful where a vision-language model (VLM) would be unnecessarily slow - Under the Hood: Fabric synced to llama.cpp v8828 (from v8189), plus GPU acceleration added to image-upscale models for faster results Full release notes:

QVAC

9,932,369 views • 1 month ago

I left my job at NVIDIA last year... and now I co-founded @outerbasis and started a position as ceo sararīman!! (also we're part of the current Y Combinator S24 batch 😎) With new tech come new formats, and with new formats come new distribution tech. We've seen this with cameras ➡️ JPEGs ➡️ social media and cgroups ➡️ containers ➡️ orchestration. Just like how cameras capture a slice of the photon stream around us, generative AI is a 'data camera' that allows us to snapshot the society that we live in. The new format from this capture device are model weights, which require tailored distribution solutions for them. @outerbasis is a registry for model weights that lets you store model weights and orchestrate the distribution of them onto GPU servers. The system uses tech optimized for streaming weights into the servers as fast as possible- so researchers can deploy updates without downtime and ops admins can orchestrate hot swapping between models. I've been super excited about streaming, compression, and neural networks for the last N years... but now I'm excited to build a system to support of this evermore exciting ecosystem of foundation model weights!! Please reach out if this sounds interesting / useful to you!! (and you can find more details on our YC launch: (The video is a small demo of our model hotswapping)
0:51

I left my job at NVIDIA last year... and now I co-founded @outerbasis and started a position as ceo sararīman!! (also we're part of the current Y Combinator S24 batch 😎) With new tech come new formats, and with new formats come new distribution tech. We've seen this with cameras ➡️ JPEGs ➡️ social media and cgroups ➡️ containers ➡️ orchestration. Just like how cameras capture a slice of the photon stream around us, generative AI is a 'data camera' that allows us to snapshot the society that we live in. The new format from this capture device are model weights, which require tailored distribution solutions for them. @outerbasis is a registry for model weights that lets you store model weights and orchestrate the distribution of them onto GPU servers. The system uses tech optimized for streaming weights into the servers as fast as possible- so researchers can deploy updates without downtime and ops admins can orchestrate hot swapping between models. I've been super excited about streaming, compression, and neural networks for the last N years... but now I'm excited to build a system to support of this evermore exciting ecosystem of foundation model weights!! Please reach out if this sounds interesting / useful to you!! (and you can find more details on our YC launch: (The video is a small demo of our model hotswapping)

Towaki Takikawa / 瀧川永遠希

51,277 views • 1 year ago

How to have your Openclaw install the new Qwen 3.5-9b Local LLM in 1 minute This came out TODAY and is one of the only local models you can run on a 16GB Mac Mini Will be testing and making some videos this week, look out for that (check below for some tests for now)
1:04

How to have your Openclaw install the new Qwen 3.5-9b Local LLM in 1 minute This came out TODAY and is one of the only local models you can run on a 16GB Mac Mini Will be testing and making some videos this week, look out for that (check below for some tests for now)

ashen

63,977 views • 4 months ago

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.
1:46

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

Sudo su

27,339 views • 1 month ago

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.
1:36

I tested Gemma 4 12b on my Macbook Pro M5 Max with 128GB of unified memory. It runs VERY slow for a 12b parameter model. 44 tokens/second. The lava lamp that it produced was also terrible. I would stick to Qwen 3.6 27b or Qwen 3.6 35b. Google is still way behind, even with their local models.

BridgeMind

70,325 views • 1 month ago

OpenAI just announced "GPT-4o". It can reason with voice, vision, and text. The model is 2x faster, 50% cheaper, and has 5x higher rate limit than GPT-4 Turbo. It will be available for free users and via the API. The voice model can even pick up on emotion and generate emotive voice.
1:45

OpenAI just announced "GPT-4o". It can reason with voice, vision, and text. The model is 2x faster, 50% cheaper, and has 5x higher rate limit than GPT-4 Turbo. It will be available for free users and via the API. The voice model can even pick up on emotion and generate emotive voice.

Lior Alexander

485,074 views • 2 years ago

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.
1:24

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,186,398 views • 2 months ago