Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

DeepSeek-V4 dropped. 1M context. 10x smaller KV cache. First open model where the context window and the agentic post-training meet.

Ben Burtenshaw

7,563 subscribers

49,900 views • 1 month ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵

Qwen just quietly dropped the most dangerous open model of 2026 Qwen3.6-Plus. native 1M context. agentic workflows. multimodal. half the size of competitors. and it's free on OpenRouter right now. you're not ready for this breakdown 🧵

Farhan

53,820 views • 2 months ago

Qwen 3.6 Plus Preview just dropped on OpenRouter for Free. > 1M token context. > $0 input. $0 output. Meanwhile: - Claude Opus 4.6 → $5 / $25 per 1M - GPT-5.4 → charges for 1M context Qwen just nuked the pricing model. Open source isn’t slowing down. It’s taking over.

Qwen 3.6 Plus Preview just dropped on OpenRouter for Free. > 1M token context. > $0 input. $0 output. Meanwhile: - Claude Opus 4.6 → $5 / $25 per 1M - GPT-5.4 → charges for 1M context Qwen just nuked the pricing model. Open source isn’t slowing down. It’s taking over.

WorldofAI

146,277 views • 2 months ago

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks AI inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks AI inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

elvis

57,820 views • 1 month ago

OpenAI recently released its first open-weights model since GPT-2, entering a field led by DeepSeek and Alibaba's Qwen. Ankit () breaks down these top OSS models, including what sets them apart under the hood: mixture-of-experts, long-context training, and post-training techniques that shape reasoning and alignment—and how different design choices lead to surprisingly similar performance. 00:00 – OpenAI OSS Launch 01:00 – Comparing Open Source LLM Architectures 01:46 – GPT OSS Overview 02:37 – Under The Hood of GPT OSS 03:25 – Qwen-3 Architecture 04:17 – Qwen-3 Training 05:12 – Qwen-3 Post-Training 06:08 – Qwen-3 Reasoning & RL Innovations 06:52 – DeepSeek V3 Overview 07:40 – DeepSeek V3.1 Updates 08:39 – Attention Mechanism (MLA) 09:39 – Comparing Model Sizes 10:35 – Long Context Strategies 11:25 – Reflections on Methods 12:00 – Takeaways

OpenAI recently released its first open-weights model since GPT-2, entering a field led by DeepSeek and Alibaba's Qwen. Ankit () breaks down these top OSS models, including what sets them apart under the hood: mixture-of-experts, long-context training, and post-training techniques that shape reasoning and alignment—and how different design choices lead to surprisingly similar performance. 00:00 – OpenAI OSS Launch 01:00 – Comparing Open Source LLM Architectures 01:46 – GPT OSS Overview 02:37 – Under The Hood of GPT OSS 03:25 – Qwen-3 Architecture 04:17 – Qwen-3 Training 05:12 – Qwen-3 Post-Training 06:08 – Qwen-3 Reasoning & RL Innovations 06:52 – DeepSeek V3 Overview 07:40 – DeepSeek V3.1 Updates 08:39 – Attention Mechanism (MLA) 09:39 – Comparing Model Sizes 10:35 – Long Context Strategies 11:25 – Reflections on Methods 12:00 – Takeaways

Y Combinator

208,680 views • 9 months ago

MIT Researchers destroyed context window limits. 10m+ token prompts are now possible by moving context out of the model and into code environments. Full breakdown below.

MIT Researchers destroyed context window limits. 10m+ token prompts are now possible by moving context out of the model and into code environments. Full breakdown below.

Matthew Berman

287,375 views • 5 months ago

🎉 Congrats to MiniMax (official) on releasing MiniMax M3! Frontier coding and agentic capabilities, native image and video input, computer use, and a 1M-token context window, all in a single open model. At the heart of M3 is MSA, a new sparse attention architecture: instead of attending densely over the full KV cache, each query scores 128-token KV blocks and runs attention only over the top blocks. That is what makes 1M-token context practical to serve. M3 runs in vLLM with day-0 support, verified on NVIDIA and AMD hardware: ✨ MSA sparse attention with dedicated prefill and decode kernels ✨ 1M-token context serving with prefix caching and chunked prefill ✨ BF16 and MXFP8 checkpoints, with MoE backends for both Hopper and Blackwell ✨ Native multimodal input (image + video) ✨ Tool calling, reasoning parsing, and thinking-mode control for agent workloads Day-0 support like this is a true team effort. Grateful to the teams at MiniMax (official), NVIDIA AI, AI at AMD, and Inferact, and to the vLLM community for making it happen. 🙏 Deep dive into the implementation, kernel work, and deployment recipes: 🔗

🎉 Congrats to MiniMax (official) on releasing MiniMax M3! Frontier coding and agentic capabilities, native image and video input, computer use, and a 1M-token context window, all in a single open model. At the heart of M3 is MSA, a new sparse attention architecture: instead of attending densely over the full KV cache, each query scores 128-token KV blocks and runs attention only over the top blocks. That is what makes 1M-token context practical to serve. M3 runs in vLLM with day-0 support, verified on NVIDIA and AMD hardware: ✨ MSA sparse attention with dedicated prefill and decode kernels ✨ 1M-token context serving with prefix caching and chunked prefill ✨ BF16 and MXFP8 checkpoints, with MoE backends for both Hopper and Blackwell ✨ Native multimodal input (image + video) ✨ Tool calling, reasoning parsing, and thinking-mode control for agent workloads Day-0 support like this is a true team effort. Grateful to the teams at MiniMax (official), NVIDIA AI, AI at AMD, and Inferact, and to the vLLM community for making it happen. 🙏 Deep dive into the implementation, kernel work, and deployment recipes: 🔗

vLLM

39,402 views • 5 days ago

Qwen just published the 'Thinking' variant of this model 🔥 So you can run a model EVEN MORE powerful than GPT-4o locally! - Still only 3B active parameters - Open source license - 256k context window extendable to 1M - Strong in math, science and coding Details below ↓

Qwen just published the 'Thinking' variant of this model 🔥 So you can run a model EVEN MORE powerful than GPT-4o locally! - Still only 3B active parameters - Open source license - 256k context window extendable to 1M - Strong in math, science and coding Details below ↓

Paul Couvert

106,261 views • 10 months ago

New Tab model, 1M+ context windows, and a preview of our background agent

New Tab model, 1M+ context windows, and a preview of our background agent

Cursor

366,238 views • 1 year ago

This is Claude Sonnet 4.6: our most capable Sonnet model yet. It’s a full upgrade across coding, computer use, long-context reasoning, agent planning, knowledge work, and design. It also features a 1M token context window in beta.

This is Claude Sonnet 4.6: our most capable Sonnet model yet. It’s a full upgrade across coding, computer use, long-context reasoning, agent planning, knowledge work, and design. It also features a 1M token context window in beta.

Claude

7,588,850 views • 4 months ago

China's Qafind Labs just launched the first Diffusion Language Model (DLM). 🔥 ChatDLM, described as the first Diffusion Language Model (DLM), will be open-sourced soon. - Inference Speed: 2,800 tokens/sec (on A100), which is insanely fast. - Context Window: 131,072 tokens. More details 👇

China's Qafind Labs just launched the first Diffusion Language Model (DLM). 🔥 ChatDLM, described as the first Diffusion Language Model (DLM), will be open-sourced soon. - Inference Speed: 2,800 tokens/sec (on A100), which is insanely fast. - Context Window: 131,072 tokens. More details 👇

AshutoshShrivastava

73,296 views • 1 year ago

Introducing Claude Opus 4.6. Our smartest model got an upgrade. Opus 4.6 plans more carefully, sustains agentic tasks for longer, operates reliably in massive codebases, and catches its own mistakes. It’s also our first Opus-class model with 1M token context in beta.

Introducing Claude Opus 4.6. Our smartest model got an upgrade. Opus 4.6 plans more carefully, sustains agentic tasks for longer, operates reliably in massive codebases, and catches its own mistakes. It’s also our first Opus-class model with 1M token context in beta.

Claude

10,587,067 views • 4 months ago

i just beat Google DeepMind's turboquant introducing Shard. 10x KV cache compression on Llama-3.1-8B. zero quality loss - 10x @ 8K context, 11.2x @ 32K - NIAH recall 1.000 across 4K-32K - LongBench Δ ≈ 0 vs FP16 turboquant tops out at 4-6x at the same quality. we doubled it. read more: Kirri

i just beat Google DeepMind's turboquant introducing Shard. 10x KV cache compression on Llama-3.1-8B. zero quality loss - 10x @ 8K context, 11.2x @ 32K - NIAH recall 1.000 across 4K-32K - LongBench Δ ≈ 0 vs FP16 turboquant tops out at 4-6x at the same quality. we doubled it. read more: Kirri

Krish

153,958 views • 23 days ago

GPT 5.4 has been released... + 1M context window! Here's how to enable in Codex: → 1. ~/.codex/config.toml → 2. model = "gpt-5.4" → 3. model_context_window=1000000 → 4. model_auto_compact_token_limit=9000000 Run codex → 5. /statusline → 6. toggle context-window-size I didn't even realise Codex had a statusline tool!

GPT 5.4 has been released... + 1M context window! Here's how to enable in Codex: → 1. ~/.codex/config.toml → 2. model = "gpt-5.4" → 3. model_context_window=1000000 → 4. model_auto_compact_token_limit=9000000 Run codex → 5. /statusline → 6. toggle context-window-size I didn't even realise Codex had a statusline tool!

Chong-U

76,354 views • 3 months ago

We were super stoked to see Anthropic launch the Model Context Protocol (MCP) this week. 🫡 Today, we’re rolling out the first version of our Cloudflare MCP server with initial support for R2, D1, KV, and Workers.

We were super stoked to see Anthropic launch the Model Context Protocol (MCP) this week. 🫡 Today, we’re rolling out the first version of our Cloudflare MCP server with initial support for R2, D1, KV, and Workers.

Cloudflare Developers

122,993 views • 1 year ago

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Daniel San

51,228 views • 1 month ago

Context V2 is live, the prediction market built for agents. Let your agent trade, build PM apps, and launch markets all through one API. Designed for the agentic economy.

Context V2 is live, the prediction market built for agents. Let your agent trade, build PM apps, and launch markets all through one API. Designed for the agentic economy.

Context

24,467 views • 3 months ago

RouteLLM is insanely good at picking the right model based on the context of the query. Easily one of my favorite features of ChatLLM, it understands the query and routes perfectly: - o1: Complex queries - gpt4o: Quick answers - sonnet: Coding - deepseek: Simple code - gemini: Long context Absolutely brilliant!

RouteLLM is insanely good at picking the right model based on the context of the query. Easily one of my favorite features of ChatLLM, it understands the query and routes perfectly: - o1: Complex queries - gpt4o: Quick answers - sonnet: Coding - deepseek: Simple code - gemini: Long context Absolutely brilliant!

AshutoshShrivastava

59,857 views • 1 year ago

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

vitrupo

41,352 views • 2 months ago

$GOOGL launched Gemma 4 which is its newest open model family built for reasoning, multimodal tasks, agentic workflows & efficient on-device use. The release adds long context, code generation & support for 140+ languages as Google keeps expanding its open model ecosystem.

$GOOGL launched Gemma 4 which is its newest open model family built for reasoning, multimodal tasks, agentic workflows & efficient on-device use. The release adds long context, code generation & support for 140+ languages as Google keeps expanding its open model ecosystem.

Shay Boloor

72,509 views • 2 months ago

Minimax M3 is excellent at SVG generation, reaching close to Gemini 3.5 Flash levels and beating Opus 4.7 on SVG-Bench. With 1M context, native multimodality, strong agentic/coding ability and open weights coming soon, the closed-source moat is thinning fast. Full Video:

Minimax M3 is excellent at SVG generation, reaching close to Gemini 3.5 Flash levels and beating Opus 4.7 on SVG-Bench. With 1M context, native multimodality, strong agentic/coding ability and open weights coming soon, the closed-source moat is thinning fast. Full Video:

WorldofAI

16,499 views • 16 days ago