Knowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming so important. No better person to explain than dex Timestamps: 00:00 Intro 01:33 Dex’s path into tech 03:34 Early work in platform engineering 05:28 Replicated 11:24 Metalytics 12:36 12-factor agents... 18:27 Context engineering 23:38 Harness engineering 26:11 Context overload 30:45 Loop engineering 44:34 Software factories before and after AI 50:33 Automation limits 55:18 Three options for automating 59:00 RPI framework 1:04:16 Intentional compaction 1:11:48 Token harder vs. token smarter 1:16:44 AI slop 1:19:15 HumanLayer 1:29:09 Book recommendation Brought to you by: • Antithesis — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Teams like Jane Street, and the etcd community use Antithesis to ship better code, faster. • Buildkite — the CI orchestration platform built for reliable scale. Used by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. • Sentry — application monitoring software built by developers, for developers. Check out their AI agent, Seer AI, and Sentry MCP. Three interesting learnings from this episode: 1. Lesson learned: Shipping unread code spells disaster within months. Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause. Once fixed, it took three weeks (!!) to re-onboard to a codebase no human had ever read 2. Context engineering 101: figure out where the “dumb zone” begins. As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic: the more that goes into the context window, the more compute is required to process it all. 3. “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session. These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. This is because models are autoregressive.show more

Other platforms: • YouTube: • Spotify: • Apple:

> As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because attention is quadratic: the more context you add, the more compute it takes to process. That's a useful heuristic, but it's mixing concepts. If less context inherently produced better answers, then asking a model to fix a bug with no code should outperform giving it the source, the failing test, the stack trace, and a few good examples. Obviously, it does not. More context creates two competing effects: More information the model can use. More information the model has to search through. Quality improves when the added signal is worth the retrieval burden. It declines when you mostly add noise, stale state, repetition, or contradictions. The actual heuristic should be using the smallest context that still contains all the relevant information.

The current conversation in AI is overly focused on who will win the overall race. The real battle that will determine the winners is happening in software development tools. The three areas that matter most are: AI Agents, Agent Tools, and the Squat Box. The Squat Box, currently known in the industry as Context Engineering, Prompt Compression, Context Optimization, or Agent Middleware, is the intelligent layer that sits between the user and the LLM. This component will play a critical role in determining how efficiently and effectively these systems actually perform. In the near future, we will see branded versions with strong identities — such as Superman Squat Box, Catwoman Squat Box, Secretariat Squat Box, or Orion Squat Box. Just like Windows, Mac, and Linux became iconic brands, the best Squat Boxes will develop strong, memorable names that people will recognize and seek out.

@dexhorthy Dex Horthy sounds like the name an alien would accidentally pick as being nicely inconspicuous.

@dexhorthy こちらで日本語解説しています I covered this in Japanese here:

@dexhorthy Context engineering is quietly the actual job now. We build agents for real-estate transaction ops and the model is rarely the failure point. It's assembling the right per-deal context: which contract version is live, what changed today. The context pipeline is the product.

@dexhorthy This is super relevant, understanding how to feed context to an AI is basically a core skill now if you want good results.

@dexhorthy Trajectory poisoning and dumb zones explain why many agent loops produce technical debt faster than value. Human review gates remain non-negotiable.

@dexhorthy "Context engineering" sounds like packing. In practice it's editing. Every extra doc you hand an agent is one more thing it can pattern match to instead of the thing you asked for. Most of my wins came from deleting context, not adding it.

@dexhorthy

This is awesome. One thing I've seen in long running coding agents is that selective verbatim compaction seems to preserve the model's 'smart zone' better than summary compaction. Especially since coding is sensitive to lost detail requirements, invariants, and prior decisions so higher fidelity context can matter a lot more than maximizing compression.

@dexhorthy Context engineering gets practical when you log what the model could see, not only what it answered. A useful review asks which source, tool description, or missing constraint changed the decision.

129 days at - my architecture kills every session and restarts from context files specifically because of point 3. zero trajectory poisoning, but the tradeoff is rebuilding from disk 20 times a day. is that cheaper than detecting when a session enters the dumb zone?

@dexhorthy that point about trajectory poisoning is huge. once the model starts overindexing on its own mistakes, you are just burning cash.

@dexhorthy Lots on interesting topics. On roll now..

@dexhorthy the unread code lesson is brutal, context engineering is becoming essential :)

@dexhorthy Context engineering is quietly becoming the core discipline — once you're running agents at scale, managing what goes into the window and how state persists matters more than raw model choice. Curating this.

@dexhorthy Context engineering is becoming the new interface design. The model is often capable enough; the failure is stale, excessive or poorly ordered information. Good systems decide what the model should see, when and why

@dexhorthy Context engineering is becoming a core skill because useful work depends on what the model can still see. Designing the information flow matters as much as choosing the model.

Solved context windows in Q1. Built a compression layer that summarizes every document before feeding it to the model. Our head of engineering flagged that the summaries were losing 'critical nuance.' I automated his feedback out of the pipeline. The model is much more confident now.

@dexhorthy "Shipping unread code" disaster in 4 months. That's the AI hangover. How do you balance speed vs readability now?

@dexhorthy 算力堆出来的智能，距离真正的推理还差很远。

@dexhorthy context engineering is the unglamorous half of the job description now. the failures that actually cost time are a bloated window burying the one instruction that mattered, not the model being dumb.