How to build long-horizon AI agents: behavior specs, ontologies, process supervision - my conversation with Mitchell Troyanovsky, co-founder of Basis 01:09 Why Everyone at Basis Was Whispering to AI when Stephanie Palazzolo walked in 04:12 Accounting as "an Intelligence Over the Economy" 06:11 What Makes an Agent Truly Long-Horizon... 08:24 Inside an Autonomous, Multi-Day Tax Return 10:19 Agents That Hand Off Like Senior Engineers 11:17 A Brief History of Agents: From ReAct to Today 12:33 Why LLMs Have No Long-Term Memory 14:13 Why AutoGPT Didn't Live Up to Its Promise 15:51 The Three Breakthroughs: Opus 3, o1, o3 17:07 Why Reasoning Models Unlocked Agents 18:23 "Let's Verify Step by Step": The Road Not Taken 20:32 Pushing Back on the METR Chart 22:09 Why Coding Agents Won First 25:14 Why Real-World Agents Are Harder 26:55 How Accountants Verify Non-Deterministic Work 29:18 You Can't Scale Tax Returns Like Math 33:16 100 Evals Pass - So What? 35:53 Right Answer, Wrong Process 36:37 Behavior Specs, Explained 39:58 How Specific Should Behaviors Be? 42:18 Context Is Runtime Training Data 44:21 Who Judges the Judge? 46:45 The Move 37 Objection 50:02 The Magic Box Mental Model 52:41 "Nothing Has Changed Since o3" 54:56 Open-Sourcing Behavior Specs with Ankur Goyal Braintrust 59:45 Ontologies: A World for Agents to Live In 01:04:20 Documentation as Codebase 01:06:33 Why the Founding Fathers Were Context Engineers 01:09:05 Onboarding 300 Brilliant Alien Employees 01:11:10 Self-Improving Agent Systems 01:12:50 The Context Mistake Agent Builders Make 01:14:29 RL on Behavior Adherence 01:17:01 Will the Bitter Lesson Swallow the Harness 01:18:46 "Technical Moats Are Not Real Moats" 01:21:03 Advice for AI Buildersshow more

This reference conversation on how to build long-horizon AI agents with @mitch_troy of @trybasis is also available on Spotify, Apple Podcasts and here on YouTube:

Quite the group of investors in Basis, btw @rabois, @Mkclements , @amasad, @ClementDelangue, @jaltma, @JeffDean, @polynoamial, @kvogt, @lennysan, @azeem, @LHSummers, @douwekiela, @pirroh , @scottbelsky, @natfriedman, @danielgross, @lloydblankfein, @vkhosla, @khoslaventures, @Accel, @GVteam, @BoxGroupVC

@mitch_troy @trybasis @steph_palazzolo 🐐

It’s striking how quickly agent design starts sounding like organizational design. Clear expectations, handoffs, context, verification and escalation are the same things teams need to work well together. Agents may force us to make a lot of formerly implicit operating practices explicit.

'behavior specs + process supervision' for long-horizon agents is exactly the shape of what I've built in Orca's Goal mode: the task carries an explicit acceptance contract, and the runtime supervises the process (terminal-state per step, stop conditions, verifier gates). the podcast's framing matches my Terminal-Bench finding — agents that 'know when they're done' beat agents that just run longer.

The “right answer, wrong process” problem is the real bottleneck for long-horizon agents in production. Behavior specs + process supervision only close the loop if the trajectory itself is independently checkable after the fact — not just evaluated inside the runtime that produced it. Out-of-path attestation of the governed path (who decided what, under which constraints, at which step) turns “who judges the judge” into a recomputable property instead of a trust assumption.

@mitch_troy @steph_palazzolo Accounting is an intelligence over the economy!

@mitch_troy @trybasis @steph_palazzolo A mature approach to quantum computing balances technical performance with ethical considerations under resource constraints.

@mitch_troy @trybasis @steph_palazzolo Long-horizon is where memory stops being optional. A spec tells the agent what good looks like, but if what a run learns dies with its context window, every long task restarts from the spec instead of from experience. An ontology only pays off if something writes back to it.

@mitch_troy @trybasis @steph_palazzolo Behavior specs show where the harness changes long-horizon outcomes. Tura uses macro commands; in current DeepSWE tests, tokens and turns drop ~80% while success rises. I’d value your independent review of whether the architecture measures that tradeoff cleanly.

@mitch_troy @trybasis @steph_palazzolo Coding agents won first because the feedback loop is tight. Compiles or it doesn't. Tax returns have that same property but with way more edge cases

Behavior specs and process supervision are table stakes, but most teams can't even define what success looks like for their agents. The gap between 'it ran' and 'it did the right thing' is where long-horizon agents live or die. What does your evaluation stack actually measure at 10,000 steps?

@mitch_troy @trybasis @steph_palazzolo good stuff