When Mudith Jayasekara and I met Gabe Pereyra, we were expecting just another vanilla intro call and instead had the best yarn about research, the state of LLMs, and where intelligence is actually heading. It's rare to meet a founder this deep in the weeds who's also building for... one of the most important verticals in this new age of intelligence So it was awesome to sit down with Gabe for an extended discussion on what it take to build agents that can reliably complete work over hours, days, or even longer? We talked about why agents today struggle with search and long context windows and how techniques like KV-cache compaction, synthetic data, and continual learning could help. 0:00 Introduction 0:36 Getting legal agents to review the whole data room 2:08 Data rooms larger than any context window 5:28 How far open-source models can go 7:58 Where specialist models fit in legal AI 10:59 Training legal models when client data is off-limits 13:06 Teaching a model how a law firm works 13:59 What belongs in context vs. model weights 15:36 From firm-wide AI to a model for every lawyer 18:37 What training adds beyond retrieving the right cases 20:26 Why context windows have plateaued 24:01 How models could learn continuously on the job 26:12 Can AI recursively improve AI research? 27:07 Research agents can run experiments but not choose them 30:00 Why open-ended research is hard to train 33:47 Why deployment, not intelligence, is the bottleneck 35:08 The cost of frontier intelligence 36:59 Different neolabs, different paths to intelligence 39:26 Using open datasets to compare research methods 41:13 Conclusionshow more

@gabepereyra Full conversation here:

@mudithj @gabepereyra We love @gabepereyra and the @harvey team!

@baseten @mudithj @gabepereyra Excited to listen!

@baseten @mudithj @gabepereyra Thanks Madison!

@mudithj @gabepereyra Such a fun conversation! Loved working on this 💚

@mudithj @gabepereyra Thanks Lan!

@mudithj @gabepereyra Great episode

I was telling @AgentM_Tech about my side project for SFT'ing a small model to create a DevOps/SRE specialist model, she told me it's wwwaaaayyyyyyyy more difficult in the legal space...I thought my non-legal brain followed along with the details she told me about, but I guess I missed where it translated into a real need for a custom model per lawyer...but now it just kinda seems obvious rt? 🤔🤦‍♂️

@mudithj @gabepereyra @AgentM_Tech Yeah legal is much harder for many reasons

@mudithj @gabepereyra handling legal data rooms larger than any context window requires shifting work onto persistent storage layers instead of inflating context limits indefinitely.

@mudithj @gabepereyra The founders worth backing right now are the ones this deep in one specific domain. Generic AI for everything is just noise. The real progress comes from people who understand one hard problem so well the model becomes a scalpel, not a search bar.

@mudithj @gabepereyra Important background:

@mudithj @gabepereyra The 13:06 chapter is the crux. "Teaching a model how a law firm works" sounds like fine-tuning but it's really archaeology — the process lives in precedent files, redline habits, and one partner's memory. Gabe's right that deployment, not intelligence, is the bottleneck.

@mudithj @gabepereyra the hard part is not put more docs in context or use a better model. it’s knowing what belongs in retrieval vs the model, and how to verify it. legal source material is massive. failures are often not dramatic hallucinations, but omissions. this convo made that concrete.

The two chapter titles that matter most sit right next to each other: teaching a model how a law firm works, and why deployment, not intelligence, is the bottleneck. They are the same problem. A firm's actual process lives in unwritten habits and matter history, and until that exists in a form a model can use, more intelligence just idles.

@mudithj @gabepereyra The chapter list tells the story: every hard problem here is firm-shaped, not model-shaped. Data rooms bigger than any context window, client data you cannot train on, teaching a model how one firm works. Deployment as bottleneck means the missing input is the firm's own process.

@mudithj @gabepereyra The deployment bottleneck is where most teams fail. Models can handle legal reasoning today. Serving them reliably at firm scale with confidentiality constraints is the unsolved problem.

There is a Reason why not very intelligent people are founders of the majority of the companies. A very intelligent person is obsessed with curiosity more than Business. Look at Elon musk. You think he still wants businesses?? Man is obsessed and he finds a way to put his curiosity to work.