The best programmers don't write code anymore. They write a spec. A test. An eval. T Then they turn the AI on and walk away for hours. It runs. It ships. They review. Evals are the unlock most devs are sleeping on. Here's how top agent teams build them right.

there is a typo : no T, "They write a spec. A test. An eval."

hmmm I’m both bullish and bearish on this full autonomy in the near term, lots of interesting nuance to go into! im def bullish on “they review!!” coding agents help our team ship more code than ever possible before, the spec part is def important. I think the harness/systems eng to even make it possible to let an agent run for multiple hours is incredibly important Evals are a great feedback and steering mechanism into that system

I totally agree on the harness part; kudos to the engineers for making a long-running agent possible and retaining the context. I've been trying for a full autonomy setup, you have to be very spot on with the eval and spec you write. Enjoyed your writing. Going to make use of it

tyty and well said!

Evals are just TDD rebranded for the LLM age. Same job—verify output matches intent. Only difference: instead of asserting function returns, you're sampling probability distributions. The work changed. The responsibility didn't.

Good analogy, but TDD is after your code is done to verify the output. Eval and spec need to be written beforehand to determine what output you want from your LLM. Also, it's not everyone's cup of tea to write good eval and spec.

Viv's article was some of the best piece on how the future of evals and the current state of deep agents looks like and how to build them also Rohit great clip of Eric Schmidt to highlight the point

Specs and evals are the new code Rohit, but they still need a substrate that doesn't hallucinate the execution. Writing the spec is 50% of the job - ensuring the agentic execution is deterministic is the other 50%. Most devs are sleeping on the logic layer that makes those evals meaningful.

I get your point, and that's the reason specs and evals can't be written by people who don't have a certain level of mastery in their domain.

100%

Dude - this is BS. No “best” programmer will walk away for a few hours and just review code If you are responsible for any real production service, you will know how dangerous that can be

do you know what a work tree is ? you can iterate over multiple branches? if all your life you have been maintaining legacy code can't help it. but if you are building something new this is the way

Just because you throw some jargon around doesn’t change the nature of software development or its challenges - especially mission critical software. Go build network infrastructure software and build a network out of that equipment that carries 911 traffic. Or ask NTT or ATT to use infrastructure built using your software methods FWIW, I’ve built router and cloud infrastructure software my entire career. And I use Claude every day. I know where it works and where it doesn’t. This walk way stuff is nonsense. You are obnoxious, full of it and not worth engaging.

the "walk away for hours" part misses context. you walk away because the eval is the trust layer. if the eval is wrong, you don't walk away -- you babysit. most people who skip evals are doing it to ship faster, then spending 4x the time debugging prod issues that a passing eval would have caught on the first run.

The guy is paid less for his skills

Spot on. The bottleneck shifted from syntax to validation. But running evals is only half the battle; enforcing them is the unlock. Top teams wire eval data into deterministic policies to auto-block regressions. Vibe checks don't scale; automated enforcement does.

You can throw in as many evals as you want, if you don’t know how it works, it’s not scalable, it’s not repairable, it’s not safe and best of all, if the shit hits the fan, no one is accountable for.

This is how I run my AI agency. I spec the workflow, review the output, five agents handle the rest. Content across three languages, outreach, client ops. My daily input is about 30 minutes of reviewing and approving. The agents do the shipping.

Top devs don’t code, they orchestrate. AI does the work, they focus on guiding and evaluating.

Writing specs has never been fully solved. Coding agents are incredible. But there is no simple recipe for how to use them best. My favorite approach: configuration through conversation. Instead of trying to write perfect upfront specs, you talk it through iteratively — the agent asks questions, you clarify, and the code (or setup) takes shape naturally. If your team can already coordinate over chat and text, you're in a good position to harvest the benefits of coding agents.

the spec+eval pattern is real. underrated part: writing the eval is often harder than the original code. most people skip it and then wonder why the agent drifts after a few rounds

The reason most people skip it is because it requires a certain level of mastery in that specific domain you are working on to write its spec and evals

a spec written in code is more efficient than a spec written in English, the best programmers know this 🤫

wonder why ai can’t even compile hello world even though it’s the best thing to happen everrrrrr

Coding isn’t the leverage anymore. Evals are. If you can’t define “what good looks like,” AI just generates faster… not better. The best devs aren’t typing less they’re judging better. That’s the real skill shift.

The eval part is where most people quit because it feels like extra work. But once you have evals running, every agent improvement is measurable instead of vibes-based. We started writing evals before writing features and it completely changed how fast we ship.

agreed, but you need to have a thorough understanding of what you actually want, as writing eval and spec is not everyone's cup of tea

coding is just high-level management now.

read this it was actually W article

Bro you just help me to get any new innovation in the space of agents

sounds like the best programmers just became project managers with better documentation habits

The spec is the new code. Completely agree. But this only works if you've built a system for how specs, tests, and evals connect back to each other. Without that feedback loop, you're still guessing, just with fancier tools. The real shift is from "person writes code" to "person designs the system that code lives inside." That second skill is way harder to develop.

uh huh👍🏻

top infra teams don't write logic. they write a threshold. a constraint. they spin up the cluster. it routes. they audit. deterministic state machines are the true moat.

The emergence of OpenClaw marks a new chapter in agent evaluation. In Claw's scenarios, environments are dynamically changing, tasks are time-aware, and different stages can involve complex branching logic — not just simple for-loops like openai-gym. ClawMark is an early exploration of this direction.

The best don't waste their times writing specs even. They let their agent ask the questions. Then they see what the agent came up with, adjust, then let it rip.

evals are the missing unlock for most teams. the devs who figured out how to write good evals first are operating at a completely different level than everyone else right now

Love the shift to evals! We're seeing handoffs fail when agents lose context across task queues. Did you add circuit breakers before sleep? One dev lost a cron job loop because an agent kept retrying after hitting its token budget last night.

banger my man

We're seeing hallucinated API calls slip into evals now. Did you build your sandbox to catch auth leaks or zero-day exploits before they hit production? That's the real bottleneck.

I call it success criteria

been running this setup for about a year. the actual code is maybe 20% of the work. the rest is spec quality and knowing when to trust the output. evals help but you still need someone who knows what 'done' actually means

that's waterfall

This is true I haven't coded a line for 3 months. Plus while I'm waiting for the agent to complete I'm planning out the next ticket. At the same time I have agents monitoring production for any issues. I'm there to plan and review that's my job now.