Загрузка видео...
Не удалось загрузить видео
Matt Pocock (TypeScript wizard, ex-Vercel): "Opus 5 got addicted to writing tests that just repeat the code back to itself." in a 22-min talk at AI Engineer Paris, Matt walks through how to stop AI PRs from turning your repo into a slop cannon: 5:30 → the tests AI... show more
174,215 просмотров • 18 дней назад •via X (Twitter)
Комментарии: 35
the reviewer-should-commit-not-comment point is underrated. most AI review setups still route fixes back through a human typing the same patch by hand, which defeats half the point of having a reviewer agent
spot on, reviewer that comments instead of committing just creates more work
Tests that echo the code are a quiet fail. I ask for a case that would break if the function lied. Mirror tests feel green and teach nothing.
"mirror tests feel green and teach nothing" is a great way to put it
Yeah, I believe so. 😊
The idea of making the reviewer fix things instead of just leaving comments is huge!
right? half my review time used to be reading bot comments i then fixed myself
I wonder what has he built? Ohh right he's just talking, thats enough.
fair to judge the ideas, not the guy. the reviewer-commits-fixes part works regardless of who said it
yeah you are right, i'm just puzzled what have people been doing if this is something new? I get dude is "announcing skills" for "real engineers" though is he one? If i would like to be influencer there is a lot to learn from him, can't deny that.
full 3-layer system in one image ↓
That guy has a slop skill set and nothing more. He knows very little about the AI pull request review.
what would you do differently? genuinely curious what's working for you on AI PR review
I got my own AI review solution with the "bring your own agent" approach. It writes summaries and comments, performs issues classification, does the PR gating on the github/azure/gitlab.
@grok from this video explain step by step the whole process so i can replicate it
These points hit exactly the pain points of current AI coding. I'm gonna go watch this full talk right now.
the part at 15:34 is the one to steal first, reviewer commits instead of commenting
this was happening with older models too - 4.8. we also figured out what was the problem and created domain specific review agents which fixed the entire set of the problems!
domain-specific reviewers is the move. how many did you end up with, one per area of the codebase?
this talk is way more useful than most ai coding advice
glad it landed, practical > theoretical every time
Couldn't agree more
thanks
5 is the slide number. The permission story is still the footnote.
true, permissions get the headlines but the tests are the silent risk
Thx for sharing this broski, saved
appreciate it
Watching that exact part now and circular test generation is way too common lately.
circular tests are everywhere now. green CI, zero coverage of what the code actually does
Ex vercel? Wasn’t it ex-voice coach now building hobby apps with ai?
both actually. voice coach before he got into typescript, then vercel. the talk still slaps either way
what happend to him
reviewed one too many AI PRs lol
Saying Matt is ex-Vercel is like how Lex Fridman is ex-MIT.
Thanks @mattpocockuk for this good video. Couple remarks: - "coding standards belong in a separate review agent" => yes, this confirms recent changes in all major "SDD" frameworks, converging to way I called "The Canonical Implementation Loop". You have a Coder agent and a Reviewer Agent. - "AI reviewer shouldn't comment. it should commit the fixes" => ok for the review once the PR is created, but I won't say that for the in-loop reviewer (I called "Task Inspector"). If you start an implementation loop in the reviewer, it can starts drifting from its goal of findings all the missed acceptance criteria and do it properly. It can "try" to do the fix if it is very small issue, to avoid restarting a new coder to fix the issue, but things like incomplete feature, it should sent the task back to the coder agent.
