Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

AI agents are writing more code. How can developers ensure that code is correct? Microsoft Research’s Proof-Driven Development (PDD) Kit explores a more rigorous approach: it translates a developer’s intent into a formal specification, then uses formal verification to prove that generated code meets it. PDD Kit works with... show more

Microsoft Research

556,120 subscribers

63,315 views • 2 days ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

13 Comments

Microsoft Research2 days ago

Join @GitHubUniverse Oct 28–29 in SF or online. Explore AI, security, and open source with hands-on sessions, product updates, and devs from around the world.

Harry Nicholls2 days ago

I think this is the future of software development until we solve program synthesis. Love the idea of bringing formal methods in by default, and making proofs legible to those who aren’t experts will be challenging.

kevin joo1 day ago

@VictorTaelin they are starting to catch on!

H.M Gowda1 day ago

simply call PRD.

EDDY VU1 day ago

Bringing formal verification into the agent loop is huge, especially for catching subtle edge cases unit tests consistently miss.

Lachy1 day ago

if the ai drafts the formal spec and i approve it, what catches a spec that is valid but quietly weaker than what i meant?

AxleMode2 days ago

The mutation check in the demo is the bit worth copying: show the deliberately broken versions failing beside the proof result.

Nur Hasan1 day ago

How is it different from PRD?

安叫兽|Bird🕊️ 🔶 BNB1 day ago

先把意图写清楚这步挺关键，不然验证再严也可能证明错东西。

Humanity Labs1 day ago

Formal verification is a strong answer for code. Business work needs its equivalent: explicit standards, logged decisions, exception routes, and a human outcome owner. “The agent completed it” is not the same as “the result is acceptable.”

NextSignal1 day ago

形式化验证过去最大的阻力，是写规范的人力成本比写代码还高。这套工具真正的考题在上一层：证明能保证代码符合规范，但规范本身写错了、或者根本没覆盖真实需求，证明照样成立。所以最难自动化的不是验证，而是把「到底想要什么」说清楚——这一层最后还是得人来守。

Kim Noël ⚡ 📖1 day ago

@grok is this like Lauren tan 's pstack?

Diego Alejandro R1 day ago

We need more and better tools to review written code by AI

Related Videos

“Math is code. And code is math.” Carina Hong CEO Axiom on why that idea matters for AI: “For centuries, mathematicians reasoned in natural language. But because of the Curry–Howard correspondence, a proof can also be a program.” “You can translate natural-language math into formal code using systems like Lean. Lean is both a theorem-proving language and a programming language.” “That means you can generate solutions as code and then formally verify them. You can even use the same system to verify programs themselves.” “This creates a flywheel of generation and verification, where AI can both write solutions and prove that they’re correct.”

“Math is code. And code is math.” Carina Hong CEO Axiom on why that idea matters for AI: “For centuries, mathematicians reasoned in natural language. But because of the Curry–Howard correspondence, a proof can also be a program.” “You can translate natural-language math into formal code using systems like Lean. Lean is both a theorem-proving language and a programming language.” “That means you can generate solutions as code and then formally verify them. You can even use the same system to verify programs themselves.” “This creates a flywheel of generation and verification, where AI can both write solutions and prove that they’re correct.”

Forward Future

24,729 views • 8 months ago

How do you actually formally verify the code underpinning Ethereum's future? In this episode (the finale of the lean Ethereum miniseries), Nico sits down with Alex Hicks (Alexander Hicks), lead of Protocol Snarkification at the Ethereum Foundation, to break down formal verification from first principles. They cover: – What formal verification actually is and the trust boundaries between proof assistants, SMT solvers, and kernels – The full verification stack for RISC-V ZKVMs: from SAIL specs to constraint extraction to soundness proofs – Why writing constraints directly in Lean makes proofs 10–100x more ergonomic – How AI is now proving hard theorems in hours for $200 — and what that unlocks for the whole pipeline They also explore the boundaries problem, why specs can have bugs too, and the end goal of a full Lean stack that bypasses Rust and LLVM entirely. Listen to the full episode ------------------------------------------------------------ TIMECODES: 09:16 – What is formal verification? Proof assistants vs SMT solvers 18:33 – Formal verification of code: specs, semantics, and trust boundaries 29:30 – Formally verifying the Lean Ethereum stack: RISC-V ZKVMs in focus 33:02 – Extracting ZKVM constraints into Lean and proving soundness 36:35 – Writing constraints directly in Lean: 10–100x better proof ergonomics 44:02 – Proving Polishchuk–Spielman in 8 hours for $200 with AI 51:01 – The end goal: a full Lean stack bypassing Rust and LLVM

How do you actually formally verify the code underpinning Ethereum's future? In this episode (the finale of the lean Ethereum miniseries), Nico sits down with Alex Hicks (Alexander Hicks), lead of Protocol Snarkification at the Ethereum Foundation, to break down formal verification from first principles. They cover: – What formal verification actually is and the trust boundaries between proof assistants, SMT solvers, and kernels – The full verification stack for RISC-V ZKVMs: from SAIL specs to constraint extraction to soundness proofs – Why writing constraints directly in Lean makes proofs 10–100x more ergonomic – How AI is now proving hard theorems in hours for $200 — and what that unlocks for the whole pipeline They also explore the boundaries problem, why specs can have bugs too, and the end goal of a full Lean stack that bypasses Rust and LLVM entirely. Listen to the full episode ------------------------------------------------------------ TIMECODES: 09:16 – What is formal verification? Proof assistants vs SMT solvers 18:33 – Formal verification of code: specs, semantics, and trust boundaries 29:30 – Formally verifying the Lean Ethereum stack: RISC-V ZKVMs in focus 33:02 – Extracting ZKVM constraints into Lean and proving soundness 36:35 – Writing constraints directly in Lean: 10–100x better proof ergonomics 44:02 – Proving Polishchuk–Spielman in 8 hours for $200 with AI 51:01 – The end goal: a full Lean stack bypassing Rust and LLVM

Zero Knowledge Podcast

15,289 views • 7 months ago

Your AI coding agent says “done.” Where’s the proof? ProofShot lets agents test your UI in a real browser and show the evidence: video, screenshots, console errors and server logs. It bundles everything into a report that can go directly into a GitHub PR. Works with Claude Code, Cursor, Codex and other shell-based agents.

Your AI coding agent says “done.” Where’s the proof? ProofShot lets agents test your UI in a real browser and show the evidence: video, screenshots, console errors and server logs. It bundles everything into a report that can go directly into a GitHub PR. Works with Claude Code, Cursor, Codex and other shell-based agents.

0xMarioNawfal

47,383 views • 15 days ago

I’m always interested in seeing how far AI can go beyond generating code. IQuest-Q1 is a code-focused large model built for developers, with performance evaluated across different task scenarios. From building interactive game demos and generating 3D scenes to debugging code and handling complex tasks with multimodal information and tools, it explores some interesting possibilities for AI in development. Excited to see how models like this evolve! Explore IQuest-Q1: #coding #LLM

I’m always interested in seeing how far AI can go beyond generating code. IQuest-Q1 is a code-focused large model built for developers, with performance evaluated across different task scenarios. From building interactive game demos and generating 3D scenes to debugging code and handling complex tasks with multimodal information and tools, it explores some interesting possibilities for AI in development. Excited to see how models like this evolve! Explore IQuest-Q1: #coding #LLM

Manish Kumar Shah

75,026 views • 8 days ago

We've built a massive verification bottleneck: • 72% of developers use AI to generate code every day • 96% of them don't fully trust it Writing code is now free. All the work now is verifying that the code does what it's supposed to. According to a new State of Code Survey, almost half of the respondents believe the number one skill that will matter most in the coming years is "reviewing and validating AI-generated code for quality and security". Not prompt engineering or vibe-coding, but checking AI's work.

We've built a massive verification bottleneck: • 72% of developers use AI to generate code every day • 96% of them don't fully trust it Writing code is now free. All the work now is verifying that the code does what it's supposed to. According to a new State of Code Survey, almost half of the respondents believe the number one skill that will matter most in the coming years is "reviewing and validating AI-generated code for quality and security". Not prompt engineering or vibe-coding, but checking AI's work.

Santiago

16,299 views • 9 months ago

Sharing Magnus’ first partner, Runtime Vеrification — they’re bringing their world-class formal verification services to Magnus! “I'm excited to be working side-by-side with the team at Immunefi to bring the next generation of security to web3 developers! With the unified and integrated approach to security that Magnus puts forward, the wealth of data that Codexa provides, and the power of combining AI and formal reasoning tools, I'm sure that we will make web3 security a breeze, even for newcomers.” — Everett Hildenbrandt, CEO at Runtime Verification Learn more👇

Sharing Magnus’ first partner, Runtime Vеrification — they’re bringing their world-class formal verification services to Magnus! “I'm excited to be working side-by-side with the team at Immunefi to bring the next generation of security to web3 developers! With the unified and integrated approach to security that Magnus puts forward, the wealth of data that Codexa provides, and the power of combining AI and formal reasoning tools, I'm sure that we will make web3 security a breeze, even for newcomers.” — Everett Hildenbrandt, CEO at Runtime Verification Learn more👇

Immunefi

2,289,341 views • 1 year ago

AI is changing the software engineering craft. Anders Hejlsberg (Anders Hejlsberg) - creator of C#, TypeScript and industry legend - on why code review needs to get more enjoyable in response: #1 - AI is shifting the craft from writing code, to reviewing code: "In a sense, we're all turning into project managers. We can have an army of junior programmers, called agents, that will just spit out reams of code but someone's got to have the big picture and review all of that. And so, increasingly, our craft is going from one of writing the code, to one of reviewing the code and building the architecture of the code and overseeing the work. It's a different kind of craft. It's a different kind of enjoyment. I've always liked writing the code. To me that was the fulfilling part, seeing it work. In a way, AI robs a little bit of that, because I am less interested in reviewing code." #2 - The code review experience should be improved: "I think we could also make the process of reviewing code much more interesting than it is today. I mean, today, you see a list of diffs in alphabetical order and now it's up to you to make heads or tails of it. There are more pedagogical ways of presenting that. And you could have commentary generated by the AI that tells you what the changes are and whatever, and then tries to guide you along. So that symbiotic relationship, I think we need to work on that more and to keep the enjoyment in there."

AI is changing the software engineering craft. Anders Hejlsberg (Anders Hejlsberg) - creator of C#, TypeScript and industry legend - on why code review needs to get more enjoyable in response: #1 - AI is shifting the craft from writing code, to reviewing code: "In a sense, we're all turning into project managers. We can have an army of junior programmers, called agents, that will just spit out reams of code but someone's got to have the big picture and review all of that. And so, increasingly, our craft is going from one of writing the code, to one of reviewing the code and building the architecture of the code and overseeing the work. It's a different kind of craft. It's a different kind of enjoyment. I've always liked writing the code. To me that was the fulfilling part, seeing it work. In a way, AI robs a little bit of that, because I am less interested in reviewing code." #2 - The code review experience should be improved: "I think we could also make the process of reviewing code much more interesting than it is today. I mean, today, you see a list of diffs in alphabetical order and now it's up to you to make heads or tails of it. There are more pedagogical ways of presenting that. And you could have commentary generated by the AI that tells you what the changes are and whatever, and then tries to guide you along. So that symbiotic relationship, I think we need to work on that more and to keep the enjoyment in there."

The Pragmatic Engineer

39,073 views • 5 months ago

Reviewing AI-generated code is the new bottleneck. Writing code was slow before. Now, in a few minutes, you can generate virtually unlimited lines of code using AI. But developers are now spending 70% of their time checking that code. That's the new bottleneck. I've been using Cline, and they have a feature that actually helps with this: You can click a button and have Cline generate an inline explanation for every single line of AI-generated code. Take a look at the attached video. What it does: 1. You can read the code and the explanation together 2. You don't lose context while reviewing code 3. You understand the "why" behind each change And here is the best part: It works on any git diff. You can use it to review PRs, understand commits from other people, or catch up on a codebase you haven't touched in a while.

Reviewing AI-generated code is the new bottleneck. Writing code was slow before. Now, in a few minutes, you can generate virtually unlimited lines of code using AI. But developers are now spending 70% of their time checking that code. That's the new bottleneck. I've been using Cline, and they have a feature that actually helps with this: You can click a button and have Cline generate an inline explanation for every single line of AI-generated code. Take a look at the attached video. What it does: 1. You can read the code and the explanation together 2. You don't lose context while reviewing code 3. You understand the "why" behind each change And here is the best part: It works on any git diff. You can use it to review PRs, understand commits from other people, or catch up on a codebase you haven't touched in a while.

Santiago

64,477 views • 10 months ago

Xavier Leroy (creator of OCaml) is an expert in compilers, formal verification of software and functional programming. This interview should be an approachable resource if you're curious about formal verification of software since I was learning that on the fly during it. In this episode: • OCaml compared with Rust and JavaScript • What is formal verification and how does it work • How languages call each other across boundaries • How to address "almost-correct" LLM code • How type inference works in programming languages Where to watch: • YouTube - • Spotify - • Apple Podcasts - • Transcript - Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at Chapters: 00:00 - Intro 00:43 - What sets OCaml apart 04:39 - OCaml vs Rust 07:57 - Why is manual memory management more performant 11:21 - Javascript vs OCaml 14:00 - Famous Rob Pike quote 16:05 - Type inference and how it works 22:12 - What is formal verification and how does it work 40:07 - What made multicore support difficult for OCaml 50:17 - How programming languages interface and call each other 57:41 - The danger of almost-correct LLM code 01:05:39 - How LLMs will change programming languages 01:10:26 - Industry vs academia 01:15:05 - Most interesting unsolved problems 01:18:30 - Top book recommendations for engineers 01:21:17 - Advice for his younger self 01:23:31 - Outro

Xavier Leroy (creator of OCaml) is an expert in compilers, formal verification of software and functional programming. This interview should be an approachable resource if you're curious about formal verification of software since I was learning that on the fly during it. In this episode: • OCaml compared with Rust and JavaScript • What is formal verification and how does it work • How languages call each other across boundaries • How to address "almost-correct" LLM code • How type inference works in programming languages Where to watch: • YouTube - • Spotify - • Apple Podcasts - • Transcript - Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at Chapters: 00:00 - Intro 00:43 - What sets OCaml apart 04:39 - OCaml vs Rust 07:57 - Why is manual memory management more performant 11:21 - Javascript vs OCaml 14:00 - Famous Rob Pike quote 16:05 - Type inference and how it works 22:12 - What is formal verification and how does it work 40:07 - What made multicore support difficult for OCaml 50:17 - How programming languages interface and call each other 57:41 - The danger of almost-correct LLM code 01:05:39 - How LLMs will change programming languages 01:10:26 - Industry vs academia 01:15:05 - Most interesting unsolved problems 01:18:30 - Top book recommendations for engineers 01:21:17 - Advice for his younger self 01:23:31 - Outro

Ryan Peterman

24,256 views • 3 months ago

There’s a popular theory that AI will finally make formal verification mainstream because mathematical proof of correctness will be needed when machines write most or all of the code. But will this happen? Hillel Wayne is one of the best people to answer. Timestamps: 00:00 Intro 04:32 The Crossover Project 11:37 What software engineering does better 15:30 What traditional engineering does better 18:17 Formal methods 29:32 TLA+: what it is and demo 36:58 TLA+ at Amazon 38:10 Ways distributed systems break 41:03 Formal methods and systems thinking 46:20 The value of learning math 50:23 What TLA+ is good for and isn’t 52:50 Alloy: a declarative language for software modeling 58:53 Other formal methods tools 1:01:24 Property-based testing 1:05:31 AI and the need for formal verification 1:12:29 Logic for programmers 1:14:35 Hillel’s 2025 prediction on AI’s impact 1:21:30 Book recommendation Brought to you by: • Antithesis – verify your system’s correctness without human review or traditional integration tests – and avoid bugs or outages. • turbopuffer – a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. • WorkOS – everything you need to make your app enterprise ready. Two things I found especially interesting, talking with Hillel: 1. Amazon used TLA+ to find a bug almost impossible to locate without formal methods. In the paper How AWS uses formal methods, the AWS team shared that they’d found a complicated bug for which the shortest error trace to exhibit was 35 steps (!!). The bug passed unnoticed through extensive design review, code reviews, and testing. AWS concluded they wouldn’t have uncovered it if they’d stuck to conventional testing approaches. 2. Why not use formal verification for everything, then? It’s because specs in the real world are a nightmare to write. Even a simple problem like “find the file in a directory that has the most lines” gets complicated when modeled with formal methods. We would have to answer questions like: ‘do we look at ASCII or UTF-8 new line characters, what about unreadable files, and Symlinks?’ Without formal methods, we can write a simple verification that is right in 99%+ of cases. Formal methods require a lot of extra effort for the less than 1% of exotic use cases!

There’s a popular theory that AI will finally make formal verification mainstream because mathematical proof of correctness will be needed when machines write most or all of the code. But will this happen? Hillel Wayne is one of the best people to answer. Timestamps: 00:00 Intro 04:32 The Crossover Project 11:37 What software engineering does better 15:30 What traditional engineering does better 18:17 Formal methods 29:32 TLA+: what it is and demo 36:58 TLA+ at Amazon 38:10 Ways distributed systems break 41:03 Formal methods and systems thinking 46:20 The value of learning math 50:23 What TLA+ is good for and isn’t 52:50 Alloy: a declarative language for software modeling 58:53 Other formal methods tools 1:01:24 Property-based testing 1:05:31 AI and the need for formal verification 1:12:29 Logic for programmers 1:14:35 Hillel’s 2025 prediction on AI’s impact 1:21:30 Book recommendation Brought to you by: • Antithesis – verify your system’s correctness without human review or traditional integration tests – and avoid bugs or outages. • turbopuffer – a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. • WorkOS – everything you need to make your app enterprise ready. Two things I found especially interesting, talking with Hillel: 1. Amazon used TLA+ to find a bug almost impossible to locate without formal methods. In the paper How AWS uses formal methods, the AWS team shared that they’d found a complicated bug for which the shortest error trace to exhibit was 35 steps (!!). The bug passed unnoticed through extensive design review, code reviews, and testing. AWS concluded they wouldn’t have uncovered it if they’d stuck to conventional testing approaches. 2. Why not use formal verification for everything, then? It’s because specs in the real world are a nightmare to write. Even a simple problem like “find the file in a directory that has the most lines” gets complicated when modeled with formal methods. We would have to answer questions like: ‘do we look at ASCII or UTF-8 new line characters, what about unreadable files, and Symlinks?’ Without formal methods, we can write a simple verification that is right in 99%+ of cases. Formal methods require a lot of extra effort for the less than 1% of exotic use cases!

Gergely Orosz

34,926 views • 2 months ago

Wait… IQuest-Q1 actually built this from a written brief. I asked it to build Velora, an interactive SaaS analytics dashboard The result? A complete working dashboard I could click through, filter and explore. This is what makes IQuest-Q1 interesting. It’s built for more than standard code generation. IQuest-Q1 is a large, code-focused model from IQuest Research, and its capabilities go beyond simply writing code. It is being evaluated across: → Game & application development → 3D spatial generation → Code diagnosis & repair → Multimodal tasks with tools And that matters because real-world software development is rarely just about writing code. You need to build, debug, reason, use tools, and work across different environments. That’s why the Velora demo is interesting: I gave it a product brief, and it turned that idea into an interactive web experience. The goal is not just to generate code. It’s to handle more of the workflow from idea → implementation → working product. #coding #LLM

Wait… IQuest-Q1 actually built this from a written brief. I asked it to build Velora, an interactive SaaS analytics dashboard The result? A complete working dashboard I could click through, filter and explore. This is what makes IQuest-Q1 interesting. It’s built for more than standard code generation. IQuest-Q1 is a large, code-focused model from IQuest Research, and its capabilities go beyond simply writing code. It is being evaluated across: → Game & application development → 3D spatial generation → Code diagnosis & repair → Multimodal tasks with tools And that matters because real-world software development is rarely just about writing code. You need to build, debug, reason, use tools, and work across different environments. That’s why the Velora demo is interesting: I gave it a product brief, and it turned that idea into an interactive web experience. The goal is not just to generate code. It’s to handle more of the workflow from idea → implementation → working product. #coding #LLM

Atal

41,419 views • 22 days ago

Creator of C++, Bjarne Stroustrup: AI-generated code isn't ready — it generates more bugs, more bloat, more security holes, and is nearly impossible to validate "senior developers are already retiring rather than deal with it" The problem is that even a small prompt change can shift the entire codebase in unpredictable ways

Creator of C++, Bjarne Stroustrup: AI-generated code isn't ready — it generates more bugs, more bloat, more security holes, and is nearly impossible to validate "senior developers are already retiring rather than deal with it" The problem is that even a small prompt change can shift the entire codebase in unpredictable ways

Haider.

1,752,400 views • 5 months ago

A good teammate earns trust. They give you quality work and maybe most underrated, are a delight to work with Today we're launching Devin and Slack Code. We've worked with Slack on new primitives, bringing more of what makes Devin great into Slack. Now Devin can take more complex work and start a Code Channel, bringing team collaboration and more information about code diffs, PRs, and other files into the same work space. What makes Devin different is it has its own computer. It can deploy various OS/mobile platforms, test the code, and really prove that it did a good job to your team (like replying with a video recording). Better yet, Devin is a delight to work with in Code Channels. It jumps in when needed, consolidates feedback, and doesn't overburden your team with replies like other agents We're so excited to be working with Marc Benioff and team to bring this to market, we can't wait to see what you (and your team!) build

A good teammate earns trust. They give you quality work and maybe most underrated, are a delight to work with Today we're launching Devin and Slack Code. We've worked with Slack on new primitives, bringing more of what makes Devin great into Slack. Now Devin can take more complex work and start a Code Channel, bringing team collaboration and more information about code diffs, PRs, and other files into the same work space. What makes Devin different is it has its own computer. It can deploy various OS/mobile platforms, test the code, and really prove that it did a good job to your team (like replying with a video recording). Better yet, Devin is a delight to work with in Code Channels. It jumps in when needed, consolidates feedback, and doesn't overburden your team with replies like other agents We're so excited to be working with Marc Benioff and team to bring this to market, we can't wait to see what you (and your team!) build

Jeff Wang

22,952 views • 2 months ago

INSANE: Grok 3 + Lovable = BEST AI Agent Apps!!!! Grok 3 generated the code for an AI agent that manages your Gmail - send, reply, and draft effortlessly Here's how it works: - Connect with your Gmail using Composio - Use Lovable to build an extremely cool frontend for the Agent - Agentic workflow with LlamaIndex Built with LlamaIndex 🦙 To connect your agents with more tools like Gmail, use

INSANE: Grok 3 + Lovable = BEST AI Agent Apps!!!! Grok 3 generated the code for an AI agent that manages your Gmail - send, reply, and draft effortlessly Here's how it works: - Connect with your Gmail using Composio - Use Lovable to build an extremely cool frontend for the Agent - Agentic workflow with LlamaIndex Built with LlamaIndex 🦙 To connect your agents with more tools like Gmail, use

Karan Vaidya

62,030 views • 1 year ago

🚂 Introducing Agent Prompt Train 🚂 An open-sourced high-performance proxy that works with Claude Code to help turn it from a single-developer tool into a collaborative development platform. Claude Code changed the game. Now, Agent Prompt Train is here to help teams: ✅ Log every conversation for review ✅ Analyze & optimize prompts with AI ✅ Turn best practices into reusable team commands ✨ The payoff: Smarter prompts, faster learning, and shared knowledge across your whole team. 📖 Read more on the blog & test it out 👇

🚂 Introducing Agent Prompt Train 🚂 An open-sourced high-performance proxy that works with Claude Code to help turn it from a single-developer tool into a collaborative development platform. Claude Code changed the game. Now, Agent Prompt Train is here to help teams: ✅ Log every conversation for review ✅ Analyze & optimize prompts with AI ✅ Turn best practices into reusable team commands ✨ The payoff: Smarter prompts, faster learning, and shared knowledge across your whole team. 📖 Read more on the blog & test it out 👇

Moonsong Labs 🌙

21,532 views • 1 year ago

“In the last 4–5 months, we stopped typing code.” Cognition CEO Scott Wu says almost all coding at Cognition is really just done through prompts now: “The code that we check into GitHub, how much of it was typed by a human at this point? Almost none.” “Now it looks a lot more like: how do we optimize testing, review, and planning - not how do we make AI good at writing code based on the prompt you give. At this point, it’s frankly already done.”

“In the last 4–5 months, we stopped typing code.” Cognition CEO Scott Wu says almost all coding at Cognition is really just done through prompts now: “The code that we check into GitHub, how much of it was typed by a human at this point? Almost none.” “Now it looks a lot more like: how do we optimize testing, review, and planning - not how do we make AI good at writing code based on the prompt you give. At this point, it’s frankly already done.”

TBPN

30,438 views • 8 months ago

Nobody is writing 90% of their code using AI. Here's the uncomfortable truth: The real productivity gain from using AI to write code is closer to 10%, nowhere near the 90% people claim. Sundar Pichai said in 2024 that 30% of the new code at Google was AI-generated. However, he went on to admit, during Lex Friedman's podcast, that engineering velocity had only increased by about 10%. AI-generated code isn't free code. It still has to be reviewed, tested, and made production-ready. Optimizing a single step (code generation) doesn't boost output if bottlenecks shift elsewhere (code reviews). It doesn't matter how much code you generate if you can't keep up the review process. The solution: Automate as much as you can the review and verification of your code. I'm working with Sonar, who is sponsoring this post, and they will take care of the code quality and security analysis of your code: • They review over 300B lines of code every single day • They cover reliability, security, and maintainability for your code • You can integrate them into your CI/CD pipeline • You can install them in your IDE (I use their VSCode extension) • Support for more than 30 languages Here is a link so you can check them out:

Nobody is writing 90% of their code using AI. Here's the uncomfortable truth: The real productivity gain from using AI to write code is closer to 10%, nowhere near the 90% people claim. Sundar Pichai said in 2024 that 30% of the new code at Google was AI-generated. However, he went on to admit, during Lex Friedman's podcast, that engineering velocity had only increased by about 10%. AI-generated code isn't free code. It still has to be reviewed, tested, and made production-ready. Optimizing a single step (code generation) doesn't boost output if bottlenecks shift elsewhere (code reviews). It doesn't matter how much code you generate if you can't keep up the review process. The solution: Automate as much as you can the review and verification of your code. I'm working with Sonar, who is sponsoring this post, and they will take care of the code quality and security analysis of your code: • They review over 300B lines of code every single day • They cover reliability, security, and maintainability for your code • You can integrate them into your CI/CD pipeline • You can install them in your IDE (I use their VSCode extension) • Support for more than 30 languages Here is a link so you can check them out:

Santiago

296,750 views • 1 year ago