Loading video...

Video Failed to Load

Go Home

The hype around AI & how it will supposedly replace developers is everywhere. I'd have been a fool not to investigate whether there was any truth to these claims. So, I embarked on a journey to learn about Anthropic Claude and its “agentic” capabilities. Let's take a look👇

82,988 views • 1 year ago •via X (Twitter)

11 Comments

Matteo Collina's profile picture
Matteo Collina1 year ago

In the quick demo, Claude Desktop raised code coverage of fgh by 2% in roughly 12 minutes of work (3x speed). The secret is a tight TDD loop based on top of @platformatic mcp-node.

Matteo Collina's profile picture
Matteo Collina1 year ago

My rough estimation is that it would have required me 1 hour of work to achieve the same result without AI. Read all about it in our blog post:

Matteo Collina's profile picture
Matteo Collina1 year ago

For this exploration, I used Claude Desktop to code a pure-typescript implementation of the JQ language. Let me introduce you to the Flowing JSON Grep Handler, or fgh:

Matteo Collina's profile picture
Matteo Collina1 year ago

So what were my findings?

Matteo Collina's profile picture
Matteo Collina1 year ago

🧠 Ultimately, AI is brilliant at grunt work, but worthless at “system thinking”. Using mcp-node to build fgh setup was fantastic for: 1. Monotonous and repetitive tasks 2. By-the-book implementations with precise requirements 3. Prototype creation & evolution

Matteo Collina's profile picture
Matteo Collina1 year ago

However, as much as I enjoyed the creation process, the code generated crumbled on itself. ✍️ I had to manually write 1318 of the 7587 lines of code (numbers taken at the time of this writing), which was ~17% of the actual implementation.

Matteo Collina's profile picture
Matteo Collina1 year ago

⭕ Here are the critical mistakes Claude made: 1. It tended to solve failing tests by adding special cases for them in the source code 2. It failed to realize the whole architecture was wrong 3. It wrote conflicting tests

Matteo Collina's profile picture
Matteo Collina1 year ago

💭 What does this mean? We need to learn to leverage AIs - and build our agents - to optimize our development workflow. Humans must stay in the loop as the conduit and auditor of AI work, or else the final product will quickly become an unmaintainable bowl of spaghetti code.

Togoda AI Search Engine's profile picture
Togoda AI Search Engine1 year ago

Togoda is Google on Steroids with AI summaries . 🚀 The only thematic AI search engine.👀 It's 100% private with third party proxy. 🧨 Try it today & experience the difference! 👉Follow us @togoda_com 👈 🚀Help us grow & share this post!🚀

Lukasz's profile picture
Lukasz1 year ago

Dismissing a brilliant teenager who is learning to code due to the need of being required to fix the code the after him is silly. How did the agentic abilities of Claude 2 look like, and how do they compare to those of 3.7? The effects of habituation and desensitization are strong.

Patrick the AI Engineer's profile picture
Patrick the AI Engineer1 year ago

@AnthropicAI Brilliant read-up and I can't agree more: AI is a tool that assists you, but should never take over completely. LLMs lack nuance and experience that a human engineer will acquire over the years.

Related Videos