Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

I put an entire codebase in a text file and gave it to Gemini 1.5 Pro. I gave it a task that required updating ~10 files across the entire codebase - so decently complex. It got about 80% of the work done for me on a 1st attempt. Watch...

85,724 görüntüleme • 2 yıl önce •via X (Twitter)

9 Yorum

Mckay Wrigley profil fotoğrafı
Mckay Wrigley2 yıl önce

This is a very realistic example. Highlights how nice the context window is for code-related work while being honest about LLMs needing better reasoning capabilities to get to 100%. Super happy with the result! Lots of gains to be had from prompting/retrieval on top of this.

FUTURESaiLOR ⚓ 🌊 profil fotoğrafı
FUTURESaiLOR ⚓ 🌊2 yıl önce

This looks interesting:

Mckay Wrigley profil fotoğrafı
Mckay Wrigley2 yıl önce

If they actually have a reasoning breakthrough then wow

Tom Bielecki profil fotoğrafı
Tom Bielecki2 yıl önce

Gemini context + Cursor RAG + Agentic testing loops

Giordano Sugar profil fotoğrafı
Giordano Sugar2 yıl önce

Thank you so much for this in depth example and your full, honest explanation. Really gives me a full sense of what to expect.

Tribhuvan Joshi profil fotoğrafı
Tribhuvan Joshi2 yıl önce

This is quite amazing. Eagerly waiting for its public release.

efwerr profil fotoğrafı
efwerr2 yıl önce

thanks for the honest evaluation

Andrew! profil fotoğrafı
Andrew!2 yıl önce

Did you just copy and paste your codebase into the txt file? Or a script or something

Mckay Wrigley profil fotoğrafı
Mckay Wrigley2 yıl önce

@AndrewW42079596 py script

Benzer Videolar

i watched gemma 4 12b build something genuinely impressive today, and then loop itself to death right in front of me. the full run is in the video, sped up but completely uncut, watch it to the end and you will catch the exact moment it stops building and starts looping right in the middle of the work. the task was clean, build a single file gravity simulator, n-body physics, orbits, collisions, running locally on one 3090 through an agent. and for ten minutes it was a joy to watch. it reached for a symplectic integrator on its own, the correct one, the kind that keeps orbits stable instead of spiralling out. real gravity with softening, proper orbital velocities, momentum conserved on collision. the physics was right. the thing actually worked. then on the very last step, writing a few tests to prove its own code, it fell into a loop. not a crash, a loop. it started repeating itself and would not stop. ten more minutes, thirty four thousand tokens into a single answer, the same fragments over and over, until i killed it myself. so it's not that gemma can't code. it did the hard part beautifully. it cannot finish. it cannot hold a long task together without unravelling, and finishing is the entire job in agentic work. here's the part that stings. i run this exact task, same harness, same card, on the chinese open models, qwen especially, and i never see this. they build it, they test it, they stop. every single time. google has the raw capability, you can see it sitting right there in the code, and then the model loops itself to death on a task a 27b from alibaba finishes clean. open weights, apache 2.0, so much to love on paper. i just need it to know when to stop talking.

Sudo su

39,574 görüntüleme • 2 ay önce