Загрузка видео...

Не удалось загрузить видео

На главную

I put an entire codebase in a text file and gave it to Gemini 1.5 Pro. I gave it a task that required updating ~10 files across the entire codebase - so decently complex. It got about 80% of the work done for me on a 1st attempt. Watch...

85,724 просмотров • 2 лет назад •via X (Twitter)

Комментарии: 9

Фото профиля Mckay Wrigley
Mckay Wrigley2 лет назад

This is a very realistic example. Highlights how nice the context window is for code-related work while being honest about LLMs needing better reasoning capabilities to get to 100%. Super happy with the result! Lots of gains to be had from prompting/retrieval on top of this.

Фото профиля FUTURESaiLOR ⚓ 🌊
FUTURESaiLOR ⚓ 🌊2 лет назад

This looks interesting:

Фото профиля Mckay Wrigley
Mckay Wrigley2 лет назад

If they actually have a reasoning breakthrough then wow

Фото профиля Tom Bielecki
Tom Bielecki2 лет назад

Gemini context + Cursor RAG + Agentic testing loops

Фото профиля Giordano Sugar
Giordano Sugar2 лет назад

Thank you so much for this in depth example and your full, honest explanation. Really gives me a full sense of what to expect.

Фото профиля Tribhuvan Joshi
Tribhuvan Joshi2 лет назад

This is quite amazing. Eagerly waiting for its public release.

Фото профиля efwerr
efwerr2 лет назад

thanks for the honest evaluation

Фото профиля Andrew!
Andrew!2 лет назад

Did you just copy and paste your codebase into the txt file? Or a script or something

Фото профиля Mckay Wrigley
Mckay Wrigley2 лет назад

@AndrewW42079596 py script

Похожие видео

i watched gemma 4 12b build something genuinely impressive today, and then loop itself to death right in front of me. the full run is in the video, sped up but completely uncut, watch it to the end and you will catch the exact moment it stops building and starts looping right in the middle of the work. the task was clean, build a single file gravity simulator, n-body physics, orbits, collisions, running locally on one 3090 through an agent. and for ten minutes it was a joy to watch. it reached for a symplectic integrator on its own, the correct one, the kind that keeps orbits stable instead of spiralling out. real gravity with softening, proper orbital velocities, momentum conserved on collision. the physics was right. the thing actually worked. then on the very last step, writing a few tests to prove its own code, it fell into a loop. not a crash, a loop. it started repeating itself and would not stop. ten more minutes, thirty four thousand tokens into a single answer, the same fragments over and over, until i killed it myself. so it's not that gemma can't code. it did the hard part beautifully. it cannot finish. it cannot hold a long task together without unravelling, and finishing is the entire job in agentic work. here's the part that stings. i run this exact task, same harness, same card, on the chinese open models, qwen especially, and i never see this. they build it, they test it, they stop. every single time. google has the raw capability, you can see it sitting right there in the code, and then the model loops itself to death on a task a 27b from alibaba finishes clean. open weights, apache 2.0, so much to love on paper. i just need it to know when to stop talking.

Sudo su

39,574 просмотров • 2 месяцев назад