Mechanize
@MechanizeWork • 14,503 subscribers
We build environments and evals for training and evaluating frontier coding agents.
Videos
Our new podcast on evals, with Max Niederman, Ege Erdil, and Stephen Yang. 0:00:00 – What's an eval, and how's it different from an RL environment? 0:19:33 – Why are models bad at building an emulator when the task is fully verifiable? 0:42:00 – How does training on bad data teach models to write terrible code? 1:04:00 – Why is continual learning still so bad? 2:25:24 – Why haven't software engineers been replaced when coding is basically solved? Listen to the Mechanize Podcast on YouTube, Spotify, etc. Enjoy!
Mechanize41,184 views • 4 months ago
Dario claims that AI will write basically all code, and that within a year thereafter a large share of software engineering jobs will be lost. While his prediction about the automation of coding is right, the conclusion about jobs does not follow. Coding models are differentially automating coding. But there are many more things that matter, such as making good design decisions and understanding what users want. Progress on these is much slower. As coding is automated, engineering effort is reallocated toward these other things, which raises the overall output of the industry. It also raises the marginal value of the time engineers spend creating software.
Mechanize21,141 views • 4 months ago
No more content to load