Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task... categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵show more

[2/n] EdgeBench covers real work across six capability families. It includes 134 day-long tasks spanning scientific and ML problems, systems and software engineering, optimization, professional knowledge work, formal math, and interactive games. Each task gives agents at least 12 hours in an executable environment with informative real-world feedback, while recorded human expert effort averages 57.2 hours per task.

[3/n] Agents continuously learn from environments and improve their performance in EdgeBench. The representative curves below, drawn from all six capability families, show that agents continually turn environmental feedback into better artifacts, strategies, and final outcomes.

[4/n] We average learning curves of different models across 134 tasks. The noisy task-specific trajectories collapse into a simple log-sigmoid, with high precision and mean R^2=0.998.

[5/n] Here's a fun theory of learning from environments: a task's score consists of many tiny units, sitting on a graph. Learning advances like a frontier: each node unlocked reveals new unseen neighbors, so progress compounds; but the shrinking pool of unexplored nodes sets the limit. The pace of the frontier depends on both. In fact, this heuristic admits a formal proof. Let the fraction of explored nodes be x, so the unexplored fraction is (1 − x). Our theory says the frontier expands at a rate proportional to the product x(1 − x). Taking the natural time scale to be u = log(t), the differential equation dx/du = c·x(1 − x) solves to exactly the log-sigmoid law. More importantly, we prove that even if each individual task's progress is jagged, the log-sigmoid law still emerges in the benchmark average over many tasks. This gives an elegant explanation of the phenomenon we observed.

[6/n] We evaluated model releases from September 2025 to May 2026, using performance improvement within 2 hours as the learning-speed metric. The frontier trend shows that AI learning speed from environments roughly doubles every three months.

[7/n] Below we show a 12-hour agent performance trace on the gravitational-wave task. Across 247 scored attempts, the performance climbs from 42.8 to 67.0, with seven turning points where the agent reframes the problem rather than just tuning.

[8/n] Here’s a time-aware leaderboard showing best-so-far performance at 2h, 6h, and 12h.

[9/n] EdgeBench was an awesome team effort. Huge thanks to the incredible team💪: @mingwuzheng @zinuxo87 @odysseusqs @zhu_xuekai @_foreverpiano @Zixin_Wen Zhonglin Xie @poppingG5

Will you keep the leaderboard updated with new models like GLM-5.2 for example?

Fun fact: no single task follows the law. Zoom into any one curve — pure chaos. The clean sigmoid only emerges when you average across 134 tasks, exactly as the graph-exploration theory predicts.

great stuff

Super cool. Reminds me of:

GLM-5.1 looks very interesting. Highest S max of them all but much shallower slope. Any ideas what that model does differently? It's like you let it churn for long enough, it eventually organizes itself to the point where it beats everything else out there?

How do humans compare? That'd be really interesting! Adding smart and lower intelligence animals like Octopus, Birds, Ants etc would really be enlightening. The questions needs to be reformulated in order for animals to even partake and the AI models be run again I guess to have a fair baseline

@tszzl Banger

crazy name

I have a few questions. My current understanding is that for each task in the benchmark, you've already created a verifiable dataset to evaluate against. The agent doesn't see the ground truth, but it does see its score. At each evaluation step, I'm not sure: is it the model itself (with the context it's built up) doing the task directly, or is it the program that the agent has created being evaluated? Thank you.

interesting! i always wondered about agent hillclimbing on long term tasks

Well I have never been a believer of "AI replacing humans". This benchmark probably justifies this ideology I guess

my take is that as AI moves beyond static knowledge retrieval and into dynamic, real-world execution—a shift highlighted by recent benchmarks like EdgeBench—it becomes clear that simply expanding context windows or providing search tools is insufficient for complex tasks! in a recent essay, I explore the concept of "Machine Studying," arguing that giving an agent access to information is not the same as the agent developing true expertise. when faced with entirely novel domains without explicit reward functions, an agent must rely on its pre-trained priors to self-direct, abstract, and essentially "study" the material to prepare for unknown future objectives. this demands a shift from measuring an AI's ability to find data toward measuring its capacity to recognize significance and convert raw exposure into durable competence. you can read the full analysis here: Access Is Not Expertise: Machine Studying and the Limits of Search on @Substack - !

Congrats team, awesome work!

Congrats team, awesome work!!!

great works😀

Must be lots of efforts for curating data. Thanks for your amazing work!

Thank you for! A quick side look on the way the studying progress is presented finds me very insightful!

38,000 hours of agent runs to find a clean scaling law is a serious amount of empirical work.

This work is really significant for long-horizon agent research, and I’ve learned a lot from EdgeBench. The log-sigmoid scaling law discovered from extensive runtime data is quite intriguing. I will read the paper and blog for more details.

Very interesting research! Looking forward to more results!

很难想象，五年前，我们还在亲身参与算法竞赛，寻找快速提升自己实力的方法，现在，我们就在研究如何让AI智能体快速提升算法效率。

any plans to release evald trajectories on the public set? even just for one task would be really useful!

Really valuable work! Curious which models will be evaluated on this benchmark next.

wow

really great work

@tszzl thanks for the benchmark!!

@tikgiau The 12-72h part is the right stress test. I’d read EdgeBench less as “who wins at 12h?” and more as “what did the agent actually learn per hour?” If extra runtime only teaches it hidden-judge quirks, that is not experience.

I am curious how stable is the fit across your 6 task categories, or does it wash out on the messy ones? Also 134 tasks × 57 human-hours each has to make this one of the most expensive benchmarks ever built.

meh looks just like avg reward over time.... wait....

the log-sigmoid fit is neat but i wonder how robust it is across different model families. the fact that individual tasks are chaotic but average smooth is a nice sanity check.

12-72h hides restart fragility, does edgebench score recovery after tool crashes

WHen will you test Fable 5 and 5.6 Sol?

@arankomatsuzaki Hi, I noticed you reposted ZCode for GLM-5.2. We provide multi-model Chinese LLM harness transfer service, support GLM, Qwen, DeepSeek at the same time, BYOK compatible, up to 20000 RPM and 10G bandwidth, no strict content restriction. If you need test quota and price sheet

edgebench: can you almost give an answer