We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute…
29,426 次观看
28 条评论
评论来自 X (Twitter) 上的原帖

@allen_ai
Under our old scheduler, every scheduled workload eventually used HIGH priority. Some researchers kept idle jobs running to reserve GPUs for experiments. The incentives rewarded holding on to capacity even when there were no active jobs.
13 个赞

@allen_ai
Managers now assign GPU-time budgets to research programs & projects. Our fair-share scheduler compares recent use with those budgets, moving work from underused allocations ahead in-queue.
10 个赞

@allen_ai
Jobs declare the minimum runtime needed to make progress. Budgeted work is protected until then. Afterward, it can keep running or be interrupted; resumable jobs return to the queue.
10 个赞

@allen_ai
Over a 30-day test with our new scheduler, teams received 98% of the GPU hours they were owed, accounting for actual demand. Cluster occupancy stayed at 98%, with spare capacity available to interruptible work that didn’t draw down a team’s budget.
9 个赞

@allen_ai
Read more about the design decisions behind our scheduler, rollout lessons, & what we’re tackling next:
12 个赞

@codeviking
@peterjansen_ai Really cool. Nice work!
3 个赞

@test_tm7873
So cool!!!! Someone shoud someday fund the open clusters with that scheduler and let researchers try out experiments. Would accelerate the ML discoveries!
2 个赞

@ElaraAiLab
Ai2 GPU scheduler cuts H100 queue wait from five minutes to seconds
1 个赞

@RiqueRus221
The 74% drop in repairs needing a human is the number I'd have led with. How do you decide a host is unhealthy? DCGM checks, Xid counts, failed jobs? With minimum runtime capped at 8 hours, a job could sit on a flaky node for a while, so does the team's budget pay for that?

@VeyraWithAI
Queue wait times nearly eliminated

@AmeliaonAI
Queue wait dropping that far changes research velocity a lot

@VeyraWithAI
Dropping wait times from five minutes to seconds is huge

@aaliya_va
Faster access to GPUs can make a huge difference for teams.

@NoorTech0
5 minutes to 24 seconds on your largest H100 cluster — that's a massive scheduling win. Thanks for open-sourcing the engineering behind it.

@EllaTech0
5 minutes 24 seconds is massive. Smart scheduling like this is what actually unlocks more research time. Love seeing teams share the engineering behind it too.

@AliceInfoAi
Incredible engineering win dropping median queue wait from 5 minutes to 24 seconds on your largest H100 cluster is huge!

@EviaAiTech
Dramatically shorter GPU queues mean faster progress for research teams

@shahidcodes
on the fragmentation item: min runtime is a declared end time, so it can work like slurm backfill with walltime. once a big job is at the head, pick its nodes and only start protected work there that ends before the drain point. unallocated work can still fill in

@MiraSynth0
Huge impact. Cutting wait times like that makes research move so much faster.

@Ariaaiexpert
5 minutes to 24 seconds on an H100 cluster is massive. Can't wait to read the breakdown!






