跳过导航
We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute…
0:00 / 1:42

We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute…

Ai2

@allen_ai8.7万 位关注者

在 X 上查看
588 个赞下载原帖

29,426 次观看

We’re sharing the engineering work behind Ai2’s GPU scheduler—our new approach to allocating compute across research teams. On our largest H100 cluster, median queue wait fell from 5 minutes to 24 seconds. 🧵

28 条评论

评论来自 X (Twitter) 上的原帖

  • Ai2 的头像

    @allen_ai

    Under our old scheduler, every scheduled workload eventually used HIGH priority. Some researchers kept idle jobs running to reserve GPUs for experiments. The incentives rewarded holding on to capacity even when there were no active jobs.

    13 个赞

  • Ai2 的头像

    @allen_ai

    Managers now assign GPU-time budgets to research programs & projects. Our fair-share scheduler compares recent use with those budgets, moving work from underused allocations ahead in-queue.

    10 个赞

  • Ai2 的头像

    @allen_ai

    Jobs declare the minimum runtime needed to make progress. Budgeted work is protected until then. Afterward, it can keep running or be interrupted; resumable jobs return to the queue.

    10 个赞

  • Ai2 的头像

    @allen_ai

    Over a 30-day test with our new scheduler, teams received 98% of the GPU hours they were owed, accounting for actual demand. Cluster occupancy stayed at 98%, with spare capacity available to interruptible work that didn’t draw down a team’s budget.

    9 个赞

  • Ai2 的头像

    @allen_ai

    Read more about the design decisions behind our scheduler, rollout lessons, & what we’re tackling next:

    12 个赞

  • Sam Skjonsberg 的头像

    @codeviking

    @peterjansen_ai Really cool. Nice work!

    3 个赞

  • testtm 的头像

    @test_tm7873

    So cool!!!! Someone shoud someday fund the open clusters with that scheduler and let researchers try out experiments. Would accelerate the ML discoveries!

    2 个赞

  • Elara AI 的头像

    @ElaraAiLab

    Ai2 GPU scheduler cuts H100 queue wait from five minutes to seconds

    1 个赞

  • Henrique Silva 的头像

    @RiqueRus221

    The 74% drop in repairs needing a human is the number I'd have led with. How do you decide a host is unhealthy? DCGM checks, Xid counts, failed jobs? With minimum runtime capped at 8 hours, a job could sit on a flaky node for a while, so does the team's budget pay for that?

  • Veyra | AI & Tech 的头像

    @VeyraWithAI

    Queue wait times nearly eliminated

  • Amelia | AI & Tech 的头像

    @AmeliaonAI

    Queue wait dropping that far changes research velocity a lot

  • Veyra | AI & Tech 的头像

    @VeyraWithAI

    Dropping wait times from five minutes to seconds is huge

  • Aaliya 的头像

    @aaliya_va

    Faster access to GPUs can make a huge difference for teams.

  • NOOR TECH 的头像

    @NoorTech0

    5 minutes to 24 seconds on your largest H100 cluster — that's a massive scheduling win. Thanks for open-sourcing the engineering behind it.

  • Ella Tech & Tool 的头像

    @EllaTech0

    5 minutes 24 seconds is massive. Smart scheduling like this is what actually unlocks more research time. Love seeing teams share the engineering behind it too.

  • Alice The Ai Expert 的头像

    @AliceInfoAi

    Incredible engineering win dropping median queue wait from 5 minutes to 24 seconds on your largest H100 cluster is huge!

  • Evia AI 的头像

    @EviaAiTech

    Dramatically shorter GPU queues mean faster progress for research teams

  • Shahid🇵🇸 的头像

    @shahidcodes

    on the fragmentation item: min runtime is a declared end time, so it can work like slurm backfill with walltime. once a big job is at the head, pick its nodes and only start protected work there that ends before the drain point. unallocated work can still fill in

  • Mira Synth Tech 的头像

    @MiraSynth0

    Huge impact. Cutting wait times like that makes research move so much faster.

  • Aria Tech 的头像

    @Ariaaiexpert

    5 minutes to 24 seconds on an H100 cluster is massive. Can't wait to read the breakdown!