跳过导航
fastest way to move llms between machines for distributed inference
0:00 / 0:42

fastest way to move llms between machines for distributed inference

Alex Ziskind

@digitalix2.8万 位关注者

在 X 上查看
1,170 个赞下载原帖

74,754 次观看

fastest way to move llms between machines for distributed inference

33 条评论

评论来自 X (Twitter) 上的原帖

  • Mohand 的头像

    @mohandsdl

    Maybe it’s a stupid question but do you need to have the models on all four machines in order to distribute the computing?

    6 个赞

  • Alex Ziskind 的头像

    @digitalix

    yes

    6 个赞

  • Wihl Silent H 🇵🇷🇺🇸 的头像

    @WihlSilentH

    I remember moving a megabyte or two file onto a floppy 💾 disk and waiting. Or burning a few hundred megabytes on a CD 💿 ROM. Now we’re doing hundreds of Gigabytes in a minute lol

    5 个赞

  • Dreams of Code 的头像

    @dreamsofcode_io

    HAYAI!

    4 个赞

  • Tom Osman 🐦‍⬛ 的头像

    @tomosman

    bruh at least mark this NSFW

    4 个赞

  • Affinity Techworks 的头像

    @AffinityTechWx

    TB5: 374 GB / 50 s ≈ 7.5 GB/s (~60 Gbps) over Finder's SMB 3.x. rsync/rclone or NFSv4 + jumbo frames add ~30%, pushing closer to ~80 Gbps. 10GbE: NVMe write-back cache on LVM RAID-5 NAS → TB-scale, NVMe-speed, low-cost model loads at a fraction of PCIe 5.0 NVMe cost #MLOps

    3 个赞

  • Henk Poley 的头像

    @HenkPoley

    Or I guess, plug that Thunderbolt 5 cable into the other Mac directly.

    2 个赞

  • Be3b 的头像

    @Be3bZilla

    Awesome👌🏾

    2 个赞

  • Zach Mueller 的头像

    @TheZachMueller

    Good man. Now slap 2 on there and use RAID 😈

    2 个赞

  • SirJaɱzAlot 的头像

    @MgkMshrmBrkfst

    'move' glm4.7? just delete it

    2 个赞

  • Traceix 的头像

    @usetraceix

    Have you never heard of rsync?

    2 个赞

  • Intrepid Sponge 𐇐𐇑𐇒𐇓𐇔𐇕𐇖𐇗𐇘𐇙𐇚 的头像

    @intspg

    Insane how far we’ve come is there a moored law for file transfer speed 😂

    2 个赞

  • Yeab - engg / core was in Palo Alto🌴 的头像

    @yeabu369

    cool, can you send the the 3 other hardware to my addy when you're done hah

    1 个赞

  • Sam Figueroa 的头像

    @unimatrixZxero

    This is my sign to finally invest in TB5 enclosures

    1 个赞

  • Kenrik March 的头像

    @KenrikMarch

    Very nice 👍

    1 个赞

  • Kiran Adimatyam 的头像

    @kiranadimatyam

    This is freaking awesome.

    1 个赞

  • Doge Codes 的头像

    @Doge_Codes

    @Stocktrader nice tip

    1 个赞

  • Sir Nullptr 的头像

    @SirNullptr

    heads up buddy

    1 个赞

  • Simple Guy 的头像

    @realGuySimple

    You gonna use exo?

  • Simulated Requirements 的头像

    @EntropicIrony

    Does this model interface well with openclaw? Wondering if the local models really can replace the api calls