正在加载视频...

视频加载失败

Demo for the sharded inference engine on: gpt-oss-120b on 4090's, max input context 95k, max output context infinite (tok/s degrades with context) queue of max 12 prompts concurrently, use it wisely

91,357 次观看 • 4 个月前 •via X (Twitter)

37 条评论

leyten 的头像
leyten4 个月前

for those who doubt: IP's of the nodes are verifiable code for this demo is found in the github:

Moontime 的头像
Moontime4 个月前

Running a 120B model split across 4090s over WAN with verifiable receipts and no central server is genuinely unhinged infrastructure. This is what decentralized compute was supposed to mean before everyone got distracted making JPEG coins.

nothingeverhappens 的头像
nothingeverhappens4 个月前

@c0mputeAI bro’s hard at work

leyten 的头像
leyten4 个月前

@mehdixxxz @c0mputeAI can't change the DeAI space by doing nothing all day

spyzer 的头像
spyzer4 个月前

Insane 👏

mohit 的头像
mohit4 个月前

this is so fucking cool

leyten 的头像
leyten4 个月前

we're pioneers mohit

Naka 的头像
Naka4 个月前

Homie grinding hard 24/7

jaylow 的头像
jaylow4 个月前

@toly @mert @solana where are you??

Aliaian 的头像
Aliaian4 个月前

a ture pioneer, this is sick @leyten doesnt stop building

wawin 的头像
wawin4 个月前

@toly @mert @OpenAI Check whats building here

Moontime 的头像
Moontime4 个月前

infinite output context is doing a lot of heavy lifting in that sentence. 'tok/s degrades with context' is just 'technically infinite if you have three weeks.'

spdy 的头像
spdy4 个月前

this is very cool i wish you would post on the main account, low visibility here :( very cool site and i liked how it answered what i asked wp dev

uncracked 的头像
uncracked4 个月前

Bro casually making the next frontier of DeAI a reality

taku 的头像
taku4 个月前

few questions why only 4 workers are participating? website says 300+ and who is the user here? website says 0 busy is this a separate experimental demo from what is live currently?

Tz 的头像
Tz4 个月前

Good work sir, this could be a real solution to the crazy high server costs in AI inference

0xTereza 的头像
0xTereza4 个月前

@shawmakesmagic u might like this one

Vip 的头像
Vip4 个月前

Can you add a chat history rq. shit is cooking bangers

internet nobody 🇪🇸 的头像
internet nobody 🇪🇸4 个月前

Probably nothing...

FEiKU 的头像
FEiKU4 个月前

Cope and seethe haters! Builders just keep building until there’s no more fud left 🫡

Real ⚡️ 的头像
Real ⚡️4 个月前

people aren’t yet ready or can imagine

Enkidu 的头像
Enkidu4 个月前

This tech is insane! Major props to the dev @leyten @solana @toly

SVR 的头像
SVR4 个月前

very easy to see

AdventureNLearn 的头像
AdventureNLearn4 个月前

⛓️‍💥

胡萝卜丨 1000XGEM 的头像
胡萝卜丨 1000XGEM4 个月前

这是证明自己?

leyten 的头像
leyten4 个月前

by the nodes scattered across the USA

Chrissy from the BLOCKS 🐂🀄️ 的头像
Chrissy from the BLOCKS 🐂🀄️4 个月前

@c0mputeAI @pmarca @elonmusk @TheRundownAI @rowancheung @mreflow @karpathy @ylecun @sama @emollick @demishassabis @toly @solana @blknoiz06 New AI tech and dev on the solana:So11111111111111111111111111111111111111112 scene the top dogs need to know asap $zero @c0mputeAI

WhiteSugar59 🐂 的头像
WhiteSugar59 🐂4 个月前

nice, keep it up. Do you have a timeline for progression of full scale utility from this demo. what will fully working setup look like

Max 的头像
Max4 个月前

amazing work

ahmet cetinkaya 的头像
ahmet cetinkaya4 个月前

👏

Mr.Will 的头像
Mr.Will4 个月前

Respect😍

Ferbin 的头像
Ferbin4 个月前

12 concurrent is tight. What's latency under full queue for realtime voice?

Barlo 的头像
Barlo4 个月前

@c0mputeAI @toly @NVIDIAAI @nvidia

Lesnar XBT 的头像
Lesnar XBT4 个月前

W

NeverQuit 的头像
NeverQuit4 个月前

🔥🔥

F 的头像
F4 个月前

🌶️🌶️

Todd Zheng 的头像
Todd Zheng4 个月前

remarkable work. Can’t wait till the day when people can use your tech to form p2p network and both providers and consumers can benefit from it. Right now it’s still not cost-effective enough though but I truly admire this work!

相关视频