Samuel Cardillo's banner
Samuel Cardillo's profile picture

Samuel Cardillo

@CardilloSamuel • 44,222 subscribers

Leveraging technologies to execute visions | OSS, Local AI & LLMs | Current @neutralityorg @sbreakintl 🛰 | Former @RTFKT @Nike @Google | Opinions are my own

Shorts

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.

i just tested to put GLM-5.2 on my rig. 753B parameter MoE. 2x RTX PRO 6000 Blackwells, Threadripper PRO 9995WX with 1TB DDR5. prefills at 64 tok/s. decode holds at 13-15. system RAM bandwidth is the bottleneck. running UD-Q4_K_XL 4-bit. 69.7% on Aider Polyglot, within two points of BF16. left the 5090 off. mismatched VRAM unbalances the split. this is what local AI looks like now. massive frontier models on a "desktop rig". how awesome is this? and this is the worst it'll ever be.

48,640 просмотров

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

direct kinetic impact. a flying sword. 450km/h. updated video showing exactly that. we're also working on the explosive variant. only for authorized partners. dms are open.
0:59
CardilloSamuel's profile picture

direct kinetic impact. a flying sword. 450km/h. updated video showing exactly that. we're also working on the explosive variant. only for authorized partners. dms are open.

Samuel Cardillo

1,279,155 просмотров • 4 месяцев назад

while lot of people are now waking up to the needs of buying hardware, i had started ~6 months ago already, slowly building the machine currently powering my devs, experiments, ... 2* 6000 pro blackwell, a 5090, a dgx spark ,a 9995 WX threadripper and 1 TB of RAM DDR5 ECC. this is currently one of my asset which literally appreciated the most, i probably doubled my money since then - and also, through contracts i got, it paid itself back. a dgx spark is there too. i am thinking of adding one more. and obviously, everything is safely kept alive with SRT 3000VA srt3000xli UPS from APC. my next investment is probably going to be a NAS to store as much weights as i can. but all of this was the 1st few steps. more important is coming. just stay tuned!
0:29
CardilloSamuel's profile picture

while lot of people are now waking up to the needs of buying hardware, i had started ~6 months ago already, slowly building the machine currently powering my devs, experiments, ... 2* 6000 pro blackwell, a 5090, a dgx spark ,a 9995 WX threadripper and 1 TB of RAM DDR5 ECC. this is currently one of my asset which literally appreciated the most, i probably doubled my money since then - and also, through contracts i got, it paid itself back. a dgx spark is there too. i am thinking of adding one more. and obviously, everything is safely kept alive with SRT 3000VA srt3000xli UPS from APC. my next investment is probably going to be a NAS to store as much weights as i can. but all of this was the 1st few steps. more important is coming. just stay tuned!

Samuel Cardillo

18,085 просмотров • 20 дней назад

I am extremely happy to be joining Verse World as an advisor for the gaming side. This is an entirely new non-technical adventure that starts where instead of using my coding knowledge, I'll use my gaming one.
0:33
CardilloSamuel's profile picture

I am extremely happy to be joining Verse World as an advisor for the gaming side. This is an entirely new non-technical adventure that starts where instead of using my coding knowledge, I'll use my gaming one.

Samuel Cardillo

56,563 просмотров • 1 год назад

i've just turned 30 in my current timezone. therefore, i am giving a gift to all clonex holders. no blur this time. today is the "official" release of the alpha version of storymode DLC. while this version is stripped from all AI integrations (for multiple reasons which i wont get into now), it integrate a ready-to-use by all assembly line. you can now just connect, select your clone, have it assembled it directly and download it as a GLB. community is more than welcome to improve the interface of storymode to make the viewer better, maybe even add a VRM exporter? the API to get the traits + files is always available for all with a documentation enjoy (and happy birthday to me) Github : Link : special thanks to beb.tools Samsy 👾🥷🏼 Ash CreaBilly.ETH for the help 🥰 PS : the 3d models aren't (at all) optimized so they might be slow to load. sorry, i have no 3d talents.
0:22
CardilloSamuel's profile picture

i've just turned 30 in my current timezone. therefore, i am giving a gift to all clonex holders. no blur this time. today is the "official" release of the alpha version of storymode DLC. while this version is stripped from all AI integrations (for multiple reasons which i wont get into now), it integrate a ready-to-use by all assembly line. you can now just connect, select your clone, have it assembled it directly and download it as a GLB. community is more than welcome to improve the interface of storymode to make the viewer better, maybe even add a VRM exporter? the API to get the traits + files is always available for all with a documentation enjoy (and happy birthday to me) Github : Link : special thanks to beb.tools Samsy 👾🥷🏼 Ash CreaBilly.ETH for the help 🥰 PS : the 3d models aren't (at all) optimized so they might be slow to load. sorry, i have no 3d talents.

Samuel Cardillo

23,054 просмотров • 10 месяцев назад

Больше нет контента для загрузки