Nathan Odle's banner

Nathan Odle

@mov_axbx • 6,874 subscribers

I drive rally cars, do computer things, and engineer stuff. pronounced “oh, dull”.

Shorts

Llama.cpp running on a Sun Ultra 45 from 2006. Dual 1.6GHz UltraSPARC IIIs kick out ~0.7 tok/s on Gemma 3 270M Q8_0. Haven’t beat the compiler’s scalar code yet as the UltraSPARC’s VIS vector stuff isn’t ideal for int8 dotprod. May find some perf with Q4_0 nibble unpack. Thanks to Georgi Gerganov for a pretty portable codebase and for whomever saved me the trouble of handling big endian with the s390x port lol

Llama.cpp running on a Sun Ultra 45 from 2006. Dual 1.6GHz UltraSPARC IIIs kick out ~0.7 tok/s on Gemma 3 270M Q8_0. Haven’t beat the compiler’s scalar code yet as the UltraSPARC’s VIS vector stuff isn’t ideal for int8 dotprod. May find some perf with Q4_0 nibble unpack. Thanks to Georgi Gerganov for a pretty portable codebase and for whomever saved me the trouble of handling big endian with the s390x port lol

70,442 Aufrufe

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

38,079 Aufrufe • vor 1 Monat

Took a little break from AI stuff to do this radio project. Freshly written FT8 implementation for HF digital comms, with a nice web interface. RX audio piped directly over LAN from my rackmount FLEX-6600 HF SDR to FT8 service running on linux box. TX functionality incoming.

Took a little break from AI stuff to do this radio project. Freshly written FT8 implementation for HF digital comms, with a nice web interface. RX audio piped directly over LAN from my rackmount FLEX-6600 HF SDR to FT8 service running on linux box. TX functionality incoming.

40,366 Aufrufe • vor 3 Monaten

Please enjoy this SGI Indigo2 workstation from 1996 running llama2.c by Andrej Karpathy. 1.4 tokens/sec with the 15M TinyStories model! Just a little porting for the big endian IRIX machine, all in an evening’s work. What would ML researchers have thought 28 years ago seeing this?!

Please enjoy this SGI Indigo2 workstation from 1996 running llama2.c by Andrej Karpathy. 1.4 tokens/sec with the 15M TinyStories model! Just a little porting for the big endian IRIX machine, all in an evening’s work. What would ML researchers have thought 28 years ago seeing this?!

117,793 Aufrufe • vor 2 Jahren

Keine weiteren Inhalte verfügbar