Loading video...

Video Failed to Load

Go Home

1-bit inference of 0.8m param gpt running inside 8192 bytes of sram

monty10x's profile picture

Monty Anderson

1,997 subscribers

66,849 views • 4 months ago •via X (Twitter)

GamingArtsScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

imagine dropping a bit in a pond of bytes #p5js
0:22

imagine dropping a bit in a pond of bytes #p5js

Prathamesh

11,036 views • 1 month ago

ITS 1AM. I GOT SERIAL PROGRAMMING WORKING ON THE 1 BIT COMPUTER!!!!!! THIS IS __AWESOME__!!!!! (ugh, so much fight) now I can *just* upload programs to it!!! Its running off an SRAM chip!!!
1:12

ITS 1AM. I GOT SERIAL PROGRAMMING WORKING ON THE 1 BIT COMPUTER!!!!!! THIS IS __AWESOME__!!!!! (ugh, so much fight) now I can *just* upload programs to it!!! Its running off an SRAM chip!!!

Rue Mohr🇨🇦

26,232 views • 2 years ago

The 2023 edition of my long-running anti-establishment art-science-fusion code-therapy smooth-baritone causal inference & Bayesian data analysis course is complete. 20 lectures, from the basics of causal inference & Bayesian updating to mixed models & Gaussian processes. 1/2
1:03

The 2023 edition of my long-running anti-establishment art-science-fusion code-therapy smooth-baritone causal inference & Bayesian data analysis course is complete. 20 lectures, from the basics of causal inference & Bayesian updating to mixed models & Gaussian processes. 1/2

Richard McElreath 🐈‍⬛

524,285 views • 3 years ago

Breaking: We can now prove that AI is correct. DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵
0:22

Breaking: We can now prove that AI is correct. DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵

LAGRANGE

249,324 views • 1 year ago

South Park inadvertently running of the best organic ads ever for Chat GPT
1:00

South Park inadvertently running of the best organic ads ever for Chat GPT

Chris Josephs

174,487 views • 3 years ago

Llama 2 (7B, 13B and 70B) is now available in Lit-GPT! 🚀 Try running inference on the chat model or finetune the foundation model ➡️ #Llama2 #MachineLearning #opensource
0:29

Llama 2 (7B, 13B and 70B) is now available in Lit-GPT! 🚀 Try running inference on the chat model or finetune the foundation model ➡️ #Llama2 #MachineLearning #opensource

Lightning AI ⚡️

17,846 views • 3 years ago

Kai-Fu Lee says GPU supply constraints are forcing Chinese AI companies to innovate, meaning they can train a frontier model for $3 million contrasted with GPT-5's $1 billion, and deliver inference costs of 10c/million tokens, 1/30th of what an American company charges
1:45

Kai-Fu Lee says GPU supply constraints are forcing Chinese AI companies to innovate, meaning they can train a frontier model for $3 million contrasted with GPT-5's $1 billion, and deliver inference costs of 10c/million tokens, 1/30th of what an American company charges

Tsarathustra

218,794 views • 1 year ago

[ Truth & Inference ] (1/2) Here are different videos of players from the Chinese server giving a showcase of the Prospector's Truth & Inference costume, accessory, and emote. ※ Video 1: ※ Video 2: ※ Video 3: ※ Video 4:
4:24

[ Truth & Inference ] (1/2) Here are different videos of players from the Chinese server giving a showcase of the Prospector's Truth & Inference costume, accessory, and emote. ※ Video 1: ※ Video 2: ※ Video 3: ※ Video 4:

Identity V | News

86,695 views • 11 months ago

🌸 Running Crew, looking for members 🌸 1. Date: April 1 - 15 2. Location: Pancake Tower's hottest running track 3. Needed: Just you! 4. Requirements: A tiny bit of courage! 5. Fee: 1 Like, 1 Sub, 1 Reply #CookieRun #TowerOfAdventures #Update #AprilFools #OvenBreak
1:22

🌸 Running Crew, looking for members 🌸 1. Date: April 1 - 15 2. Location: Pancake Tower's hottest running track 3. Needed: Just you! 4. Requirements: A tiny bit of courage! 5. Fee: 1 Like, 1 Sub, 1 Reply #CookieRun #TowerOfAdventures #Update #AprilFools #OvenBreak

CookieRun: Tower of Adventures

43,811 views • 5 months ago

Why are you paying GPT 5.5 prices for requests that don't need GPT 5.5? I'm tired of waiting... One line of code. Every request goes to a live auction between providers. Cheapest reliable one wins. You get a normal response back, like nothing changed. Switchboard. AI inference as a market.
1:13

Why are you paying GPT 5.5 prices for requests that don't need GPT 5.5? I'm tired of waiting... One line of code. Every request goes to a live auction between providers. Cheapest reliable one wins. You get a normal response back, like nothing changed. Switchboard. AI inference as a market.

Matthew Korshunov

16,088 views • 2 months ago

GPTs are generating sequences in a left-to-right order. Is there another way? With François Fleuret and @evanncourdier, in partnership with SkySoft-ATM, we developed σ-GPT, capable of generating sequences in any order chosen dynamically at inference time. 1/6
0:38

GPTs are generating sequences in a left-to-right order. Is there another way? With François Fleuret and @evanncourdier, in partnership with SkySoft-ATM, we developed σ-GPT, capable of generating sequences in any order chosen dynamically at inference time. 1/6

Arnaud Pannatier

380,009 views • 2 years ago

Skate Bytes Be the CEO of a market making desk, run your company of trading agents
24:25

Skate Bytes Be the CEO of a market making desk, run your company of trading agents

Skate

21,817 views • 1 month ago

OpenAI CEO, Sam Altman: Someday, we'll make a powerful consumer device that is capable of running a GPT-5 or GPT-6-level model entirely locally, using very little power
0:32

OpenAI CEO, Sam Altman: Someday, we'll make a powerful consumer device that is capable of running a GPT-5 or GPT-6-level model entirely locally, using very little power

Haider.

186,781 views • 10 months ago

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.
0:32

I can't believe this works, but I got DeepSeek-V4-Flash (284B params) running on a Raspberry Pi 5 (8GB edition) at >1tok/s @ ~8W during full-tilt inference! It uses an untouched copy of antirez's GGUF. Took 160+ experiments over 5 days between GPT-5.5 xhigh and Opus 4.8 max.

Dan Woods

185,249 views • 3 months ago

1-bit Bonsai 8B running locally on an M4 Pro (MLX) alongside a standard 16-bit 8B model. Same class of model, very different deployment profile: far lower memory use and substantially higher throughput.
0:32

1-bit Bonsai 8B running locally on an M4 Pro (MLX) alongside a standard 16-bit 8B model. Same class of model, very different deployment profile: far lower memory use and substantially higher throughput.

PrismML

93,021 views • 5 months ago