Загрузка видео...

Не удалось загрузить видео

На главную

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than...

noisyb0y1's profile picture

Noisy

21,532 subscribers

78,209 просмотров • 2 дней назад •via X (Twitter)

ИгрыНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less
24:17

THIS DEVELOPER CONNECTED 8 NVIDIA DGX SPARKS INTO ONE CLUSTER - AND RAN AN 800GB MODEL THAT MADE HIM 10X MORE PRODUCTIVE 21:47 he says it straight - "this is a terabyte of VRAM - we ran Quen 3.5, 800GB on disk, a model that doesn't even fit on a single Mac Studio - 24 tokens per second - I'd say that's a win" 8 Sparks connected through a $1,300 switch via RDMA over Ethernet - each node adding 128GB of memory into one unified pool of 1TB started with one Spark at 3 tokens per second - every added node doubled the speed - and eight together deliver 24 tokens on a model that physically cannot run anywhere else Kimi K2 at 600GB loaded in 15 minutes, 115GB per node, 13 tokens per second - a model that simply cannot run on anything smaller Claude helped configure the entire cluster - SSH mesh across all 8 machines, network config, jumbo frames, QSFP port speeds - all from one terminal most people rent cloud compute for models this size at $2,000+/month - he built the cluster once and now every token costs 20x less

Noisy

168,251 просмотров • 1 день назад

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."
2:14

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."

DogeDesigner

322,011 просмотров • 7 месяцев назад

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵
1:39

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

726,131 просмотров • 2 месяцев назад

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓
1:21

TWO BOXES THE SIZE OF A MAC MINI JUST RAN A 235 BILLION PARAMETER MODEL ON A DESK It is two NVIDIA DGX Spark units linked by a single cable. A year ago a model this size meant renting a GPU cluster by the hour. Now it sits next to your monitor for around $8,000. Here is the twist most people miss. Linking them does not create one shared 256GB memory pool. The model is split across both boxes, and that is the only reason a 235B model fits at all. It answers at roughly 10 tokens per second, and both chips sit at just 74 degrees while sipping around 50 watts. Every token stays on the desk. Nothing touches a cloud, and nothing leaves the room. The ceiling for what you can run at home just jumped from 70B to 235B. Bookmark this & Watch it run ↓

slash1s

99,713 просмотров • 7 дней назад

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓
0:29

4 MAC MINIS RUNNING A 671B PARAMETER MODEL AS A CLUSTER No data center, no Cloud, no expensive hardware and not a single API call.. Just 4 Mac minis connected through EXO running DeepSeek v3.1 671b locally and actually fast. The part nobody talks about is that you don’t need one monster machine, you can cluster old computers you already have and split the load between them. The full breakdown of tools that make this possible is in the article below. Save and Read it today ↓

slash1s

58,724 просмотров • 12 дней назад

671-billion-parameter DeepSeek-R1 model at up to 3,872 tokens per second
0:30

671-billion-parameter DeepSeek-R1 model at up to 3,872 tokens per second

AK

227,579 просмотров • 1 год назад

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:
0:41

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:

Awni Hannun

500,714 просмотров • 7 месяцев назад

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.
0:31

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.

Cline

21,291 просмотров • 3 месяцев назад

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).
0:50

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).

Bin Lin

27,703 просмотров • 3 месяцев назад

Jensen Huang said Grok 5 will be 7 Trillion parameter model. On time to train, with the training window fixed at 1 month, the new Vera-Rubin GPU system of Nvidia needs 1/4 the number of systems compared with Blackwell to train the same frontier model. “factory throughput” improves by about 10x over Blackwell, and Blackwell itself was about 10x over Hopper. i.e. overall Rubin is roughly 100x Hopper in factory throughput per watt, which matters hugely, because a 1 GW, $50B data center is power-limited and revenue scales with throughput per watt. --- From Nvidia YT channel
1:24

Jensen Huang said Grok 5 will be 7 Trillion parameter model. On time to train, with the training window fixed at 1 month, the new Vera-Rubin GPU system of Nvidia needs 1/4 the number of systems compared with Blackwell to train the same frontier model. “factory throughput” improves by about 10x over Blackwell, and Blackwell itself was about 10x over Hopper. i.e. overall Rubin is roughly 100x Hopper in factory throughput per watt, which matters hugely, because a 1 GW, $50B data center is power-limited and revenue scales with throughput per watt. --- From Nvidia YT channel

Rohan Paul

165,770 просмотров • 5 месяцев назад

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.
1:11

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

Dwarkesh Patel

65,123 просмотров • 2 месяцев назад

GLM5 running on a cluster of 4 Mac Studios using tensor parallelism thanks to MLX distributed. Yes, those are the Alex drawers.
0:26

GLM5 running on a cluster of 4 Mac Studios using tensor parallelism thanks to MLX distributed. Yes, those are the Alex drawers.

Alex Ziskind

20,594 просмотров • 4 месяцев назад

𝗞𝗶𝗺𝗶 𝗞𝟮.𝟲 𝗶𝘀 𝗖𝗵𝗶𝗻𝗮'𝘀 𝗻𝗲𝘄 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲 𝗮𝗻𝘀𝘄𝗲𝗿 𝘁𝗼 𝗖𝗹𝗮𝘂𝗱𝗲 𝗖𝗼𝗱𝗲. It has 1 trillion parameters but only uses 32 billion at a time. Context window holds 256,000 tokens. It scored 85% on Live Code Bench. Claude hit 64%. It plugs into OpenClaw, Cursor, and Cline with no restrictions. Kimi Claw deploys in the cloud in 2 minutes. It's way cheaper to run than Claude day to day. Save this. Your coding stack just got a real rival.
8:31

𝗞𝗶𝗺𝗶 𝗞𝟮.𝟲 𝗶𝘀 𝗖𝗵𝗶𝗻𝗮'𝘀 𝗻𝗲𝘄 𝗼𝗽𝗲𝗻 𝘀𝗼𝘂𝗿𝗰𝗲 𝗮𝗻𝘀𝘄𝗲𝗿 𝘁𝗼 𝗖𝗹𝗮𝘂𝗱𝗲 𝗖𝗼𝗱𝗲. It has 1 trillion parameters but only uses 32 billion at a time. Context window holds 256,000 tokens. It scored 85% on Live Code Bench. Claude hit 64%. It plugs into OpenClaw, Cursor, and Cline with no restrictions. Kimi Claw deploys in the cloud in 2 minutes. It's way cheaper to run than Claude day to day. Save this. Your coding stack just got a real rival.

Julian Goldie SEO

102,114 просмотров • 2 месяцев назад

Qwen 3.6 35B running at over 100 tokens per second on my $5,399 MacBook Pro M5 Max. This is the best local AI model I have ever run. 128GB of unified memory. No cloud. No API costs. No rate limits. Just raw local inference at speeds I didn't think were possible on a laptop. This model is more intelligent than GPT 5 on benchmarks. Running locally. On a MacBook. For free after the hardware cost. I said local AI would never compete with frontier. I'm starting to rethink that. The gap is closing faster than anyone expected.
1:04

Qwen 3.6 35B running at over 100 tokens per second on my $5,399 MacBook Pro M5 Max. This is the best local AI model I have ever run. 128GB of unified memory. No cloud. No API costs. No rate limits. Just raw local inference at speeds I didn't think were possible on a laptop. This model is more intelligent than GPT 5 on benchmarks. Running locally. On a MacBook. For free after the hardware cost. I said local AI would never compete with frontier. I'm starting to rethink that. The gap is closing faster than anyone expected.

BridgeMind

62,340 просмотров • 26 дней назад

1. A $34 Trillion Problem The U.S. debt just crossed $34 trillion. That’s up $11 trillion in 4 years — the fastest pace in history. Druckenmiller compared it to a “fiscal doomsday machine” — because interest costs alone are now $1 trillion per year. That’s more than the entire U.S. defense budget.
7:00

1. A $34 Trillion Problem The U.S. debt just crossed $34 trillion. That’s up $11 trillion in 4 years — the fastest pace in history. Druckenmiller compared it to a “fiscal doomsday machine” — because interest costs alone are now $1 trillion per year. That’s more than the entire U.S. defense budget.

Dividend Talks on YouTube

51,077 просмотров • 1 год назад

CHINESE AI TEACHER STACKED 4 NVIDIA DGX SPARK CHIPS INTO A CLUSTER - AND NOW RUNS A BUSINESS ON IT FROM HIS DESK 4 chips at $2,999 each - one time investment - and now he has a cluster that handles client inference requests around the clock while he does something else he's running a Qwen 32 35B model with 4 concurrent requests at the same time and the speed barely drops compared to a single request what used to require a $10,000+/month cloud setup now sits on his desk and costs him $40-60/month in electricity 4 chips at $2,999 each - one time investment - and now he has a cluster that handles client inference requests around the clock while he does something else the people still paying per token are financing someone else's infrastructure
1:42

CHINESE AI TEACHER STACKED 4 NVIDIA DGX SPARK CHIPS INTO A CLUSTER - AND NOW RUNS A BUSINESS ON IT FROM HIS DESK 4 chips at $2,999 each - one time investment - and now he has a cluster that handles client inference requests around the clock while he does something else he's running a Qwen 32 35B model with 4 concurrent requests at the same time and the speed barely drops compared to a single request what used to require a $10,000+/month cloud setup now sits on his desk and costs him $40-60/month in electricity 4 chips at $2,999 each - one time investment - and now he has a cluster that handles client inference requests around the clock while he does something else the people still paying per token are financing someone else's infrastructure

Sprytix

51,010 просмотров • 22 дней назад

DEEP SEEK R2 LEAK - READY TO EAT GPT-4 FOR BREAKFAST? Alleged leaks say DeepSeek R2 has 1.2 trillion parameters, is 10x bigger than GPT-4—and 97% cheaper to run. Yes, cheaper and smarter. Forget cat facts and casual convo—this thing was supposedly trained on 5.2 petabytes of legal, financial, and patent-grade data. Using a MoE (Mixture of Experts) setup—where only the most relevant parts of the model are activated for each task—it turns on just 78 billion of its 1.2 trillion parameters at a time, staying efficient without losing brainpower. No press release, no hype machine—just a potential industry killer quietly lurking. Source: TheAIGRID
1:44

DEEP SEEK R2 LEAK - READY TO EAT GPT-4 FOR BREAKFAST? Alleged leaks say DeepSeek R2 has 1.2 trillion parameters, is 10x bigger than GPT-4—and 97% cheaper to run. Yes, cheaper and smarter. Forget cat facts and casual convo—this thing was supposedly trained on 5.2 petabytes of legal, financial, and patent-grade data. Using a MoE (Mixture of Experts) setup—where only the most relevant parts of the model are activated for each task—it turns on just 78 billion of its 1.2 trillion parameters at a time, staying efficient without losing brainpower. No press release, no hype machine—just a potential industry killer quietly lurking. Source: TheAIGRID

Mario Nawfal

141,031 просмотров • 1 год назад

NVIDIA QUIETLY DROPPED A $249 BOX THAT REPLACES YOUR $200/MONTH OPENAI SUBSCRIPTION WITH $2 IN ELECTRICITY it's called the jetson orin nano super. smaller than a wallet, runs at 25 watts, does 70 trillion ai operations per second. runs llama 3, mistral, gemma and deepseek locally with no api fees and no data leaving your house a developer running automations and coding assistants pays $200 a month to openai. the same workload on this box costs $2 a month in electricity and breaks even in 10 weeks install ollama with one command. change one line in your code. point it at localhost instead of openai. everything else works identically 7 billion parameter models handle 80% of what people use chatgpt for. summarization, drafting, coding, document q&a, automation pipelines. total monthly cost drops from $200 to $22 cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who set this up in 2025 are going to look very smart in 2027 bookmark this and read the article below
0:37

NVIDIA QUIETLY DROPPED A $249 BOX THAT REPLACES YOUR $200/MONTH OPENAI SUBSCRIPTION WITH $2 IN ELECTRICITY it's called the jetson orin nano super. smaller than a wallet, runs at 25 watts, does 70 trillion ai operations per second. runs llama 3, mistral, gemma and deepseek locally with no api fees and no data leaving your house a developer running automations and coding assistants pays $200 a month to openai. the same workload on this box costs $2 a month in electricity and breaks even in 10 weeks install ollama with one command. change one line in your code. point it at localhost instead of openai. everything else works identically 7 billion parameter models handle 80% of what people use chatgpt for. summarization, drafting, coding, document q&a, automation pipelines. total monthly cost drops from $200 to $22 cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who set this up in 2025 are going to look very smart in 2027 bookmark this and read the article below

starmex

2,347,817 просмотров • 25 дней назад

THE CHINESE INSTALLED 6 AI AGENTS RUNNING 24/7 FOR $11 A MONTH. THE BOX UNDER ONE DESK REPLACED $1,200 IN CLOUD SUBSCRIPTIONS. The Minisforum MS-S1 Max ships with 128GB of unified memory shared between CPU and GPU. Qwen3-Coder 30B runs at 40 to 50 tokens per second. 6 agents run simultaneously without anyone watching the meter. An inbox sorter drafts replies every 15 minutes. A research monitor watches 30+ feeds and lands a digest by 7 AM, a job that costs $15 to $20 a day on cloud APIs. The same stack on cloud subscriptions ran $800 to $1,200 a month. The hardware costs $3,000 once. Break-even hits month 3 to 4. The real shift is behavior, not savings. A meeting prep agent never would have justified per-token cost. Running it free overnight made it obvious.
0:53

THE CHINESE INSTALLED 6 AI AGENTS RUNNING 24/7 FOR $11 A MONTH. THE BOX UNDER ONE DESK REPLACED $1,200 IN CLOUD SUBSCRIPTIONS. The Minisforum MS-S1 Max ships with 128GB of unified memory shared between CPU and GPU. Qwen3-Coder 30B runs at 40 to 50 tokens per second. 6 agents run simultaneously without anyone watching the meter. An inbox sorter drafts replies every 15 minutes. A research monitor watches 30+ feeds and lands a digest by 7 AM, a job that costs $15 to $20 a day on cloud APIs. The same stack on cloud subscriptions ran $800 to $1,200 a month. The hardware costs $3,000 once. Break-even hits month 3 to 4. The real shift is behavior, not savings. A meeting prep agent never would have justified per-token cost. Running it free overnight made it obvious.

Lummox

14,093 просмотров • 1 день назад

JENSEN HUANG SOLD A $249 AI COMPUTER ON STAGE THAT KILLS YOUR $200/MONTH OPENAI BILL. THE VIDEO HAS 217,000 LIKES the box is called the jetson orin nano super. 70 trillion ai operations per second, 25 watts, smaller than a wallet. it runs llama 3, mistral, gemma and deepseek locally with no api fees and no data leaving your house a developer running automations and coding assistants pays $200 a month to openai. the same workload on this box costs $2 a month in electricity and breaks even on the hardware in 10 weeks you install ollama with one command. change one line in your code. point it at localhost instead of openai. everything else works identically 7 billion parameter models handle 80% of what people use chatgpt for. summarization, drafting, coding, document q&a, automation pipelines. the hard 20% you keep claude or gpt for. total monthly cost drops from $200 to $22 cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who set this up in 2025 are going to look very smart in 2027 bookmark this and read the article below
1:18

JENSEN HUANG SOLD A $249 AI COMPUTER ON STAGE THAT KILLS YOUR $200/MONTH OPENAI BILL. THE VIDEO HAS 217,000 LIKES the box is called the jetson orin nano super. 70 trillion ai operations per second, 25 watts, smaller than a wallet. it runs llama 3, mistral, gemma and deepseek locally with no api fees and no data leaving your house a developer running automations and coding assistants pays $200 a month to openai. the same workload on this box costs $2 a month in electricity and breaks even on the hardware in 10 weeks you install ollama with one command. change one line in your code. point it at localhost instead of openai. everything else works identically 7 billion parameter models handle 80% of what people use chatgpt for. summarization, drafting, coding, document q&a, automation pipelines. the hard 20% you keep claude or gpt for. total monthly cost drops from $200 to $22 cloud subscriptions keep getting more expensive and rate limits keep getting tighter. the people who set this up in 2025 are going to look very smart in 2027 bookmark this and read the article below

starmex

1,993,849 просмотров • 27 дней назад