Загрузка видео...

Не удалось загрузить видео

На главную

Positron AI wants to run 16 trillion parameter models on a single server.

SemiAnalysis_'s profile picture

SemiAnalysis

148,437 subscribers

20,293 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables
21:59

APPLE RESEARCH SCIENTIST JUST SHOWED HOW 4 MAC STUDIOS RUN A TRILLION PARAMETER MODEL LOCALLY ZERO COSTS 13:18 she shows the main thing - connect 4 Mac Studios and you get 1TB of shared memory - exactly enough to run a trillion parameter model right on your desk Apple's library - and four machines start working as one cluster tensor parallelism: every machine holds part of every layer - all process the same token simultaneously - speed increases 3x compared to a single device fine-tuning: one Mac Studio processes 180 tokens per second four together process 600 and not a single byte of data leaves the room one command in the terminal - and a trillion parameter model answers from your desk and runs 24/7 data centers took years to build to run models like this - Apple did it with four Thunderbolt cables

Noisy

48,354 просмотров • 1 месяц назад

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."
0:50

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."

Deirdre Bosa

84,505 просмотров • 3 месяцев назад

What does it take to build chips that train trillion-parameter AI models? 🛠️💻 #AWS Trainium's chief architect breaks it down in this must-listen episode. #AI #TrainiumTuesdays 👉
0:40

What does it take to build chips that train trillion-parameter AI models? 🛠️💻 #AWS Trainium's chief architect breaks it down in this must-listen episode. #AI #TrainiumTuesdays 👉

Amazon Web Services

11,670 просмотров • 1 год назад

📢 ASUS ExpertCenter Pro ET900N G3 is now available to order worldwide. Built on NVIDIA GB300 Grace Blackwell Ultra Desktop Superchip: 🧠 748GB coherent memory 🤖 1 trillion parameter models ⚡ Data-center-class performance Learn more: #NVIDIA #AI
0:59

📢 ASUS ExpertCenter Pro ET900N G3 is now available to order worldwide. Built on NVIDIA GB300 Grace Blackwell Ultra Desktop Superchip: 🧠 748GB coherent memory 🤖 1 trillion parameter models ⚡ Data-center-class performance Learn more: #NVIDIA #AI

ASUS North America

98,621 просмотров • 2 месяцев назад

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."
2:14

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."

DogeDesigner

322,328 просмотров • 9 месяцев назад

Slash AI costs by 99%, run 1000+ models on a single GPU! OpenLoRA lets you run thousands of models on one GPU instead of one per model, drastically cutting costs and boosting flexibility. Excited to work with Aethir, Hyperbolic for infrastructure and for verifiability. Watch the demo:
3:18

Slash AI costs by 99%, run 1000+ models on a single GPU! OpenLoRA lets you run thousands of models on one GPU instead of one per model, drastically cutting costs and boosting flexibility. Excited to work with Aethir, Hyperbolic for infrastructure and for verifiability. Watch the demo:

OpenLedger

155,311 просмотров • 1 год назад

Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.
0:30

Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.

Cohere Labs

194,108 просмотров • 5 месяцев назад

Jane Street brought in Horace He to explain how to squeeze the most out of a single GPU. And then how to scale it to tens of thousands. Graduated from Cornell in 2020. Works on the PyTorch team. Training Llama 3 takes 40 trillion trillion floating point operations. He wrote the two things those models run on: torch.compile and FlexAttention.
1:02:33

Jane Street brought in Horace He to explain how to squeeze the most out of a single GPU. And then how to scale it to tens of thousands. Graduated from Cornell in 2020. Works on the PyTorch team. Training Llama 3 takes 40 trillion trillion floating point operations. He wrote the two things those models run on: torch.compile and FlexAttention.

zostaff

51,764 просмотров • 2 месяцев назад

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.
3:22

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.

DigitalOcean

2,030,708 просмотров • 22 дней назад

The Gemma family continues to grow today! 🩺 Announcing new models for MedGemma, our collection of multimodal open models specifically designed for health AI development that can run on a single GPU. 1️⃣ MedGemma 27B Multimodal 2️⃣ MedSigLIP (Read the thread to learn more)
0:56

The Gemma family continues to grow today! 🩺 Announcing new models for MedGemma, our collection of multimodal open models specifically designed for health AI development that can run on a single GPU. 1️⃣ MedGemma 27B Multimodal 2️⃣ MedSigLIP (Read the thread to learn more)

Google AI Developers

61,037 просмотров • 1 год назад

AMD Ryzen AI Halo is here! Yes, it can run 200B parameter models! This comes in Windows 11 Pro and Linux OS versions. Shout-out to AMD for sending this out for review! Available at Micro Center
4:45

AMD Ryzen AI Halo is here! Yes, it can run 200B parameter models! This comes in Windows 11 Pro and Linux OS versions. Shout-out to AMD for sending this out for review! Available at Micro Center

DaPoets

25,601 просмотров • 1 месяц назад

.RunAnywhere (YC W26) makes it easy for app developers to run AI models directly on phones and edge devices — fast, private, offline — with a single SDK and a dashboard to manage everything across user devices. Congrats on the launch Sanchit monga and Shubham Malhotra!
1:22

.RunAnywhere (YC W26) makes it easy for app developers to run AI models directly on phones and edge devices — fast, private, offline — with a single SDK and a dashboard to manage everything across user devices. Congrats on the launch Sanchit monga and Shubham Malhotra!

Y Combinator

69,645 просмотров • 6 месяцев назад

Inspired by Hugging Face's official MCP server, I built my own to expose my semantic search API for the HF ecosystem! Features AI-powered search, parameter analysis via safetensors, and tools to find similar models/datasets. Try: "Find non maths reasoning datasets from 2025"!
0:23

Inspired by Hugging Face's official MCP server, I built my own to expose my semantic search API for the HF ecosystem! Features AI-powered search, parameter analysis via safetensors, and tools to find similar models/datasets. Try: "Find non maths reasoning datasets from 2025"!

Daniel van Strien

15,147 просмотров • 1 год назад

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.
2:41

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Milk Road AI

37,388 просмотров • 27 дней назад

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below
16:10

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below

Paul Couvert

41,245 просмотров • 5 месяцев назад

A 10B Chinese model beats models 20x its size. And it runs on your laptop. Step 3 VL10B uses parallel reasoning to think like 16 experts at once. → Beats 200B parameter models → Scores 92.2% on multimodal benchmarks → Runs 16 reasoning paths simultaneously → Open source on Hugging Face → Works offline on your hardware OCR. GUI understanding. STEM problems. All covered. Completely free to download. Want the technical breakdown? DM me.
8:24

A 10B Chinese model beats models 20x its size. And it runs on your laptop. Step 3 VL10B uses parallel reasoning to think like 16 experts at once. → Beats 200B parameter models → Scores 92.2% on multimodal benchmarks → Runs 16 reasoning paths simultaneously → Open source on Hugging Face → Works offline on your hardware OCR. GUI understanding. STEM problems. All covered. Completely free to download. Want the technical breakdown? DM me.

Julian Goldie SEO

37,223 просмотров • 6 месяцев назад

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇
0:28

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

Hamza Khalid

26,707 просмотров • 10 месяцев назад

2.8 trillion parameters. Number one on frontend code arena. The largest open-weight model ever. Sputnik moment? It’s totally mind boggling that Elon Musk can spend $16 billion on Colossus 2 for a 20 trillion parameter model, and there's a 1% cost distilled version that’s coming out of China within weeks. Alex Karp said last week that you can’t throw all your proprietary weights over the wall to Anthropic. But he didn’t give a roadmap. Here we are, a week later, and it’s suddenly a free-for-all. Intelligence wants to be free.
1:07

2.8 trillion parameters. Number one on frontend code arena. The largest open-weight model ever. Sputnik moment? It’s totally mind boggling that Elon Musk can spend $16 billion on Colossus 2 for a 20 trillion parameter model, and there's a 1% cost distilled version that’s coming out of China within weeks. Alex Karp said last week that you can’t throw all your proprietary weights over the wall to Anthropic. But he didn’t give a roadmap. Here we are, a week later, and it’s suddenly a free-for-all. Intelligence wants to be free.

Dave Blundin

11,513 просмотров • 24 дней назад