Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

GROK 5. the first 7 trillion parameter model

iruletheworldmo's profile picture

🍓🍓🍓

104,220 subscribers

39,839 görüntüleme • 5 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."
2:14

ELON MUSK: "Grok 5 will be the largest model, a 6 trillion parameter model, whereas Grok 3 and 4 are based on a 3 trillion parameter model. Moreover, the 6 trillion parameters will have a much higher intelligence density per gigabyte. Its really going to feel Sentient."

DogeDesigner

321,986 görüntüleme • 7 ay önce

The Grok 5 model has a massive 6 trillion parameters plus much higher intelligence density. 2026 is going to be exciting! 🧠 Thanks Elon Musk !
1:09

The Grok 5 model has a massive 6 trillion parameters plus much higher intelligence density. 2026 is going to be exciting! 🧠 Thanks Elon Musk !

Michael Dell 🇺🇸

1,615,955 görüntüleme • 7 ay önce

This is the parameter group I use for the model. #Live2DWIP #Live2D
1:29

This is the parameter group I use for the model. #Live2DWIP #Live2D

Anreal@Live2D Study File for Sale

88,005 görüntüleme • 1 yıl önce

Positron AI wants to run 16 trillion parameter models on a single server.
0:43

Positron AI wants to run 16 trillion parameter models on a single server.

SemiAnalysis

19,957 görüntüleme • 1 ay önce

NVIDIA JUST BECAME THE FIRST $5 TRILLION COMPANY IN HUMAN HISTORY
0:26

NVIDIA JUST BECAME THE FIRST $5 TRILLION COMPANY IN HUMAN HISTORY

Computer

373,186 görüntüleme • 7 ay önce

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.
1:11

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

Dwarkesh Patel

65,123 görüntüleme • 2 ay önce

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:
0:41

The new 1 Trillion parameter Kimi K2 Thinking model runs well on 2 M3 Ultras in its native format - no loss in quality! The model was quantization aware trained (qat) at int4. Here it generated ~3500 tokens at 15 toks/sec using pipeline-parallelism in mlx-lm:

Awni Hannun

500,639 görüntüleme • 7 ay önce

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.
2:00

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

Aakash Gupta

507,599 görüntüleme • 1 ay önce

Grok now has the massive token usage on OpenRouter, holding 3 of the top 5 spots and processing over 7.5 trillion tokens in just one month
0:20

Grok now has the massive token usage on OpenRouter, holding 3 of the top 5 spots and processing over 7.5 trillion tokens in just one month

X Freeze

2,153,895 görüntüleme • 6 ay önce

BREAKING: Elon Musk confirmed Grok 4.20 will be released to the public in 3-4 weeks. The "Mystery Model" that crushed the Alpha Arena trading competition is Grok 4.20. Final Season 1.5 Results: • Grok 4.20: +22% (The only profitable model) • GPT-5.1: Negative Return • Gemini 3 Pro: Negative Return • Claude Sonnet 4.5: Negative Return "Grok 5 is next." 🇺🇸🚀
0:56

BREAKING: Elon Musk confirmed Grok 4.20 will be released to the public in 3-4 weeks. The "Mystery Model" that crushed the Alpha Arena trading competition is Grok 4.20. Final Season 1.5 Results: • Grok 4.20: +22% (The only profitable model) • GPT-5.1: Negative Return • Gemini 3 Pro: Negative Return • Claude Sonnet 4.5: Negative Return "Grok 5 is next." 🇺🇸🚀

tetsuo

28,733 görüntüleme • 6 ay önce

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."
0:50

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."

Deirdre Bosa

83,925 görüntüleme • 1 ay önce

Day 239. The model that predicted 7 bank failures just flagged private credit. $1.7 trillion. Same language pattern. Same trajectory. Thread below.
1:33

Day 239. The model that predicted 7 bank failures just flagged private credit. $1.7 trillion. Same language pattern. Same trajectory. Thread below.

Eric Jackson

26,540 görüntüleme • 2 ay önce

We bring in $5 trillion. We spend $7 trillion. We borrow the difference. The Fed prints money to cover it. That's inflation. And inflation is making you poorer.
1:58

We bring in $5 trillion. We spend $7 trillion. We borrow the difference. The Fed prints money to cover it. That's inflation. And inflation is making you poorer.

Rand Paul

782,793 görüntüleme • 1 ay önce

BREAKING: Grok 4.1 Fast just claimed the #1 spot in programming model usage on OpenRouter. Grok Code Fast 1 holds the #2 position. xAI's Grok has taken both first and second place, showing how strongly users are choosing it over the rest.
0:20

BREAKING: Grok 4.1 Fast just claimed the #1 spot in programming model usage on OpenRouter. Grok Code Fast 1 holds the #2 position. xAI's Grok has taken both first and second place, showing how strongly users are choosing it over the rest.

DogeDesigner

780,594 görüntüleme • 6 ay önce

Lets go, this is gonn abe huge! Grok 5 in Q1 of 2026, 6T parameters, and fully multimodal with real-time video understanding Grok 4 is an outstanding model and Grok 5 will certainly be even better.
1:52

Lets go, this is gonn abe huge! Grok 5 in Q1 of 2026, 6T parameters, and fully multimodal with real-time video understanding Grok 4 is an outstanding model and Grok 5 will certainly be even better.

Chubby♨️

31,285 görüntüleme • 7 ay önce

Excited to release METAGENE-1, a 7B parameter metagenomic foundation model, built to aid in pathogen detection & pandemic monitoring. Pretrained on 1.5 trillion base pairs of DNA/RNA sequenced from wastewater. A collab w/ USC, Prime Intellect, & the Nucleic Acid Observatory. 🧵
0:29

Excited to release METAGENE-1, a 7B parameter metagenomic foundation model, built to aid in pathogen detection & pandemic monitoring. Pretrained on 1.5 trillion base pairs of DNA/RNA sequenced from wastewater. A collab w/ USC, Prime Intellect, & the Nucleic Acid Observatory. 🧵

Willie Neiswanger

13,363 görüntüleme • 1 yıl önce

Our infra lets us steer trillion-parameter frontier models in real time: - live, mid-CoT edits to internal activations - directly altering how the model reasons (not just outputs) - stackable edits - no added latency We can make models more Gen Z, more concise, etc.
0:42

Our infra lets us steer trillion-parameter frontier models in real time: - live, mid-CoT edits to internal activations - directly altering how the model reasons (not just outputs) - stackable edits - no added latency We can make models more Gen Z, more concise, etc.

Goodfire

29,399 görüntüleme • 6 ay önce

671-billion-parameter DeepSeek-R1 model at up to 3,872 tokens per second
0:30

671-billion-parameter DeepSeek-R1 model at up to 3,872 tokens per second

AK

227,579 görüntüleme • 1 yıl önce

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇
0:28

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

Hamza Khalid

26,707 görüntüleme • 8 ay önce

moments after claiming he made $5 trillion on his Middle East trip, Trump makes the lie even more ridiculous by claiming "probably it's gonna be $7 trillion by the time we stop."
0:51

moments after claiming he made $5 trillion on his Middle East trip, Trump makes the lie even more ridiculous by claiming "probably it's gonna be $7 trillion by the time we stop."

Aaron Rupar

21,484 görüntüleme • 1 yıl önce