Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Let's talk about MoE: 🔶 How many experts should you use? 🔶 How does dynamic routing actually behave in production? 🔶 How do you debug a model that won’t train? 🔶 What does 8x7B actually mean for memory and compute? 🔶 What hardware optimizations matter for sparse models? Mixture... show more

Cerebras

58,428 subscribers

345,732 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Undrafted rookie QB Jalon Daniels WENT OFF in his preseason debut, including a game-winning rushing TD. The former Kansas Jayhawk showcased his accuracy and athleticism in the Bucs first win: 🔶 10/15 completions 🔶 111 passing yards 🔶 1 rushing TD 🔶 88.5 passer rating Daniels is still behind Baker Mayfield and Jake Browning on the depth chart, but it’s going to be fun to see how the Bucs develop him. What a start for the undrafted rookie. 👏 (via Tampa Bay Buccaneers, NFL

Undrafted rookie QB Jalon Daniels WENT OFF in his preseason debut, including a game-winning rushing TD. The former Kansas Jayhawk showcased his accuracy and athleticism in the Bucs first win: 🔶 10/15 completions 🔶 111 passing yards 🔶 1 rushing TD 🔶 88.5 passer rating Daniels is still behind Baker Mayfield and Jake Browning on the depth chart, but it’s going to be fun to see how the Bucs develop him. What a start for the undrafted rookie. 👏 (via Tampa Bay Buccaneers, NFL

Preme Football

63,694 views • 1 month ago

A common misconception in AI right now is that everything we’re excited about is new. When I graduated from MIT in 2023, only 3% of students majored in AI (Course 6-4). I wasn’t one of them...oops. What I know about models, agents, frameworks, and hardware I learned after graduating. And if you’re newer to the field like me, it’s easy to assume today’s breakthroughs are fairly recent. After all, GPT-1 shipped in 2018... so it’s tempting for us to think the important ideas are less than a decade old. They aren’t. Mixture-of-Experts, the backbone of top models from companies like OpenAI and Google was invented in 1991 by Jacobs, Jordan, and Nowlan in “Adaptive Mixtures of Local Experts.” Over 30 years ago! In this conversation with Daria Soboleva, we discuss why MoEs are resurfacing now, the rapidly emerging compute bottlenecks, and how newer hardware architectures change the equation (Cerebras). A reminder that what feels “frontier” today is often decades in the making :)

A common misconception in AI right now is that everything we’re excited about is new. When I graduated from MIT in 2023, only 3% of students majored in AI (Course 6-4). I wasn’t one of them...oops. What I know about models, agents, frameworks, and hardware I learned after graduating. And if you’re newer to the field like me, it’s easy to assume today’s breakthroughs are fairly recent. After all, GPT-1 shipped in 2018... so it’s tempting for us to think the important ideas are less than a decade old. They aren’t. Mixture-of-Experts, the backbone of top models from companies like OpenAI and Google was invented in 1991 by Jacobs, Jordan, and Nowlan in “Adaptive Mixtures of Local Experts.” Over 30 years ago! In this conversation with Daria Soboleva, we discuss why MoEs are resurfacing now, the rapidly emerging compute bottlenecks, and how newer hardware architectures change the equation (Cerebras). A reminder that what feels “frontier” today is often decades in the making :)

Sarah Chieng

360,067 views • 8 months ago

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

Andrew Ng

387,172 views • 2 years ago

.Scott Wu to Joe Lonsdale on why Cognition uses AI in job interviews: "One or two years ago, people had this mentality of 'How do we make sure people aren't using AI while we're interviewing them?' And I think that has totally flipped. Honestly, I think that was wrong... If you're asking the question of like, how do we evaluate people on exactly the thing that AI can already do, that seems kind of the wrong. Our interview process has always been... you can use as much AI as you want to use. We're going to give you a few hours. Just build your whole own product surface. A lot of these are projects that, frankly, is like, if you were trying to do this by hand, you would not be able to get done in a few hours. So you kind of have to use AI for this. But in reality, what we actually want to test is, in addition to how familiar you are with these tools, what we actually want to test is, what do you think is the right thing to build? Or how do you make these product decisions? How do you make these trade-offs? How do you decide or collect information about what you should be doing?"

.Scott Wu to Joe Lonsdale on why Cognition uses AI in job interviews: "One or two years ago, people had this mentality of 'How do we make sure people aren't using AI while we're interviewing them?' And I think that has totally flipped. Honestly, I think that was wrong... If you're asking the question of like, how do we evaluate people on exactly the thing that AI can already do, that seems kind of the wrong. Our interview process has always been... you can use as much AI as you want to use. We're going to give you a few hours. Just build your whole own product surface. A lot of these are projects that, frankly, is like, if you were trying to do this by hand, you would not be able to get done in a few hours. So you kind of have to use AI for this. But in reality, what we actually want to test is, in addition to how familiar you are with these tools, what we actually want to test is, what do you think is the right thing to build? Or how do you make these product decisions? How do you make these trade-offs? How do you decide or collect information about what you should be doing?"

American Optimist

13,182 views • 5 months ago

6ix9ine insists lawyer Moe Gangat is "loud and wrong" for covering the Lil Durk trial and Moe responds. On Tuesday (Aug. 25), Tekashi shared a post on Instagram criticizing Moe's daily coverage of Durk's legal case. "Stop listening to this ni**a right here," 69 said in the video before sharing clips of Moe offering opinions on the Pooh Shiesty case. "This ni**a is not a lawyer. He gets everything wrong." "LOUD AND WRONG KEEP BELIEVING THESE INSTAGRAM LAWYERS," 6ix9ine captioned the post. A short time later, Moe responded. According to his LinkedIn page, Moe has a law degree from Georgetown, has been practicing law for over a decade and has his own employment law firm in New York. He defended his reporting. "This page is not into the gossip and the rumors," Moe said before taking a shot at Tekashi. "As you remember, we covered your case. And we covered how you had a lot of defenses and there was very little evidence against you. And then we heard how you snitched and cooperated."

6ix9ine insists lawyer Moe Gangat is "loud and wrong" for covering the Lil Durk trial and Moe responds. On Tuesday (Aug. 25), Tekashi shared a post on Instagram criticizing Moe's daily coverage of Durk's legal case. "Stop listening to this nia right here," 69 said in the video before sharing clips of Moe offering opinions on the Pooh Shiesty case. "This nia is not a lawyer. He gets everything wrong." "LOUD AND WRONG KEEP BELIEVING THESE INSTAGRAM LAWYERS," 6ix9ine captioned the post. A short time later, Moe responded. According to his LinkedIn page, Moe has a law degree from Georgetown, has been practicing law for over a decade and has his own employment law firm in New York. He defended his reporting. "This page is not into the gossip and the rumors," Moe said before taking a shot at Tekashi. "As you remember, we covered your case. And we covered how you had a lot of defenses and there was very little evidence against you. And then we heard how you snitched and cooperated."

XXL Magazine

97,958 views • 22 days ago

🏀 RUMBLE KONG LEAGUE: 3 THINGS WE LOVED IN PLAYTEST 🏀 Rumble Kong League SZN 1 Coastal Clash playtest was an absolute blast! This 3v3 mobile basketball had us hyped as we competed and climbed the ranks. Let’s dive into the 3 things we loved about the game that got us hooked ⬇️ — 🟠 THE RUMBLE PASS 🔶 Collect and unlock exclusive rewards by playing and winning matches. 🔶 Earn $FAME points, in-game accessories, and other valuable items. 🔶 After reaching the final level, continue earning unlimited rewards for every 3 wins. — 🟠 SET UP YOUR KONG 🔶 Choose 3 Kongs for left, right, and center positions to shape your team strategy. 🔶 Equip your Kongs with items that match your playstyle, enhancing stats and performance. 🔶 Select special attack and defense abilities that can influence the outcome of battles. — 🟠 DYNAMIC GAMEPLAY 🔶 Enjoy fast-paced, back-and-forth action from jumpball to final buzzer. 🔶 Special skills come to life with incredible visual effects that elevate the gameplay experience. 🔶 Shooting, stealing, and blocking are smooth, and we can’t wait for enhanced dribbling moves. — With its intense 3v3 matches, strategic gameplay, and endless rewards, Rumble Kong League is a must-play for gamers. 🏀 Are you ready for the next playtest and dominate the court? Stay updated and connect with the community on Discord! [link in bio] 🎮

🏀 RUMBLE KONG LEAGUE: 3 THINGS WE LOVED IN PLAYTEST 🏀 Rumble Kong League SZN 1 Coastal Clash playtest was an absolute blast! This 3v3 mobile basketball had us hyped as we competed and climbed the ranks. Let’s dive into the 3 things we loved about the game that got us hooked ⬇️ — 🟠 THE RUMBLE PASS 🔶 Collect and unlock exclusive rewards by playing and winning matches. 🔶 Earn $FAME points, in-game accessories, and other valuable items. 🔶 After reaching the final level, continue earning unlimited rewards for every 3 wins. — 🟠 SET UP YOUR KONG 🔶 Choose 3 Kongs for left, right, and center positions to shape your team strategy. 🔶 Equip your Kongs with items that match your playstyle, enhancing stats and performance. 🔶 Select special attack and defense abilities that can influence the outcome of battles. — 🟠 DYNAMIC GAMEPLAY 🔶 Enjoy fast-paced, back-and-forth action from jumpball to final buzzer. 🔶 Special skills come to life with incredible visual effects that elevate the gameplay experience. 🔶 Shooting, stealing, and blocking are smooth, and we can’t wait for enhanced dribbling moves. — With its intense 3v3 matches, strategic gameplay, and endless rewards, Rumble Kong League is a must-play for gamers. 🏀 Are you ready for the next playtest and dominate the court? Stay updated and connect with the community on Discord! [link in bio] 🎮

Beam Gamers

10,628 views • 1 year ago

We are told from a young age to pursue love. But how do you actually define love? And once you do, how do you find it? Poet Cleo Wade is the right person to ask, and our conversation about love led down many paths. Listen to our full conversation:

We are told from a young age to pursue love. But how do you actually define love? And once you do, how do you find it? Poet Cleo Wade is the right person to ask, and our conversation about love led down many paths. Listen to our full conversation:

Simon Sinek

62,457 views • 2 years ago

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).

Cerebras

346,446 views • 9 months ago

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.

OpenAI

225,959 views • 5 months ago

AI is changing everything, but how does it actually work? We break down machine learning, training, inference, and what happens behind the scenes when you use AI in the latest episode of #TheBringUp, featuring a hands-on model demo. 🎥 Watch now:

AI is changing everything, but how does it actually work? We break down machine learning, training, inference, and what happens behind the scenes when you use AI in the latest episode of #TheBringUp, featuring a hands-on model demo. 🎥 Watch now:

AMD

23,492 views • 2 months ago

Jensen Huang says, In the future, we will be able to talk to a protein We'll ask it: "what are you? are you soluble? how do you react to heat or chemicals?" This is how AI can talk to an image today, and how we've moved from only processing data to actually understanding it

Jensen Huang says, In the future, we will be able to talk to a protein We'll ask it: "what are you? are you soluble? how do you react to heat or chemicals?" This is how AI can talk to an image today, and how we've moved from only processing data to actually understanding it

Haider.

149,843 views • 10 months ago

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 views • 1 month ago

.Jon Durbin pre-trained a 20B MoE for under $10 an hour of compute. DropZone Episode 1 is him explaining how, in full. - a surrogate 48x smaller than the real expert, standing in for it - workers that train on data they never see - what ternary weights change about the hardware you need - why the existing sync algorithms all failed, so he wrote one Here's the full episode 👇 $TAO

.Jon Durbin pre-trained a 20B MoE for under $10 an hour of compute. DropZone Episode 1 is him explaining how, in full. - a surrogate 48x smaller than the real expert, standing in for it - workers that train on data they never see - what ternary weights change about the hardware you need - why the existing sync algorithms all failed, so he wrote one Here's the full episode 👇 $TAO

Chutes

10,849 views • 1 month ago

The biggest danger isn’t AI. It’s getting left behind. “You need access to compute. You need access to your own data.” “The stack is compute, data, ability to manage data, ability to control the models, ability to train the models, and ability to compete against your competitors and adversaries.” “You are gonna need a lot of electricity. How are you gonna get it?” “I don’t think the AI revolution is a hundred percent good. There are huge dangers. But being left behind is probably the biggest danger now.”

The biggest danger isn’t AI. It’s getting left behind. “You need access to compute. You need access to your own data.” “The stack is compute, data, ability to manage data, ability to control the models, ability to train the models, and ability to compete against your competitors and adversaries.” “You are gonna need a lot of electricity. How are you gonna get it?” “I don’t think the AI revolution is a hundred percent good. There are huge dangers. But being left behind is probably the biggest danger now.”

Chad Wahlquist

30,401 views • 14 days ago

EVERYTHING you NEED to know about Jack Dorsey's AI agent "Slack killer" Buzz (set up, use-cases etc in 38 mins) What we get into: 1. What Buzz actually is and should founders switch from Slack? 2. How to swap the model under any agent and keep all your context? 3. How to talk to your agents live with audio huddles? 4. How to get agents to build and deploy real apps for you, like a full CRM from one ask? 5. How to set up the context loop that feeds your live app data back to your agents? 6. How to share compute so a few people split one machine running a local model? 7. Who it's actually for right now, and what's still rough? Full breakdown on the pod The Startup Ideas Podcast (SIP) 🧃. Thanks to Vinny for jumping on and clearly explaining jack and team's latest product. My TLDR take is Buzz is a glimpse into the future of work. Some of you will roll your eyes at that, and I get it, it's alpha software and it's slow in places. But the core idea, that your context is the foundation and agents build out from there, is right, and that's worth seeing early. Watch Curious what you think

EVERYTHING you NEED to know about Jack Dorsey's AI agent "Slack killer" Buzz (set up, use-cases etc in 38 mins) What we get into: 1. What Buzz actually is and should founders switch from Slack? 2. How to swap the model under any agent and keep all your context? 3. How to talk to your agents live with audio huddles? 4. How to get agents to build and deploy real apps for you, like a full CRM from one ask? 5. How to set up the context loop that feeds your live app data back to your agents? 6. How to share compute so a few people split one machine running a local model? 7. Who it's actually for right now, and what's still rough? Full breakdown on the pod The Startup Ideas Podcast (SIP) 🧃. Thanks to Vinny for jumping on and clearly explaining jack and team's latest product. My TLDR take is Buzz is a glimpse into the future of work. Some of you will roll your eyes at that, and I get it, it's alpha software and it's slow in places. But the core idea, that your context is the foundation and agents build out from there, is right, and that's worth seeing early. Watch Curious what you think

GREG ISENBERG

558,824 views • 1 month ago

Built an AI maturity assessment. It's designed to answer... • how AI-native your team really is • where the bottlenecks are • what good looks like • and what to do next We turned the AI transformation model into something practical you can actually use. Takes about 3 minutes. Chat with it and it gives back: • a detailed AI maturity assessment • strengths and gaps across your workflows • practical recommendations • a downloadable report you can share internally The goal wasn’t to make a gimmicky quiz (please give me direct feedback on how to improve). It’s meant to be a practical diagnostic for how modern teams are actually adopting AI at work. Give it shot!

Built an AI maturity assessment. It's designed to answer... • how AI-native your team really is • where the bottlenecks are • what good looks like • and what to do next We turned the AI transformation model into something practical you can actually use. Takes about 3 minutes. Chat with it and it gives back: • a detailed AI maturity assessment • strengths and gaps across your workflows • practical recommendations • a downloadable report you can share internally The goal wasn’t to make a gimmicky quiz (please give me direct feedback on how to improve). It’s meant to be a practical diagnostic for how modern teams are actually adopting AI at work. Give it shot!

Hurley

33,232 views • 4 months ago

This is wild ChatGPT 6 is asked how many of the letter “E” are in the word seventeen. It can’t get the correct answer no matter how many times you ask. Even if you make it count out the letters, it still can’t get it right It will actually get an attitude and show frustration with you for trying to get the model to say the correct answer I ran the test multiple times and it keeps getting it wrong. When it does get it right, if you ask why it keeps getting it wrong it will then “correct” itself back to the wrong answer…. It even claims to be the smartest AI available According to Grok many AI models get 17 wrong. It has to do with LLMs. I asked Grok and it gets it right no problem It’s really wild that we are putting so much faith in AI when currently there are issues like this on such simple things

This is wild ChatGPT 6 is asked how many of the letter “E” are in the word seventeen. It can’t get the correct answer no matter how many times you ask. Even if you make it count out the letters, it still can’t get it right It will actually get an attitude and show frustration with you for trying to get the model to say the correct answer I ran the test multiple times and it keeps getting it wrong. When it does get it right, if you ask why it keeps getting it wrong it will then “correct” itself back to the wrong answer…. It even claims to be the smartest AI available According to Grok many AI models get 17 wrong. It has to do with LLMs. I asked Grok and it gets it right no problem It’s really wild that we are putting so much faith in AI when currently there are issues like this on such simple things

Wall Street Apes

45,553 views • 16 hours ago

They Want To Know How Many Bedrooms Are In Your Home Why? The Reason Should Worry You! The 2026 Census long form doesn't just ask how many people live in your home. It asks how many rooms are in your dwelling. Then it asks how many of those rooms are bedrooms. Why does the government need to know that? Look at what is already happening in the UK. Some local councils are implementing what they call under-occupancy premiums. If you have rooms in your home that you are not using they are taxing you for them. This is not a conspiracy theory. It is happening right now. And the Canadian government just made you tell them exactly how many rooms you have. Connect the dots. watch my full breakdown 👇

They Want To Know How Many Bedrooms Are In Your Home Why? The Reason Should Worry You! The 2026 Census long form doesn't just ask how many people live in your home. It asks how many rooms are in your dwelling. Then it asks how many of those rooms are bedrooms. Why does the government need to know that? Look at what is already happening in the UK. Some local councils are implementing what they call under-occupancy premiums. If you have rooms in your home that you are not using they are taxing you for them. This is not a conspiracy theory. It is happening right now. And the Canadian government just made you tell them exactly how many rooms you have. Connect the dots. watch my full breakdown 👇

Nico Lagan

48,160 views • 4 months ago

Yesterday, I joined TBPN to talk about what it really means to give AI a memory. And, why that’s becoming the next big moat. We discussed: - Why today’s AI apps are stateless - How memory is quickly becoming the real moat for AI systems - How Mem0 is decoupling memory from models - Why users should be able to own and carry their AI memories across apps - And, how 200 cold emails changed my life🙂 Appreciate John Coogan, Jordi Hays & the TBPN team for having me and for such a thoughtful discussion around the future of AI infrastructure.

Yesterday, I joined TBPN to talk about what it really means to give AI a memory. And, why that’s becoming the next big moat. We discussed: - Why today’s AI apps are stateless - How memory is quickly becoming the real moat for AI systems - How Mem0 is decoupling memory from models - Why users should be able to own and carry their AI memories across apps - And, how 200 cold emails changed my life🙂 Appreciate John Coogan, Jordi Hays & the TBPN team for having me and for such a thoughtful discussion around the future of AI infrastructure.

Taranjeet

14,913 views • 10 months ago