Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Let's talk about MoE: 🔶 How many experts should you use? 🔶 How does dynamic routing actually behave in production? 🔶 How do you debug a model that won’t train? 🔶 What does 8x7B actually mean for memory and compute? 🔶 What hardware optimizations matter for sparse models? Mixture... show more

Cerebras

56,091 subscribers

345,662 Aufrufe • vor 11 Monaten •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

MoE 101 - Episode 3: You don’t need Google-scale compute to train effective MoE models We proved it with a hands-on experiment that beats dense GPT-2 scale model. Daria Soboleva will show you how with methodology (and debugging secrets)

MoE 101 - Episode 3: You don’t need Google-scale compute to train effective MoE models We proved it with a hands-on experiment that beats dense GPT-2 scale model. Daria Soboleva will show you how with methodology (and debugging secrets)

Cerebras

35,877 Aufrufe • vor 9 Monaten

Why you MUST attend the Bitcoin Conference 2026 🔶 Meet Bitcoiners 🔶 Get a job in crypto 🔶 Hang out with Michael Saylor (actually) 🔶 & MUCH MORE! Use 'ALTCOINDAILY' for 10% off:

Why you MUST attend the Bitcoin Conference 2026 🔶 Meet Bitcoiners 🔶 Get a job in crypto 🔶 Hang out with Michael Saylor (actually) 🔶 & MUCH MORE! Use 'ALTCOINDAILY' for 10% off:

Altcoin Daily

32,633 Aufrufe • vor 2 Monaten

🔥 another win for open source AI SNOWFLAKE ARCTIC: 1] Mixture of Experts model with 128 (!) experts 2] open weights, open code 3] Company is sharing "cookbook" and data and teaching people how to build their own world class MoE models. 4] very cheap model to train (16x times less than Llama 3)

🔥 another win for open source AI SNOWFLAKE ARCTIC: 1] Mixture of Experts model with 128 (!) experts 2] open weights, open code 3] Company is sharing "cookbook" and data and teaching people how to build their own world class MoE models. 4] very cheap model to train (16x times less than Llama 3)

Wes Roth

10,233 Aufrufe • vor 2 Jahren

🔶How to make $3.6k in 30 seconds🔶

🔶How to make $3.6k in 30 seconds🔶

gigac

25,385 Aufrufe • vor 4 Monaten

No matter what kind of Bitcoiner you are, Core has something for you. 🔶 This is how we fully unlock BTCfi. 🔓 🔑

No matter what kind of Bitcoiner you are, Core has something for you. 🔶 This is how we fully unlock BTCfi. 🔓 🔑

Core DAO 🔶

28,842 Aufrufe • vor 1 Jahr

🔶How To Make 40k In 1 Minute 🔶 LaunchBlitz

🔶How To Make 40k In 1 Minute 🔶 LaunchBlitz

zeropnl

21,059 Aufrufe • vor 4 Monaten

🔶How to make $53,000 on BNB 🔶

🔶How to make $53,000 on BNB 🔶

zeropnl

67,998 Aufrufe • vor 8 Monaten

The lightweight customizable model production is currently in the final stage and is being tested! Let's take a look at how many hairstyles are available to match in this preview! illust:Annimiao_VArtist🔶COMMS CLOSED rigger:me #Live2D #Live2DWIP #Live2DShowcase #CustomizableVtuber

The lightweight customizable model production is currently in the final stage and is being tested! Let's take a look at how many hairstyles are available to match in this preview! illust:Annimiao_VArtist🔶COMMS CLOSED rigger:me #Live2D #Live2DWIP #Live2DShowcase #CustomizableVtuber

Rikou

13,141 Aufrufe • vor 4 Monaten

How to easily bridge your assets to the Core blockchain. 🎥 🔶

How to easily bridge your assets to the Core blockchain. 🎥 🔶

Core DAO 🔶

21,986 Aufrufe • vor 1 Jahr

Elusive Coaching 🔜 🔶 Unique Dribbling Techniques 🔶 Learn how to Build up, Attack and Control Games with Tiki-Taka style football 🔶 Understand the Importance and create a strong Mental Skillset 🟢 If you are interested - let me know here ⬇️ and I will add you

Elusive Coaching 🔜 🔶 Unique Dribbling Techniques 🔶 Learn how to Build up, Attack and Control Games with Tiki-Taka style football 🔶 Understand the Importance and create a strong Mental Skillset 🟢 If you are interested - let me know here ⬇️ and I will add you

Elusive

74,851 Aufrufe • vor 2 Jahren

Foundation models are powerful machine learning algorithms that sit at the core of many generative AI tools today. How are they built and deployed and how are they changing society? Here’s a quick intro for non-experts:

Foundation models are powerful machine learning algorithms that sit at the core of many generative AI tools today. How are they built and deployed and how are they changing society? Here’s a quick intro for non-experts:

Stanford HAI

20,769 Aufrufe • vor 3 Jahren

.FLO of EHC CHUR joined Graham at TOKEN2049 to discuss: 🔶 Why the rink features a giant Bitcoin logo 🔶 Paying players in BTC 🔶 Mining Bitcoin from ice rink heat 🔶 Building a Bitcoin treasury model in sports

Bitcoin.com News

12,473 Aufrufe • vor 8 Monaten

How to make $10,000 in 1 minute🔶

How to make $10,000 in 1 minute🔶

maku

93,298 Aufrufe • vor 8 Monaten

The Cowboy Code calls us to turn big dreams into deeds done. To do the impossible – and then do it again. The #CowboyFamily is invited to join us on next Thursday for a celebration as we launch a new era at #okstate. The Cowboy Code Rally 🔶 Thursday, Nov. 13 at 10:45 am 🔶 The McKnight Center Plaza 🔶 Block party, food trucks, giveaways Our word is our bond. The Code is our call. How will you answer? Learn more: #TheCodeCalls

The Cowboy Code calls us to turn big dreams into deeds done. To do the impossible – and then do it again. The #CowboyFamily is invited to join us on next Thursday for a celebration as we launch a new era at #okstate. The Cowboy Code Rally 🔶 Thursday, Nov. 13 at 10:45 am 🔶 The McKnight Center Plaza 🔶 Block party, food trucks, giveaways Our word is our bond. The Code is our call. How will you answer? Learn more: #TheCodeCalls

Oklahoma State Univ.

28,775 Aufrufe • vor 7 Monaten

A common misconception in AI right now is that everything we’re excited about is new. When I graduated from MIT in 2023, only 3% of students majored in AI (Course 6-4). I wasn’t one of them...oops. What I know about models, agents, frameworks, and hardware I learned after graduating. And if you’re newer to the field like me, it’s easy to assume today’s breakthroughs are fairly recent. After all, GPT-1 shipped in 2018... so it’s tempting for us to think the important ideas are less than a decade old. They aren’t. Mixture-of-Experts, the backbone of top models from companies like OpenAI and Google was invented in 1991 by Jacobs, Jordan, and Nowlan in “Adaptive Mixtures of Local Experts.” Over 30 years ago! In this conversation with Daria Soboleva, we discuss why MoEs are resurfacing now, the rapidly emerging compute bottlenecks, and how newer hardware architectures change the equation (Cerebras). A reminder that what feels “frontier” today is often decades in the making :)

A common misconception in AI right now is that everything we’re excited about is new. When I graduated from MIT in 2023, only 3% of students majored in AI (Course 6-4). I wasn’t one of them...oops. What I know about models, agents, frameworks, and hardware I learned after graduating. And if you’re newer to the field like me, it’s easy to assume today’s breakthroughs are fairly recent. After all, GPT-1 shipped in 2018... so it’s tempting for us to think the important ideas are less than a decade old. They aren’t. Mixture-of-Experts, the backbone of top models from companies like OpenAI and Google was invented in 1991 by Jacobs, Jordan, and Nowlan in “Adaptive Mixtures of Local Experts.” Over 30 years ago! In this conversation with Daria Soboleva, we discuss why MoEs are resurfacing now, the rapidly emerging compute bottlenecks, and how newer hardware architectures change the equation (Cerebras). A reminder that what feels “frontier” today is often decades in the making :)

Sarah Chieng

360,067 Aufrufe • vor 5 Monaten

The insane gap moe of Kaneko Lumi How are these even the same person?

The insane gap moe of Kaneko Lumi How are these even the same person?

Daikazoku 大家族 🖤❤️ / 🐟

20,642 Aufrufe • vor 7 Monaten

🔥Bitcoin beyond the base layer.Kyle Ellicott, Exec Director of Stacks Asia, joins David to discuss: 🔶 What can be built on Bitcoin 🔶 Unlocking new capital for the ecosystem 🔶 The culture & community fueling it

🔥Bitcoin beyond the base layer.Kyle Ellicott, Exec Director of Stacks Asia, joins David to discuss: 🔶 What can be built on Bitcoin 🔶 Unlocking new capital for the ecosystem 🔶 The culture & community fueling it

Bitcoin.com News

10,882 Aufrufe • vor 8 Monaten

During #BinanceOnline, CZ shared his insights and He said: “You may think a stablecoin is just a simple stablecoin, but it is actually financial access for millions of people in different parts of the world.” My POV: CZ 🔶 BNB always makes every detail count🫡 If you know what #BNB stands for in full, you already understand the message behind this background🔶💛 Build N Build is the mission🔶 #SuperApp #CZ

During #BinanceOnline, CZ shared his insights and He said: “You may think a stablecoin is just a simple stablecoin, but it is actually financial access for millions of people in different parts of the world.” My POV: CZ 🔶 BNB always makes every detail count🫡 If you know what #BNB stands for in full, you already understand the message behind this background🔶💛 Build N Build is the mission🔶 #SuperApp #CZ

ErnestoRw🔶

123,916 Aufrufe • vor 1 Monat

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

Andrew Ng

386,934 Aufrufe • vor 2 Jahren