Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Running multi agents on open source models has always been a hit or miss until... Llama 3.1 on Groq Inc with 405b and 70b

FlowiseAI

28,790 subscribers

48,208 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Very, very fast voice bots. Llama 3.1 running on Groq Inc. 🚀 500ms voice-to-voice response times

Very, very fast voice bots. Llama 3.1 running on Groq Inc. 🚀 500ms voice-to-voice response times

kwindla

386,488 views • 1 year ago

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.

Ahmad Al-Dahle

866,261 views • 1 year ago

Llama 3.3 70B is now available on LlamaCoder through the Together AI API! Achieves similar performance to Llama 3.1 405B 🤯

Llama 3.3 70B is now available on LlamaCoder through the Together AI API! Achieves similar performance to Llama 3.1 405B 🤯

Hassan

35,133 views • 1 year ago

Introducing LlamaCoder! An open source Claude Artifacts app that can generate full React apps and components with Llama 3.1 405B. 100% free and open source.

Introducing LlamaCoder! An open source Claude Artifacts app that can generate full React apps and components with Llama 3.1 405B. 100% free and open source.

Hassan

501,207 views • 1 year ago

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context window and improved support for 8 languages among other improvements. Llama 3.1 405B rivals leading closed source models on state-of-the-art capabilities across a range of tasks in general knowledge, steerability, math, tool use and multilingual translation. The models are available to download now directly from Meta or Hugging Face. With today’s release the ecosystem is also ready to go with 25+ partners rolling out our latest models — including Amazon Web Services, NVIDIA, Databricks, Groq Inc, Dell Technologies, Microsoft Azure and Google Cloud ready on day one. More details in the full announcement ➡️ Download Llama 3.1 models ➡️ With these releases we’re setting the stage for unprecedented new opportunities and we can’t wait to see the innovation our newest models will unlock across all levels of the AI community.

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context window and improved support for 8 languages among other improvements. Llama 3.1 405B rivals leading closed source models on state-of-the-art capabilities across a range of tasks in general knowledge, steerability, math, tool use and multilingual translation. The models are available to download now directly from Meta or Hugging Face. With today’s release the ecosystem is also ready to go with 25+ partners rolling out our latest models — including Amazon Web Services, NVIDIA, Databricks, Groq Inc, Dell Technologies, Microsoft Azure and Google Cloud ready on day one. More details in the full announcement ➡️ Download Llama 3.1 models ➡️ With these releases we’re setting the stage for unprecedented new opportunities and we can’t wait to see the innovation our newest models will unlock across all levels of the AI community.

AI at Meta

1,268,336 views • 1 year ago

deepseek-r1-distill-llama-70b on groq

deepseek-r1-distill-llama-70b on groq

Developers Digest

143,277 views • 1 year ago

My open source Claude Artifacts app using Llama 3.1 405B is dropping in 24 hours! Free & fully open source.

My open source Claude Artifacts app using Llama 3.1 405B is dropping in 24 hours! Free & fully open source.

Hassan

272,373 views • 1 year ago

With the release of Llama 3.1 405B, @TogetherCompute built LlamaCoder — an open source web app that can generate an entire app from a prompt. The repo has now been cloned by hundreds of devs on GitHub and starred 2K+ times. More on this project ➡️

With the release of Llama 3.1 405B, @TogetherCompute built LlamaCoder — an open source web app that can generate an entire app from a prompt. The repo has now been cloned by hundreds of devs on GitHub and starred 2K+ times. More on this project ➡️

AI at Meta

260,503 views • 1 year ago

Multi-agent orchestration - Niko spinning 100 AI agents in ultra-fast sandboxes ~ AI at Meta Llama 4 on Groq Inc executing code in Daytona

Multi-agent orchestration - Niko spinning 100 AI agents in ultra-fast sandboxes ~ AI at Meta Llama 4 on Groq Inc executing code in Daytona

Daytona.io

73,057 views • 1 year ago

$Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:$

Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:

Matt Shumer

278,353 views • 1 year ago

Stagehand is getting really fast ... See how Llama 3 70b on Groq Inc outperforms OpenAI 4o in speed on this form fill 54.48 seconds for groq VS 62.75 for 4o🤯

Stagehand is getting really fast ... See how Llama 3 70b on Groq Inc outperforms OpenAI 4o in speed on this form fill 54.48 seconds for groq VS 62.75 for 4o🤯

Kyle Jeong

26,095 views • 1 year ago

Llama 3.3 70B 4-bit running on a 128GB M4 Max with MLX LM (~12 toks/sec) with mactop on the right. "Good enough", local, private and open models are now more accessible than ever!

Llama 3.3 70B 4-bit running on a 128GB M4 Max with MLX LM (~12 toks/sec) with mactop on the right. "Good enough", local, private and open models are now more accessible than ever!

Ivan Fioravanti ᯅ

43,929 views • 1 year ago

Wanna know whether different LLM providers serve the same LLama 3.1 70B? I sure did! So I ran a quick eval to get some surprising results + open sourced my code 👇 Check out my comparison between Groq Inc Fireworks AI octoaicloud DeepInfra and Together AI

Wanna know whether different LLM providers serve the same LLama 3.1 70B? I sure did! So I ran a quick eval to get some surprising results + open sourced my code 👇 Check out my comparison between Groq Inc Fireworks AI octoaicloud DeepInfra and Together AI

Alex Volkov (Thursd/AI)

26,549 views • 1 year ago

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,505 views • 1 year ago

Exclusive: Meta just released Llama 3.1 405B — the first-ever open-sourced frontier AI model, beating top closed models like GPT-4o across several benchmarks. I sat down with Mark Zuckerberg, diving into why this marks a major moment in AI history. Timestamps: 00:00 Intro 00:38 Meta’s Llama 3.1 rundown 03:44 Real-world use cases for Llama 3.1 06:15 Educating developers on open-source AI tools 09:43 Societal implications of open-source AI 13:00 Balancing power and managing bad actors 14:40 Open source and global competition 16:59 Accelerating innovation and economic growth 20:04 Zuck on Apple and lessons from the past 24:22 Future of AI: Llama 3 and beyond 26:43 Prediction: Billions of personalized AI agents 31:32 Factors to changing anti-AI sentiment

Exclusive: Meta just released Llama 3.1 405B — the first-ever open-sourced frontier AI model, beating top closed models like GPT-4o across several benchmarks. I sat down with Mark Zuckerberg, diving into why this marks a major moment in AI history. Timestamps: 00:00 Intro 00:38 Meta’s Llama 3.1 rundown 03:44 Real-world use cases for Llama 3.1 06:15 Educating developers on open-source AI tools 09:43 Societal implications of open-source AI 13:00 Balancing power and managing bad actors 14:40 Open source and global competition 16:59 Accelerating innovation and economic growth 20:04 Zuck on Apple and lessons from the past 24:22 Future of AI: Llama 3 and beyond 26:43 Prediction: Billions of personalized AI agents 31:32 Factors to changing anti-AI sentiment

Rowan Cheung

2,621,398 views • 1 year ago

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model *this good* running *this fast* is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models *screams* on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model this good running this fast is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models screams on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

Santiago

130,934 views • 1 year ago

Llama 3.3 70B 4-bit runs nicely on a 64GB M3 Max with in MLX LM (~10 toks/sec). Would be even faster on an M4 Max. Yesterday's server-only 405B is today's laptop 70B:

Llama 3.3 70B 4-bit runs nicely on a 64GB M3 Max with in MLX LM (~10 toks/sec). Would be even faster on an M4 Max. Yesterday's server-only 405B is today's laptop 70B:

Awni Hannun

110,375 views • 1 year ago

Llama2 70B - blazing fast, low wattage, low cost Groq Inc LPU (language processing unit) running on Groq racks put together in days. High T/s, low T/watt, low $/T specifically for inference.

Llama2 70B - blazing fast, low wattage, low cost Groq Inc LPU (language processing unit) running on Groq racks put together in days. High T/s, low T/watt, low $/T specifically for inference.

Jay Z

15,799 views • 2 years ago