Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Llama2 70B - blazing fast, low wattage, low cost Groq Inc LPU (language processing unit) running on Groq racks put together in days. High T/s, low T/watt, low $/T specifically for inference.

superjz's profile picture

Jay Z

5,446 subscribers

15,806 Aufrufe • vor 3 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.
0:35

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.

Lin

92,571 Aufrufe • vor 8 Monaten

🔊🔛🔥 ... Groq Inc launched voice generation today. GroqCloud now has realtime transcription, LLMs, *and* text-to-speech. You can build super-responsive, ultra low-latency voice agents end-to-end entirely on Groq! At Daily, we're big fans of Groq's fast, low-latency inference. Pipecat AI supports all the Groq models, including the new voice model. Lately, we've been obsessively playing a voice chat game that Mark Backman wrote. (My high score is 6, so far.) Here's Mark, with Groq's `Celeste-PlayAI` voice.
0:47

🔊🔛🔥 ... Groq Inc launched voice generation today. GroqCloud now has realtime transcription, LLMs, *and* text-to-speech. You can build super-responsive, ultra low-latency voice agents end-to-end entirely on Groq! At Daily, we're big fans of Groq's fast, low-latency inference. Pipecat AI supports all the Groq models, including the new voice model. Lately, we've been obsessively playing a voice chat game that Mark Backman wrote. (My high score is 6, so far.) Here's Mark, with Groq's `Celeste-PlayAI` voice.

kwindla

31,843 Aufrufe • vor 1 Jahr

Working on low flighted shot for this week AT&T Pebble Beach Pro-Am
0:23

Working on low flighted shot for this week AT&T Pebble Beach Pro-Am

Jason Day

334,411 Aufrufe • vor 1 Jahr

BANK OF CANADA GOVERNOR T MACK THE 🐐HAS SAID "Productivity levels in Canada are reaching emergency levels" He also blows the whistle on the 'import as many low-wage, low-cost workers as you can' ... as it is failing. 🎥👇
2:03

BANK OF CANADA GOVERNOR T MACK THE 🐐HAS SAID "Productivity levels in Canada are reaching emergency levels" He also blows the whistle on the 'import as many low-wage, low-cost workers as you can' ... as it is failing. 🎥👇

Tablesalt 🇨🇦🇺🇸

298,379 Aufrufe • vor 2 Jahren

In operations where milliseconds matter, routing data closer to where work happens improves speed. Edge Control enables low-latency performance and local data processing - and is centrally managed through T-Platform. More: T-Mobile Business Partner
3:56

In operations where milliseconds matter, routing data closer to where work happens improves speed. Edge Control enables low-latency performance and local data processing - and is centrally managed through T-Platform. More: T-Mobile Business Partner

Antonio Grasso

12,488 Aufrufe • vor 6 Monaten

Just dropped on HF: Supertonic TTS, a blazing fast speech model. 🤯 > RTF as low as 0.001 on RTX4090 > Runs on-device (no latency, full privacy) > 66M params + 8+ language SDKs > Browser, mobile, edge — it just works. > You can adjust inference steps, batch processing, and other parameters to match your specific needs! > Open-source. Ready to build with!
0:37

Just dropped on HF: Supertonic TTS, a blazing fast speech model. 🤯 > RTF as low as 0.001 on RTX4090 > Runs on-device (no latency, full privacy) > 66M params + 8+ language SDKs > Browser, mobile, edge — it just works. > You can adjust inference steps, batch processing, and other parameters to match your specific needs! > Open-source. Ready to build with!

steven

17,784 Aufrufe • vor 9 Monaten

Steak has: - B vitamins for your metabolism - Minerals and fat for your hormones - Nutrients you CAN´T get in a vegan diet. High in protein & low in calories to help you stay lean and build muscle. It´s even the best food for gut health But doesn´t it cause cancer? NO.
0:59

Steak has: - B vitamins for your metabolism - Minerals and fat for your hormones - Nutrients you CAN´T get in a vegan diet. High in protein & low in calories to help you stay lean and build muscle. It´s even the best food for gut health But doesn´t it cause cancer? NO.

Ábris Babicz | The Demi Cycle Protocol 🧬

588,297 Aufrufe • vor 1 Jahr

Gorgeous Comedian DESTROYS low-T heckler!! 😭🤣 New tour AH NO starts in March Tickets for all shows here More dates to come
0:43

Sensitive content

Gorgeous Comedian DESTROYS low-T heckler!! 😭🤣 New tour AH NO starts in March Tickets for all shows here More dates to come

Peter McGann

23,160 Aufrufe • vor 1 Jahr

Fast entry matters especially at the Las Vegas Grand Prix. With reliable, low-latency 5G, ticketing stays online and fans get through the gates faster—keeping operations smooth from the start. Follow along T-Mobile Business T-Mobile for Business Partner
1:38

Fast entry matters especially at the Las Vegas Grand Prix. With reliable, low-latency 5G, ticketing stays online and fans get through the gates faster—keeping operations smooth from the start. Follow along T-Mobile Business T-Mobile for Business Partner

Linda Grasso

13,504 Aufrufe • vor 9 Monaten

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?
30:38

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?

Julia Turc

46,967 Aufrufe • vor 2 Monaten

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.
1:24

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

171,478 Aufrufe • vor 3 Monaten

Central/SE #Texas have been on fire w/COVlD for 21 days. Levels are extremely high in College Station, followed by near Houston, then Waco. The CDC defines Texas as "very low" because 52% of wastewater sites are very low. That misses the public health opportunity.
2:58

Central/SE #Texas have been on fire w/COVlD for 21 days. Levels are extremely high in College Station, followed by near Houston, then Waco. The CDC defines Texas as "very low" because 52% of wastewater sites are very low. That misses the public health opportunity.

Mike Hoerger, PhD MSCR MBA

69,709 Aufrufe • vor 2 Monaten

“Listen to me. Have you seen the sh*t going on in Congress? I mean, really looked at the people we have representing this country? It is wildly out of hand. The bar to sit in Congress or to be in Congress is unrealistically low.” The bar is way too low!! Listen to this list 👇🏼
2:00

“Listen to me. Have you seen the sh*t going on in Congress? I mean, really looked at the people we have representing this country? It is wildly out of hand. The bar to sit in Congress or to be in Congress is unrealistically low.” The bar is way too low!! Listen to this list 👇🏼

TONY™

185,818 Aufrufe • vor 9 Monaten

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model *this good* running *this fast* is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models *screams* on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!
1:34

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model *this good* running *this fast* is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models *screams* on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

Santiago

131,019 Aufrufe • vor 1 Jahr

‘26 Luke Gremelspacher (Luke Gremelspacher) providing a solid look. 6’3” 215, Prep Baseball Uncommitted Quick, compact AA from a low slot. Sweeping SL + running FB in on RHHs. FB 85-87 T 88 (2x) SL 72-75 (2600s, some manip.) CH 78-79 #MPL25
0:35

‘26 Luke Gremelspacher (Luke Gremelspacher) providing a solid look. 6’3” 215, Prep Baseball Uncommitted Quick, compact AA from a low slot. Sweeping SL + running FB in on RHHs. FB 85-87 T 88 (2x) SL 72-75 (2600s, some manip.) CH 78-79 #MPL25

Prep Baseball Indiana Scouting Coverage

16,562 Aufrufe • vor 1 Jahr

'Fluoride reduces IQ' — RFK Jr 'If you have kids, would you rather them have cavities or low IQ?' Theo Von: 'I’d rather them have holes in their teeth than holes in their ideas' — print it on T-shirts
1:18

'Fluoride reduces IQ' — RFK Jr 'If you have kids, would you rather them have cavities or low IQ?' Theo Von: 'I’d rather them have holes in their teeth than holes in their ideas' — print it on T-shirts

RT

22,179 Aufrufe • vor 6 Monaten

For the last several months, we’ve been working on the design of our Starcloud-3 distributed constellation of 200kW inference nodes in dawn-dusk sun-synchronous orbit, providing many gigawatts of low-cost AI compute. It seems we’re not alone! 🙌
0:36

For the last several months, we’ve been working on the design of our Starcloud-3 distributed constellation of 200kW inference nodes in dawn-dusk sun-synchronous orbit, providing many gigawatts of low-cost AI compute. It seems we’re not alone! 🙌

Philip Johnston

49,612 Aufrufe • vor 7 Monaten