Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Llama2 70B - blazing fast, low wattage, low cost Groq Inc LPU (language processing unit) running on Groq racks put together in days. High T/s, low T/watt, low $/T specifically for inference.

Jay Z

5,446 subscribers

15,806 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.

This is why Nvidia bought Groq. If Groq deploys a ton of low-cost inference chips, demand for training will increase massively. The more inference you have, the more training you need, and vice versa. Nvidia can keep its high margins selling chips for training forever.

Lin

92,571 views • 6 months ago

🔊🔛🔥 ... Groq Inc launched voice generation today. GroqCloud now has realtime transcription, LLMs, *and* text-to-speech. You can build super-responsive, ultra low-latency voice agents end-to-end entirely on Groq! At Daily, we're big fans of Groq's fast, low-latency inference. Pipecat AI supports all the Groq models, including the new voice model. Lately, we've been obsessively playing a voice chat game that Mark Backman wrote. (My high score is 6, so far.) Here's Mark, with Groq's `Celeste-PlayAI` voice.

🔊🔛🔥 ... Groq Inc launched voice generation today. GroqCloud now has realtime transcription, LLMs, and text-to-speech. You can build super-responsive, ultra low-latency voice agents end-to-end entirely on Groq! At Daily, we're big fans of Groq's fast, low-latency inference. Pipecat AI supports all the Groq models, including the new voice model. Lately, we've been obsessively playing a voice chat game that Mark Backman wrote. (My high score is 6, so far.) Here's Mark, with Groq's `Celeste-PlayAI` voice.

kwindla

31,791 views • 1 year ago

Working on low flighted shot for this week AT&T Pebble Beach Pro-Am

Working on low flighted shot for this week AT&T Pebble Beach Pro-Am

Jason Day

334,411 views • 1 year ago

BANK OF CANADA GOVERNOR T MACK THE 🐐HAS SAID "Productivity levels in Canada are reaching emergency levels" He also blows the whistle on the 'import as many low-wage, low-cost workers as you can' ... as it is failing. 🎥👇

BANK OF CANADA GOVERNOR T MACK THE 🐐HAS SAID "Productivity levels in Canada are reaching emergency levels" He also blows the whistle on the 'import as many low-wage, low-cost workers as you can' ... as it is failing. 🎥👇

Tablesalt 🇨🇦🇺🇸

298,362 views • 2 years ago

In operations where milliseconds matter, routing data closer to where work happens improves speed. Edge Control enables low-latency performance and local data processing - and is centrally managed through T-Platform. More: T-Mobile Business Partner

In operations where milliseconds matter, routing data closer to where work happens improves speed. Edge Control enables low-latency performance and local data processing - and is centrally managed through T-Platform. More: T-Mobile Business Partner

Antonio Grasso

12,488 views • 4 months ago

Just dropped on HF: Supertonic TTS, a blazing fast speech model. 🤯 > RTF as low as 0.001 on RTX4090 > Runs on-device (no latency, full privacy) > 66M params + 8+ language SDKs > Browser, mobile, edge — it just works. > You can adjust inference steps, batch processing, and other parameters to match your specific needs! > Open-source. Ready to build with!

Just dropped on HF: Supertonic TTS, a blazing fast speech model. 🤯 > RTF as low as 0.001 on RTX4090 > Runs on-device (no latency, full privacy) > 66M params + 8+ language SDKs > Browser, mobile, edge — it just works. > You can adjust inference steps, batch processing, and other parameters to match your specific needs! > Open-source. Ready to build with!

steven

17,784 views • 7 months ago

Steak has: - B vitamins for your metabolism - Minerals and fat for your hormones - Nutrients you CAN´T get in a vegan diet. High in protein & low in calories to help you stay lean and build muscle. It´s even the best food for gut health But doesn´t it cause cancer? NO.

Steak has: - B vitamins for your metabolism - Minerals and fat for your hormones - Nutrients you CAN´T get in a vegan diet. High in protein & low in calories to help you stay lean and build muscle. It´s even the best food for gut health But doesn´t it cause cancer? NO.

Ábris Babicz | The Demi Cycle Protocol 🧬

588,297 views • 1 year ago

Gorgeous Comedian DESTROYS low-T heckler!! 😭🤣 New tour AH NO starts in March Tickets for all shows here More dates to come

Sensitive content

Gorgeous Comedian DESTROYS low-T heckler!! 😭🤣 New tour AH NO starts in March Tickets for all shows here More dates to come

Peter McGann

23,151 views • 1 year ago

Fast entry matters especially at the Las Vegas Grand Prix. With reliable, low-latency 5G, ticketing stays online and fans get through the gates faster—keeping operations smooth from the start. Follow along T-Mobile Business T-Mobile for Business Partner

Fast entry matters especially at the Las Vegas Grand Prix. With reliable, low-latency 5G, ticketing stays online and fans get through the gates faster—keeping operations smooth from the start. Follow along T-Mobile Business T-Mobile for Business Partner

Linda Grasso

13,504 views • 7 months ago

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?

Julia Turc

46,905 views • 1 month ago

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

I didn't expect DeepSeek v4 PRO (not Flash) to run well on the Mac Studio M3 Ultra with 512GB of RAM. This is 2 bit quantized with the same DwarfStar recipe used for Flash. 433GB GGUF file. 130 t/s prefill, 13 t/s generation. Prefill in the video is low because small prompt.

antirez

169,759 views • 1 month ago

Just saw the future of live sports at T-Mobile Innovation Week 👀 The RIEDELCommunications RefCam streams ref POV in real time over 5G — ultra-light, low-latency, and game-changing for broadcast + coaching. T-Mobile Business Partner

Just saw the future of live sports at T-Mobile Innovation Week 👀 The RIEDELCommunications RefCam streams ref POV in real time over 5G — ultra-light, low-latency, and game-changing for broadcast + coaching. T-Mobile Business Partner

Beverley Eve

11,045 views • 10 months ago

Central/SE #Texas have been on fire w/COVlD for 21 days. Levels are extremely high in College Station, followed by near Houston, then Waco. The CDC defines Texas as "very low" because 52% of wastewater sites are very low. That misses the public health opportunity.

Central/SE #Texas have been on fire w/COVlD for 21 days. Levels are extremely high in College Station, followed by near Houston, then Waco. The CDC defines Texas as "very low" because 52% of wastewater sites are very low. That misses the public health opportunity.

Mike Hoerger, PhD MSCR MBA

68,853 views • 19 days ago

“Listen to me. Have you seen the sh*t going on in Congress? I mean, really looked at the people we have representing this country? It is wildly out of hand. The bar to sit in Congress or to be in Congress is unrealistically low.” The bar is way too low!! Listen to this list 👇🏼

“Listen to me. Have you seen the sh*t going on in Congress? I mean, really looked at the people we have representing this country? It is wildly out of hand. The bar to sit in Congress or to be in Congress is unrealistically low.” The bar is way too low!! Listen to this list 👇🏼

TONY™

185,786 views • 7 months ago

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model *this good* running *this fast* is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models *screams* on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model this good running this fast is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models screams on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

Santiago

130,934 views • 1 year ago

‘26 Luke Gremelspacher (Luke Gremelspacher) providing a solid look. 6’3” 215, Prep Baseball Uncommitted Quick, compact AA from a low slot. Sweeping SL + running FB in on RHHs. FB 85-87 T 88 (2x) SL 72-75 (2600s, some manip.) CH 78-79 #MPL25

‘26 Luke Gremelspacher (Luke Gremelspacher) providing a solid look. 6’3” 215, Prep Baseball Uncommitted Quick, compact AA from a low slot. Sweeping SL + running FB in on RHHs. FB 85-87 T 88 (2x) SL 72-75 (2600s, some manip.) CH 78-79 #MPL25

Prep Baseball Indiana Scouting Coverage

16,495 views • 1 year ago

'Fluoride reduces IQ' — RFK Jr 'If you have kids, would you rather them have cavities or low IQ?' Theo Von: 'I’d rather them have holes in their teeth than holes in their ideas' — print it on T-shirts

'Fluoride reduces IQ' — RFK Jr 'If you have kids, would you rather them have cavities or low IQ?' Theo Von: 'I’d rather them have holes in their teeth than holes in their ideas' — print it on T-shirts

RT

22,179 views • 5 months ago

For the last several months, we’ve been working on the design of our Starcloud-3 distributed constellation of 200kW inference nodes in dawn-dusk sun-synchronous orbit, providing many gigawatts of low-cost AI compute. It seems we’re not alone! 🙌

For the last several months, we’ve been working on the design of our Starcloud-3 distributed constellation of 200kW inference nodes in dawn-dusk sun-synchronous orbit, providing many gigawatts of low-cost AI compute. It seems we’re not alone! 🙌

Philip Johnston

49,351 views • 5 months ago