正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Try now Hyperfast LLM running on custom built GPUs Answers in miliseconds, not seconds How? 🤯

@levelsio

905,424 subscribers

602,378 次观看 • 2 年前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

10 条评论

@levelsio 的头像

@levelsio2 年前

Via @mattshumer_

Beff – e/acc 的头像

Beff – e/acc2 年前

Hyper-optimizing a particular program for a TPU can yield 100×+ speedup. Speaking from experience

Suhail 的头像

Suhail2 年前

"An LPU Inference Engine, with LPU standing for Language Processing Unit™, is a new type of end-to-end processing unit system that provides the fastest inference for computationally intensive applications with a sequential component to them, such as AI language applications (LLMs)."

ben sima 的头像

ben sima2 年前

Answer: Custom hardware and great software (I work at Groq)

Tony Dinh 🎯 的头像

Tony Dinh 🎯2 年前

Seriously impressed. At this speed I think it's possible to build a human-like conversation experience where the AI can even interrupt you while you're speaking.

@levelsio 的头像

@levelsio2 年前

Not affiliated btw, it's made by @JonathanRoss321

Justin / Get Impeached 2! 的头像

Justin / Get Impeached 2!2 年前

waiting for someone to get confused between grok and groq

Groq Inc 的头像

Groq Inc2 年前

@levelsio No confusion, we trademarked our name in 2016.

Jay Scambler 的头像

Jay Scambler2 年前

They developed their own hardware that are built using LPUs instead of GPUs!

Krishiv 的头像

Krishiv2 年前

This is insane. What are some novel use-cases now possible because of this speed?

相关视频

Tools from AI SDK +  on-device LLM = 🤯 Access anything on your device with built-in LLM ⚡ Access contacts, calendar events, information about device & send emails 📩 Try it now with react-native-ai 🔥

Tools from AI SDK +  on-device LLM = 🤯 Access anything on your device with built-in LLM ⚡ Access contacts, calendar events, information about device & send emails 📩 Try it now with react-native-ai 🔥

Szymon Rybczak

25,338 次观看 • 1 年前

Dozens of teams have asked my advice on running LLMs. How fast is DeepSeek V3 with vLLM on 8 GPUs? What's the max throughput of Qwen 2.5 Coder with SGLang on one H100? Running & sharing benchmarks ad hoc was too slow So we built a tiny app, the LLM Engine Advisor

Dozens of teams have asked my advice on running LLMs. How fast is DeepSeek V3 with vLLM on 8 GPUs? What's the max throughput of Qwen 2.5 Coder with SGLang on one H100? Running & sharing benchmarks ad hoc was too slow So we built a tiny app, the LLM Engine Advisor

Charles 🎉 Frye

86,412 次观看 • 1 年前

This is how an LLM works in 60 seconds

This is how an LLM works in 60 seconds

Victor

52,396 次观看 • 1 年前

custom tools on demand mesh gradient editor built and running right in claude. took maybe 30 minutes

custom tools on demand mesh gradient editor built and running right in claude. took maybe 30 minutes

Aaron Ng

51,909 次观看 • 2 年前

IT WORKED!! Here's how you run Claude Code + Local LLM on your own machine in under 90 seconds

IT WORKED!! Here's how you run Claude Code + Local LLM on your own machine in under 90 seconds

Ziwen

466,143 次观看 • 5 个月前

Not only have I remade #Balatro’s card behaviour in WebGL I’ve now got custom #glsl shaders running on the cards too! Built with #threejs If this post hits 1k likes, I’ll drop a free & beginner-friendly video explaining exactly how I did it #threejsjourney #100DaysOfCode

Not only have I remade #Balatro’s card behaviour in WebGL I’ve now got custom #glsl shaders running on the cards too! Built with #threejs If this post hits 1k likes, I’ll drop a free & beginner-friendly video explaining exactly how I did it #threejsjourney #100DaysOfCode

Alex

37,795 次观看 • 1 年前

react-native-ai now supports ↓ ◆ streaming & generating text ◆ generating structured objects ◆ generating speech ◆ generating transcribes ◆ tool calling all running on-device, supported by  LLM🤯

react-native-ai now supports ↓ ◆ streaming & generating text ◆ generating structured objects ◆ generating speech ◆ generating transcribes ◆ tool calling all running on-device, supported by  LLM🤯

Szymon Rybczak

39,477 次观看 • 1 年前

Already running an inference engine? So where does NVIDIA Dynamo fit in? In five minutes, we break down how Dynamo sits around engines like SGLang, vLLM and TensorRT-LLM to scale inference across GPUs and nodes. Full video in the comments 🔽

Already running an inference engine? So where does NVIDIA Dynamo fit in? In five minutes, we break down how Dynamo sits around engines like SGLang, vLLM and TensorRT-LLM to scale inference across GPUs and nodes. Full video in the comments 🔽

NVIDIA AI

30,666 次观看 • 23 天前

Twitter Sentiment, in seconds. Try it now on The Hive!

Twitter Sentiment, in seconds. Try it now on The Hive!

The Hive

39,979 次观看 • 1 年前

air traffic control game, running on iOS/Android/Desktop built ENTIRELY with AI in 10 days. 🤯 #vibejam

air traffic control game, running on iOS/Android/Desktop built ENTIRELY with AI in 10 days. 🤯 #vibejam

Scobel

98,939 次观看 • 1 年前

SHAQ ordered the first custom built ‘64 Impala boat in the world 🤯

SHAQ ordered the first custom built ‘64 Impala boat in the world 🤯

808s & Car Shakes

11,983 次观看 • 2 个月前

Searching for answers in Skinwalker Canyon takes a dangerous turn... 🤯 #ExpeditionX starts NOW on Discovery!

Searching for answers in Skinwalker Canyon takes a dangerous turn... 🤯 #ExpeditionX starts NOW on Discovery!

Discovery

52,329 次观看 • 18 天前

🇺🇲 Vegas Sphere rendered entirely on $RENDER 🤯🤯🤯🫶🚀 Do you understand this? You are not bullish enough on Render! #Crypto in AI starts with GPUs. GPU blockchain The Render Network will lead. 🚀📈✍️ Not Financial Advice

🇺🇲 Vegas Sphere rendered entirely on $RENDER 🤯🤯🤯🫶🚀 Do you understand this? You are not bullish enough on Render! #Crypto in AI starts with GPUs. GPU blockchain The Render Network will lead. 🚀📈✍️ Not Financial Advice

Seth

62,879 次观看 • 1 年前

UNLIMITED KLING 3.0 — now on SeaArt. Multishot with six image references Consistent characters. Custom clips up to 15 seconds. Voices, languages, accents — upgraded. Right now - try UNLIMITED Kling 3.0 on SeaArt!

UNLIMITED KLING 3.0 — now on SeaArt. Multishot with six image references Consistent characters. Custom clips up to 15 seconds. Voices, languages, accents — upgraded. Right now - try UNLIMITED Kling 3.0 on SeaArt!

SeaArt.Ai🐋

12,278 次观看 • 7 个月前

🚨 xApple is going full launchpad aggregator🍎 We just integrated — built on top of LaunchLabs by @RaydiumProtocol Now anyone can launch tokens on directly from telegram in seconds. First of many integrations. Try it out now:

🚨 xApple is going full launchpad aggregator🍎 We just integrated — built on top of LaunchLabs by @RaydiumProtocol Now anyone can launch tokens on directly from telegram in seconds. First of many integrations. Try it out now:

xApple

59,276 次观看 • 1 年前

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

Vikas gupta

14,138 次观看 • 9 天前

this is how smooth i render in like 12 GPUs all for myself lol now try this in any other render platform, like all of them are useless outdated crap

this is how smooth i render in like 12 GPUs all for myself lol now try this in any other render platform, like all of them are useless outdated crap

Boring Always Bored

15,751 次观看 • 1 年前

how to make $67,000 in 6.7 seconds 🤯

how to make $67,000 in 6.7 seconds 🤯

blood

139,110 次观看 • 11 个月前

how to make $1,200 in 15 seconds 🤯

how to make $1,200 in 15 seconds 🤯

blood

15,575 次观看 • 1 年前

how to make $2,000 in 12 seconds 🤯

how to make $2,000 in 12 seconds 🤯

blood

131,255 次观看 • 1 年前