正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Can you spot the difference? 👀 Speculative decoding helps accelerate generative AI at the edge, especially on NVIDIA Jetson. Qwen 3.8 27B went from 13 to 35 tokens/sec, while Nemotron 3.5 Lightning went from 65 to 115. See how it works:

NVIDIA Robotics

142,784 subscribers

44,587 次观看 • 6 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

NVIDIA AI

211,099 次观看 • 1 个月前

“There's two reasons why we're building Nemotron. The first is because it helps us build the GPUs and systems for the future.” The second reason is because trust is the foundation of impactful AI–and it starts with understanding. That’s why Nemotron was built open source at its core—so developers, researchers, and enterprises can build together. Hear from Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, on why making AI open benefits everyone. 👉

“There's two reasons why we're building Nemotron. The first is because it helps us build the GPUs and systems for the future.” The second reason is because trust is the foundation of impactful AI–and it starts with understanding. That’s why Nemotron was built open source at its core—so developers, researchers, and enterprises can build together. Hear from Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, on why making AI open benefits everyone. 👉

NVIDIA

64,826 次观看 • 11 个月前

ABSOLUTE MILLSAGE-nema. OUR NOTES YOU COOKED AGAIN. Honestly from this I can really see the difference between the debut from Roselia, Aglow, and more to now. Its so beautiful, I love how far they went.

ABSOLUTE MILLSAGE-nema. OUR NOTES YOU COOKED AGAIN. Honestly from this I can really see the difference between the debut from Roselia, Aglow, and more to now. Its so beautiful, I love how far they went.

Daily Our Notes | Awacrashnote.id

20,477 次观看 • 1 个月前

Every big MoE model I test on my 3090 is basically to see how good it is compared to Qwen3.6-27B. "How good is it compared to Qwen3.6-27B?" is the question I get asked the most, and that shows how much people love and, more importantly, trust the 27B 27B is battle tested. It has proven its worth and has nothing left to prove. Now it just needs to make way for the next 27B. Here’s an example video: a Three.js task, same prompt, not oneshot but two shots for all models. All ran on my 3090 + 64GB VRAM Qwen-3.6-27b fully on vram while others have experts offloaded to system ram. Pinned on the left: Qwen3.6-27B-Q5 On the right, i'm switching between Inkling-Small-Q4_K_M DSV4-Flash-0731-UD-IQ1 Ling-3.0-Flash-Q4_K_M Qwen3.6-27B nailed it. The effects and visuals are just spot on.The other models got some things right, but not everything. Second place would have to go to Inkling-Small while DSV4-Flash-0731 and Ling-3.0-Flash failed to capture the full requirements. This is also why we’re all excited for Qwen Qwen3.8-27B!

Every big MoE model I test on my 3090 is basically to see how good it is compared to Qwen3.6-27B. "How good is it compared to Qwen3.6-27B?" is the question I get asked the most, and that shows how much people love and, more importantly, trust the 27B 27B is battle tested. It has proven its worth and has nothing left to prove. Now it just needs to make way for the next 27B. Here’s an example video: a Three.js task, same prompt, not oneshot but two shots for all models. All ran on my 3090 + 64GB VRAM Qwen-3.6-27b fully on vram while others have experts offloaded to system ram. Pinned on the left: Qwen3.6-27B-Q5 On the right, i'm switching between Inkling-Small-Q4_K_M DSV4-Flash-0731-UD-IQ1 Ling-3.0-Flash-Q4_K_M Qwen3.6-27B nailed it. The effects and visuals are just spot on.The other models got some things right, but not everything. Second place would have to go to Inkling-Small while DSV4-Flash-0731 and Ling-3.0-Flash failed to capture the full requirements. This is also why we’re all excited for Qwen Qwen3.8-27B!

AJ

11,550 次观看 • 16 天前

THIS ENGINEER RUNS CLAUDE CODE WITHOUT PAYING ANTHROPIC A SINGLE DOLLAR RTX 5090 under the desk, Qwen 3.5 35B, 140 tokens per second Claude Code pointed at localhost instead of Anthropic’s servers override two environment variables and your API bill drops to zero he built a full-stack Next.js app from scratch to prove it works then showed every bug and limitation other YouTubers hide same setup I run with two GPUs and Qwen 3.6 27B full breakdown ↓

THIS ENGINEER RUNS CLAUDE CODE WITHOUT PAYING ANTHROPIC A SINGLE DOLLAR RTX 5090 under the desk, Qwen 3.5 35B, 140 tokens per second Claude Code pointed at localhost instead of Anthropic’s servers override two environment variables and your API bill drops to zero he built a full-stack Next.js app from scratch to prove it works then showed every bug and limitation other YouTubers hide same setup I run with two GPUs and Qwen 3.6 27B full breakdown ↓

leopardracer

29,756 次观看 • 3 个月前

imagine thinking you could trust your best friend with your purse at the club while you went to the bathroom, only to see camera footage of her stealing $500 from it behind your back.

imagine thinking you could trust your best friend with your purse at the club while you went to the bathroom, only to see camera footage of her stealing $500 from it behind your back.

LASHY BILLS

80,797 次观看 • 8 个月前

At Amazon Web Services re:Invent, our SVP of Devices & Services, Panos Panay, put his skills to the test in a game of "This or That" — trying to spot the difference between real photos and AI-generated images created by Amazon Nova. Spoiler: It's harder than you think! We've unveiled major AI innovations at re:Invent that are transforming how we create, work, and imagine. Amazon Nova's image generation capabilities showcase just how far generative AI has come. See if you can spot the difference.

At Amazon Web Services re:Invent, our SVP of Devices & Services, Panos Panay, put his skills to the test in a game of "This or That" — trying to spot the difference between real photos and AI-generated images created by Amazon Nova. Spoiler: It's harder than you think! We've unveiled major AI innovations at re:Invent that are transforming how we create, work, and imagine. Amazon Nova's image generation capabilities showcase just how far generative AI has come. See if you can spot the difference.

Amazon

58,206 次观看 • 8 个月前

🇺🇦🇺🇸 Secretary of the Army: "What we've learned from the Ukrainians [is when you jailbreak your equipment] you can start to layer generative AI models [on top] to help with decision making." Senator Roger Wicker: "How long have the Ukrainians been using this concept?" Sec Army: "The entire war."

🇺🇦🇺🇸 Secretary of the Army: "What we've learned from the Ukrainians [is when you jailbreak your equipment] you can start to layer generative AI models [on top] to help with decision making." Senator Roger Wicker: "How long have the Ukrainians been using this concept?" Sec Army: "The entire war."

Colby Badhwar

223,283 次观看 • 3 个月前

We went inside Amazon's AI chip lab. Tucked inside an ordinary-looking office in Austin, Texas is a business taking on Nvidia. Amazon figured out how to design cutting-edge AI chips for its AWS customers, including some of AI's biggest stars, like Anthropic. Here's the breakdown: 0:00 Inside Amazon 1:12 The Amazon playbook 3:15 Why chips 5:52 How they build it 11:08 Why it matters to Amazon 13:43 The AI story 14:25 Where Nvidia fits 16:13 What it means for investors

We went inside Amazon's AI chip lab. Tucked inside an ordinary-looking office in Austin, Texas is a business taking on Nvidia. Amazon figured out how to design cutting-edge AI chips for its AWS customers, including some of AI's biggest stars, like Anthropic. Here's the breakdown: 0:00 Inside Amazon 1:12 The Amazon playbook 3:15 Why chips 5:52 How they build it 11:08 Why it matters to Amazon 13:43 The AI story 14:25 Where Nvidia fits 16:13 What it means for investors

Jon Erlichman

17,019 次观看 • 1 个月前

Doran finding out that Oner went to WOODZ’s concert 🥰 (While looking at Woodz’s concert info 👀) 🐿️: 6:00, 4:00... it must be really hard to get tickets now, right? If I wanted to buy them now, how would I even do it? 💬: Oner went to see it. 🐿️: Oner went? Oner went to the WOODZ concert? 🐿️: Oh~

Doran finding out that Oner went to WOODZ’s concert 🥰 (While looking at Woodz’s concert info 👀) 🐿️: 6:00, 4:00... it must be really hard to get tickets now, right? If I wanted to buy them now, how would I even do it? 💬: Oner went to see it. 🐿️: Oner went? Oner went to the WOODZ concert? 🐿️: Oh~

안개새벽 𓏲ּ𝄢

21,040 次观看 • 5 个月前

Got (enough) milk? 🥛 PayPal CTO Srini Venkatesan hosted a conversation with Jane Polak Scowcroft, Sr. Director, Generative AI Data Strategy at NVIDIA, on how agents are beginning to anticipate needs and take action on behalf of consumers. See how AI is moving from '”recommending” to “replenishing” in the next phase of commerce. 👇 #PayPalBeyond

Got (enough) milk? 🥛 PayPal CTO Srini Venkatesan hosted a conversation with Jane Polak Scowcroft, Sr. Director, Generative AI Data Strategy at NVIDIA, on how agents are beginning to anticipate needs and take action on behalf of consumers. See how AI is moving from '”recommending” to “replenishing” in the next phase of commerce. 👇 #PayPalBeyond

PayPal

13,238 次观看 • 4 个月前

Here's a comparison i never thought i would make Qwen-3.8-27b vs Gemini-3.7-flash Yes, you read it right! I don't know what to say, this is soo damn impressive. the difference is hugeee🤯 Take a look at this result 😍 (watch it in highest quality) Same prompt, both one shot on web-chat ui Qwen-3.8-27b produces consistent beautiful, visually stunning results. This is one of the best water surface simulations i have seen across every local models i have run and at the same time beating Gemini-3.7-flash result. And this is running on my 3090. And yes its Q5, not even f8.

Here's a comparison i never thought i would make Qwen-3.8-27b vs Gemini-3.7-flash Yes, you read it right! I don't know what to say, this is soo damn impressive. the difference is hugeee🤯 Take a look at this result 😍 (watch it in highest quality) Same prompt, both one shot on web-chat ui Qwen-3.8-27b produces consistent beautiful, visually stunning results. This is one of the best water surface simulations i have seen across every local models i have run and at the same time beating Gemini-3.7-flash result. And this is running on my 3090. And yes its Q5, not even f8.

AJ

262,976 次观看 • 10 天前

When powerful open-source models are accessible to all, differentiation shifts from the technology you choose to the expertise you build into them. Konstantine Buhler Buhler, Partner at Sequoia Capital, joins Kari Briski, VP of Generative AI Software for Enterprise at NVIDIA, to unpack how the agent economy transforms work and what it takes to stay ahead. 🎥 Watch the full conversation:

When powerful open-source models are accessible to all, differentiation shifts from the technology you choose to the expertise you build into them. Konstantine Buhler Buhler, Partner at Sequoia Capital, joins Kari Briski, VP of Generative AI Software for Enterprise at NVIDIA, to unpack how the agent economy transforms work and what it takes to stay ahead. 🎥 Watch the full conversation:

NVIDIA AI

11,938 次观看 • 8 个月前

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Avi Chawla

157,390 次观看 • 3 个月前

I spent today turning a blank Hermes install into a sovereign local agent on my RTX 5090. always-on, reachable over telegram, fully local. here's everything it can do. it runs Qwen3.6-27B on the 5090 and answers from my phone over telegram. ~~~ I first made it faster. switched to the MTP build for self-speculative decoding: 62 > 115 tokens/sec. ~~~ it now benchmarks models on command. I ask it to speed-test a gguf and it stops its own model server to free the gpu, runs llama-bench, renders a card, sends it to my phone, then restarts itself. it's never left offline. ~~~ it also knows me very well. I pointed it to my private HF repo where I store all my AI traces since I started using it. I created a local markdown memory vault, seeded from my own history, with semantic search over it. ~~~ the whole stack stays home: weights on disk, inference on the 5090, the agent in my pocket. here is the video it created to showcase everything it has now.

I spent today turning a blank Hermes install into a sovereign local agent on my RTX 5090. always-on, reachable over telegram, fully local. here's everything it can do. it runs Qwen3.6-27B on the 5090 and answers from my phone over telegram. ~ I first made it faster. switched to the MTP build for self-speculative decoding: 62 > 115 tokens/sec. ~ it now benchmarks models on command. I ask it to speed-test a gguf and it stops its own model server to free the gpu, runs llama-bench, renders a card, sends it to my phone, then restarts itself. it's never left offline. ~ it also knows me very well. I pointed it to my private HF repo where I store all my AI traces since I started using it. I created a local markdown memory vault, seeded from my own history, with semantic search over it. ~ the whole stack stays home: weights on disk, inference on the 5090, the agent in my pocket. here is the video it created to showcase everything it has now.

witcheer

15,041 次观看 • 2 个月前

I went back on Matt Turck’s podcast to talk about the state of the AI economy from Stripe's front-row seat, and how we’re building the economic infrastructure for AI. Empowering agents, monetizing tokens, vibe-deployment, token theft, and how company-building is changing.

I went back on Matt Turck’s podcast to talk about the state of the AI economy from Stripe's front-row seat, and how we’re building the economic infrastructure for AI. Empowering agents, monetizing tokens, vibe-deployment, token theft, and how company-building is changing.

Emily Sands

19,742 次观看 • 1 个月前

Qwen 3.8 Max beat Fable 5 at building 3D physics scenes for 7x cheaper! We gave two models the same task. Build three self-contained 3D scenes, each one HTML file with real physics that runs itself. Prompts: - A marble machine that lifts marbles up a wheel and drops them on a loop - A car factory assembly line - A sawmill cutting logs into planks Outputs: Qwen 3.8 Max: 46.6K tokens, $0.28 Fable 5: 38.7K tokens, $1.93 Qwen worked much harder on the details. Its textures and shadows look real. Fable 5 built a fast prototype that works and left it there. In the marble machine, Fable's marbles show up from nowhere at the top of the wheel. In Qwen's, you can see them go up. On the car line, Qwen's cars look much better and more real. Fable's sawmill is very strange. The saw looks wrong, and the log goes right through a beam before the saw cuts it. Atomic Chat will have day zero support to run Qwen 3.8 Max locally!

Sensitive content

Qwen 3.8 Max beat Fable 5 at building 3D physics scenes for 7x cheaper! We gave two models the same task. Build three self-contained 3D scenes, each one HTML file with real physics that runs itself. Prompts: - A marble machine that lifts marbles up a wheel and drops them on a loop - A car factory assembly line - A sawmill cutting logs into planks Outputs: Qwen 3.8 Max: 46.6K tokens, $0.28 Fable 5: 38.7K tokens, $1.93 Qwen worked much harder on the details. Its textures and shadows look real. Fable 5 built a fast prototype that works and left it there. In the marble machine, Fable's marbles show up from nowhere at the top of the wheel. In Qwen's, you can see them go up. On the car line, Qwen's cars look much better and more real. Fable's sawmill is very strange. The saw looks wrong, and the log goes right through a beam before the saw cuts it. Atomic Chat will have day zero support to run Qwen 3.8 Max locally!

atomic.chat

210,569 次观看 • 22 天前

Agent & the “Rari Boys” share a moment reflecting on how they went from rags to riches while looking at their brand new Ferraris 😭

ryan 🤿

659,115 次观看 • 1 年前

The Times went through the warp pipe into #SuperNintendoWorld at Universal Studio Hollywood to ride Mario Kart: Bowser’s Challenge, eat at Toadstool Cafe and collect coins. Here's how it went. Read Todd Martens on what to expect from the Mushroom Kingdom:

The Times went through the warp pipe into #SuperNintendoWorld at Universal Studio Hollywood to ride Mario Kart: Bowser’s Challenge, eat at Toadstool Cafe and collect coins. Here's how it went. Read Todd Martens on what to expect from the Mushroom Kingdom:

Los Angeles Times

12,324 次观看 • 3 年前