Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Can you spot the difference? 👀 Speculative decoding helps accelerate generative AI at the edge, especially on NVIDIA Jetson. Qwen 3.8 27B went from 13 to 35 tokens/sec, while Nemotron 3.5 Lightning went from 65 to 115. See how it works:

NVIDIARobotics's profile picture

NVIDIA Robotics

142,784 subscribers

44,587 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

PERPLEXITY 🔥: Perplexity Pro and Max subscribers can now use a new Portable Computer on DGX Spark from NVIDIA, powered purely by local models! Available models: > PPLX 27B is a new post-trained model from Perplexity. > Qwen 3.8 27B > Nemotron 3.5 Lightning (coming soon) They really did it! 🤖
1:37

PERPLEXITY 🔥: Perplexity Pro and Max subscribers can now use a new Portable Computer on DGX Spark from NVIDIA, powered purely by local models! Available models: > PPLX 27B is a new post-trained model from Perplexity. > Qwen 3.8 27B > Nemotron 3.5 Lightning (coming soon) They really did it! 🤖

🚨 AI News | TestingCatalog

42,207 Aufrufe • vor 1 Monat

One shot builds aren’t cloud only. Qwen 3.8 Flash plans the build, then hands it to Qwen 3.8 27B, all running locally across the GPU, CPU and 128GB of unified memory on one AMD Ryzen AI Max+ processor. Now, we iterate. See how Ryzen AI Max+ helps you build.
0:41

One shot builds aren’t cloud only. Qwen 3.8 Flash plans the build, then hands it to Qwen 3.8 27B, all running locally across the GPU, CPU and 128GB of unified memory on one AMD Ryzen AI Max+ processor. Now, we iterate. See how Ryzen AI Max+ helps you build.

AMD Ryzen

17,829 Aufrufe • vor 25 Tagen

When will AI personal assistants be fast enough to be useful? Using Qwen 3.8 27B on Cerebras at ~1,500 tokens/sec, we made an AI personal assistant 19x faster than a suite of other AI personal assistants - Grok Bot, Meta Muse, and Claude Cowork - on the same dinner reservation task.
2:45

When will AI personal assistants be fast enough to be useful? Using Qwen 3.8 27B on Cerebras at ~1,500 tokens/sec, we made an AI personal assistant 19x faster than a suite of other AI personal assistants - Grok Bot, Meta Muse, and Claude Cowork - on the same dinner reservation task.

Cerebras

27,910 Aufrufe • vor 19 Tagen

you can just hallucinate the entire internet with Qwen 3.8 27b running at 2,000 tokens/second? part 2 of turning Cerebras + Qwen 3.8 27B into an OS: built an offline browser with zero network calls and mounted it directly the JIT ubuntu desktop. no wifi. no scraping. zero packets sent to external CDNs. you search a site, set a year, and qwen 27b at 1,950 tok/s synthesizes the entire DOM on the fly. here is youtube in 2045 vs 1999: → search google for youtube inside the OS → scrub to 2045: instant futuristic feed → scrub to 1999: raw web 1.0 time capsule in seconds at this speed, browsing isn't retrieving files from a server, it's querying an alternate reality. a 2D browser window is just step zero. imagine full operating systems, virtual worlds, and complex simulation engines existing purely as model weights. zero gigabytes stored on disk, just pure interactive reality streamed on demand. What else becomes a possibility with the qwen 3.8 27b (dense) at 2000 tokens/sec?
0:33

you can just hallucinate the entire internet with Qwen 3.8 27b running at 2,000 tokens/second? part 2 of turning Cerebras + Qwen 3.8 27B into an OS: built an offline browser with zero network calls and mounted it directly the JIT ubuntu desktop. no wifi. no scraping. zero packets sent to external CDNs. you search a site, set a year, and qwen 27b at 1,950 tok/s synthesizes the entire DOM on the fly. here is youtube in 2045 vs 1999: → search google for youtube inside the OS → scrub to 2045: instant futuristic feed → scrub to 1999: raw web 1.0 time capsule in seconds at this speed, browsing isn't retrieving files from a server, it's querying an alternate reality. a 2D browser window is just step zero. imagine full operating systems, virtual worlds, and complex simulation engines existing purely as model weights. zero gigabytes stored on disk, just pure interactive reality streamed on demand. What else becomes a possibility with the qwen 3.8 27b (dense) at 2000 tokens/sec?

Alok

1,696,262 Aufrufe • vor 1 Monat

Operating System powered by Qwen 3.8 27B at 1950 tokens/sec! here is what 1,950 tokens/second Qwen's 3.8 27b actually looks like on Cerebras: i wrote a minimal python web server that turns cerebras inference into a live operating system. zero apps on disk. when you double click an icon: 1) python proxies a raw SSE stream from qwen 27b at 1,950 tok/s 2) calculator compiles & mounts in 11s 3) full canvas paint studio with brush engine compiles in 10s. at 2,000 tokens/second, software is just an on demand hallucination that runs instantly. the model weights ARE the operating system runtime. what else would you build at 1,950 tokens/second?
1:00

Operating System powered by Qwen 3.8 27B at 1950 tokens/sec! here is what 1,950 tokens/second Qwen's 3.8 27b actually looks like on Cerebras: i wrote a minimal python web server that turns cerebras inference into a live operating system. zero apps on disk. when you double click an icon: 1) python proxies a raw SSE stream from qwen 27b at 1,950 tok/s 2) calculator compiles & mounts in 11s 3) full canvas paint studio with brush engine compiles in 10s. at 2,000 tokens/second, software is just an on demand hallucination that runs instantly. the model weights ARE the operating system runtime. what else would you build at 1,950 tokens/second?

Alok

453,919 Aufrufe • vor 1 Monat

Four weeks. From concept to protected generative AI. See how the United States Golf Association (USGA) brought Rules AI to golfers—and how Cisco AI Defense helps secure the AI behind it.
1:42

Four weeks. From concept to protected generative AI. See how the United States Golf Association (USGA) brought Rules AI to golfers—and how Cisco AI Defense helps secure the AI behind it.

Cisco AI

5,379,444 Aufrufe • vor 2 Monaten

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.
0:22

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

NVIDIA AI

212,322 Aufrufe • vor 3 Monaten

Nemotron 3 Full Breakdown With the help of Joey Conway from NVIDIA AI getting into the specifics around why Nemotron 3 is kind of a big deal Biggest headline with Nemotron is: Hybrid Mamba Transformer, Latent MoE, and MTP Hybrid Mamba Transformer essentially attacks right at the Attention mechanism to make the overhead sub-quadratic, but unlike quantizing KV Cache or swapping out attention head, NVIDIA chose Mamba-2 Latent MoE helps further optimize on sparsity by down projecting the dimensions so you're doing less math and less memory movement between HBM and SRAM, you're saving a ton, and NVIDIA made a conscious choice to add more experts given the surplus Finally, MTP or multi token prediction where the model can see future tokens to be more expressive in training and also option to use for speculative decoding during inference Oh, also the model adopts the new OpenMDW 1.1 License
12:11

Nemotron 3 Full Breakdown With the help of Joey Conway from NVIDIA AI getting into the specifics around why Nemotron 3 is kind of a big deal Biggest headline with Nemotron is: Hybrid Mamba Transformer, Latent MoE, and MTP Hybrid Mamba Transformer essentially attacks right at the Attention mechanism to make the overhead sub-quadratic, but unlike quantizing KV Cache or swapping out attention head, NVIDIA chose Mamba-2 Latent MoE helps further optimize on sparsity by down projecting the dimensions so you're doing less math and less memory movement between HBM and SRAM, you're saving a ton, and NVIDIA made a conscious choice to add more experts given the surplus Finally, MTP or multi token prediction where the model can see future tokens to be more expressive in training and also option to use for speculative decoding during inference Oh, also the model adopts the new OpenMDW 1.1 License

Caleb Eom

50,467 Aufrufe • vor 4 Monaten

It's out: Meeting Recorder 1.1 It finds speakers with NVIDIA Nemotron 3 Diarization: on two real meetings, the wrong-speaker rate went from 17% and 54% to 2% and 6%. What?!
0:30

It's out: Meeting Recorder 1.1 It finds speakers with NVIDIA Nemotron 3 Diarization: on two real meetings, the wrong-speaker rate went from 17% and 54% to 2% and 6%. What?!

Jankees

26,684 Aufrufe • vor 24 Tagen

See how teams at dbt Labs went from information overload to instant insights using the new Notion AI 👉
0:30

See how teams at dbt Labs went from information overload to instant insights using the new Notion AI 👉

Notion

22,358 Aufrufe • vor 2 Jahren

If you see it, you can be it 🥹 Coco Gauff really went from the stands to thanking her own fans at the US Open 👏
0:25

If you see it, you can be it 🥹 Coco Gauff really went from the stands to thanking her own fans at the US Open 👏

espnW

16,100 Aufrufe • vor 1 Monat

“There's two reasons why we're building Nemotron. The first is because it helps us build the GPUs and systems for the future.” The second reason is because trust is the foundation of impactful AI–and it starts with understanding. That’s why Nemotron was built open source at its core—so developers, researchers, and enterprises can build together. Hear from Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, on why making AI open benefits everyone. 👉
1:01

“There's two reasons why we're building Nemotron. The first is because it helps us build the GPUs and systems for the future.” The second reason is because trust is the foundation of impactful AI–and it starts with understanding. That’s why Nemotron was built open source at its core—so developers, researchers, and enterprises can build together. Hear from Bryan Catanzaro, VP of Applied Deep Learning Research at NVIDIA, on why making AI open benefits everyone. 👉

NVIDIA

64,904 Aufrufe • vor 1 Jahr

ABSOLUTE MILLSAGE-nema. OUR NOTES YOU COOKED AGAIN. Honestly from this I can really see the difference between the debut from Roselia, Aglow, and more to now. Its so beautiful, I love how far they went.
2:03

ABSOLUTE MILLSAGE-nema. OUR NOTES YOU COOKED AGAIN. Honestly from this I can really see the difference between the debut from Roselia, Aglow, and more to now. Its so beautiful, I love how far they went.

Daily Our Notes | Awacrashnote.id

20,477 Aufrufe • vor 3 Monaten

Introducing XPU Grasshopper, an AI chip co-designed with AI for the era of recursive self-improvement It went from its first tokens to 816x faster in 13 weeks Pre-order the FPGA devkit:
0:34

Introducing XPU Grasshopper, an AI chip co-designed with AI for the era of recursive self-improvement It went from its first tokens to 816x faster in 13 weeks Pre-order the FPGA devkit:

Elias

107,025 Aufrufe • vor 11 Tagen

Figuring out a data center’s carbon footprint means piecing together a lot of messy data, from parts lists and documents to supplier emails. Sluicebox built an AI supplier agent called Lucy to help. Using NVIDIA Nemotron 3 Ultra, Lucy collects and checks supplier data so engineers can understand the impact of their choices while they’re still designing. In Sluicebox’s tests, Nemotron improved accuracy, cut costs by 51–80% and delivered responses up to 2x faster. See how it works:
0:40

Figuring out a data center’s carbon footprint means piecing together a lot of messy data, from parts lists and documents to supplier emails. Sluicebox built an AI supplier agent called Lucy to help. Using NVIDIA Nemotron 3 Ultra, Lucy collects and checks supplier data so engineers can understand the impact of their choices while they’re still designing. In Sluicebox’s tests, Nemotron improved accuracy, cut costs by 51–80% and delivered responses up to 2x faster. See how it works:

NVIDIA AI

18,347 Aufrufe • vor 26 Tagen

Very quick comparison between Ornith 1.5 35B MoE and Qwen 3.8 27B Dense. 📣 Clearly it's not a fair one, but let's in any case see how it went. 397B download in progress! In the Videos below: - Brick SA -> Qwen 3.8 - Lego Streets -> Ornith 1.5 Context: - Pi agent used in both cases, prompt below - M3 Ultra 512GB - Ornith-1.5-35B-A3B-oQ8e MTP hosted on oMLX - mlx-community/Qwen3.8-27B-8bit DFlash 2 hosted on mlx-dspark - Ornith has been incredibly fast with speed from 75 t/s to 45 t/s (160K+ context) - Qwen3.8 suffered context much more reaching 5 t/s above 160K context, but it can be due to engine tested still work in progress - Prompt: using threejs, and cdn, create a lego like game that's inspired by gta san andreas, with beautiful aesthetics and graphics and ability to steal cars. it should be 3d and have a nice and large map and areas. the graphics should be decent and nice. it should feature iconic things from gta san andreas Notes: - Qwen 3.8 result is 0-shot, while Ornith 1.5 required 6 iterations - Ornith 1.5 is not at the same level of autonomy as Qwen 3.8 27B honestly. To try getting the same results I'm constantly nudging, steering and 🤬 at it. - Qwen 3.8 took 6 hours to complete, but it was more a problem of timeout of Chrome headless used for testing, real prefill/decode TBD. I'll test again now with oMLX - Pi agent has a nearly perfect Cache Hit ratio that for local models is MEGA important!
0:31

Very quick comparison between Ornith 1.5 35B MoE and Qwen 3.8 27B Dense. 📣 Clearly it's not a fair one, but let's in any case see how it went. 397B download in progress! In the Videos below: - Brick SA -> Qwen 3.8 - Lego Streets -> Ornith 1.5 Context: - Pi agent used in both cases, prompt below - M3 Ultra 512GB - Ornith-1.5-35B-A3B-oQ8e MTP hosted on oMLX - mlx-community/Qwen3.8-27B-8bit DFlash 2 hosted on mlx-dspark - Ornith has been incredibly fast with speed from 75 t/s to 45 t/s (160K+ context) - Qwen3.8 suffered context much more reaching 5 t/s above 160K context, but it can be due to engine tested still work in progress - Prompt: using threejs, and cdn, create a lego like game that's inspired by gta san andreas, with beautiful aesthetics and graphics and ability to steal cars. it should be 3d and have a nice and large map and areas. the graphics should be decent and nice. it should feature iconic things from gta san andreas Notes: - Qwen 3.8 result is 0-shot, while Ornith 1.5 required 6 iterations - Ornith 1.5 is not at the same level of autonomy as Qwen 3.8 27B honestly. To try getting the same results I'm constantly nudging, steering and 🤬 at it. - Qwen 3.8 took 6 hours to complete, but it was more a problem of timeout of Chrome headless used for testing, real prefill/decode TBD. I'll test again now with oMLX - Pi agent has a nearly perfect Cache Hit ratio that for local models is MEGA important!

Ivan Fioravanti ᯅ

12,150 Aufrufe • vor 1 Monat

i tested Qwen3.8 27B locally on my M4 Pro to see how it behaves as a coding agent inside my project showing step-by-step the full guide on how you can run it, perfect for your first local model 00:00 Why I’m testing Qwen3.8 00:49 What Qwen3.8 includes 00:59 Qwen3.8 Max vs 27B 01:14 Qwen3.8 Max explained 02:13 Qwen’s benchmark claims 02:31 Its autonomous coding project 03:18 Qwen3.8 27B explained 04:30 Why I’m testing it locally 05:05 Download Qwen 3.8 in LM Studio 06:00 Pick the Q4 model 06:41 Run it as a local server 07:33 How fast is it? 08:36 Set up OpenCode 09:33 Connect LM Studio to OpenCode 10:03 Add Qwen 3.8 to OpenCode 11:34 Match the context settings 15:00 Give Qwen a real coding task 15:49 A mistake I caught live: session compaction 18:00 Review the code 19:30 My verdict
21:02

i tested Qwen3.8 27B locally on my M4 Pro to see how it behaves as a coding agent inside my project showing step-by-step the full guide on how you can run it, perfect for your first local model 00:00 Why I’m testing Qwen3.8 00:49 What Qwen3.8 includes 00:59 Qwen3.8 Max vs 27B 01:14 Qwen3.8 Max explained 02:13 Qwen’s benchmark claims 02:31 Its autonomous coding project 03:18 Qwen3.8 27B explained 04:30 Why I’m testing it locally 05:05 Download Qwen 3.8 in LM Studio 06:00 Pick the Q4 model 06:41 Run it as a local server 07:33 How fast is it? 08:36 Set up OpenCode 09:33 Connect LM Studio to OpenCode 10:03 Add Qwen 3.8 to OpenCode 11:34 Match the context settings 15:00 Give Qwen a real coding task 15:49 A mistake I caught live: session compaction 18:00 Review the code 19:30 My verdict

Nidhi Singh

43,169 Aufrufe • vor 2 Monaten

Every big MoE model I test on my 3090 is basically to see how good it is compared to Qwen3.6-27B. "How good is it compared to Qwen3.6-27B?" is the question I get asked the most, and that shows how much people love and, more importantly, trust the 27B 27B is battle tested. It has proven its worth and has nothing left to prove. Now it just needs to make way for the next 27B. Here’s an example video: a Three.js task, same prompt, not oneshot but two shots for all models. All ran on my 3090 + 64GB VRAM Qwen-3.6-27b fully on vram while others have experts offloaded to system ram. Pinned on the left: Qwen3.6-27B-Q5 On the right, i'm switching between Inkling-Small-Q4_K_M DSV4-Flash-0731-UD-IQ1 Ling-3.0-Flash-Q4_K_M Qwen3.6-27B nailed it. The effects and visuals are just spot on.The other models got some things right, but not everything. Second place would have to go to Inkling-Small while DSV4-Flash-0731 and Ling-3.0-Flash failed to capture the full requirements. This is also why we’re all excited for Qwen Qwen3.8-27B!
2:20

Every big MoE model I test on my 3090 is basically to see how good it is compared to Qwen3.6-27B. "How good is it compared to Qwen3.6-27B?" is the question I get asked the most, and that shows how much people love and, more importantly, trust the 27B 27B is battle tested. It has proven its worth and has nothing left to prove. Now it just needs to make way for the next 27B. Here’s an example video: a Three.js task, same prompt, not oneshot but two shots for all models. All ran on my 3090 + 64GB VRAM Qwen-3.6-27b fully on vram while others have experts offloaded to system ram. Pinned on the left: Qwen3.6-27B-Q5 On the right, i'm switching between Inkling-Small-Q4_K_M DSV4-Flash-0731-UD-IQ1 Ling-3.0-Flash-Q4_K_M Qwen3.6-27B nailed it. The effects and visuals are just spot on.The other models got some things right, but not everything. Second place would have to go to Inkling-Small while DSV4-Flash-0731 and Ling-3.0-Flash failed to capture the full requirements. This is also why we’re all excited for Qwen Qwen3.8-27B!

AJ

11,670 Aufrufe • vor 2 Monaten

THIS ENGINEER RUNS CLAUDE CODE WITHOUT PAYING ANTHROPIC A SINGLE DOLLAR RTX 5090 under the desk, Qwen 3.5 35B, 140 tokens per second Claude Code pointed at localhost instead of Anthropic’s servers override two environment variables and your API bill drops to zero he built a full-stack Next.js app from scratch to prove it works then showed every bug and limitation other YouTubers hide same setup I run with two GPUs and Qwen 3.6 27B full breakdown ↓
16:33

THIS ENGINEER RUNS CLAUDE CODE WITHOUT PAYING ANTHROPIC A SINGLE DOLLAR RTX 5090 under the desk, Qwen 3.5 35B, 140 tokens per second Claude Code pointed at localhost instead of Anthropic’s servers override two environment variables and your API bill drops to zero he built a full-stack Next.js app from scratch to prove it works then showed every bug and limitation other YouTubers hide same setup I run with two GPUs and Qwen 3.6 27B full breakdown ↓

leopardracer

29,756 Aufrufe • vor 5 Monaten