Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

First steps for a specialized DeepSeek v4 Flash inference engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.

antirez's profile picture

antirez

64,264 subscribers

14,155 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

DeepSeek DeepSeek is now live on Heurist. Open source AI just reached new heights. Access DeepSeek-R1 and DeepSeek-V3 for smarter inference at a fraction of cost. Try it via Heurist API or explore on Pondera.
0:19

DeepSeek DeepSeek is now live on Heurist. Open source AI just reached new heights. Access DeepSeek-R1 and DeepSeek-V3 for smarter inference at a fraction of cost. Try it via Heurist API or explore on Pondera.

Heurist

25,657 Aufrufe • vor 1 Jahr

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)
0:14

AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)

Alex Cheema

1,934,459 Aufrufe • vor 1 Jahr

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.
0:14

Running DeepSeek R1 on my desk Uses EXO Labs with Thunderbolt 5 interconnect (80Gbps) to run the full (671B, 8-bit) DeepSeek R1 distributed across 2 M3 Ultra 512GB Mac Studios (1TB total Unified Memory). Runs at 11 tok/sec. Theoretical max is ~20 tok/sec.

Alex Cheema

992,062 Aufrufe • vor 1 Jahr

Batching for vision models is now available in Beta with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mode, choose the beta runtime channel, and select LM Studio MLX v1.8.1.
0:11

Batching for vision models is now available in Beta with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mode, choose the beta runtime channel, and select LM Studio MLX v1.8.1.

LM Studio

46,015 Aufrufe • vor 25 Tagen

Nemotron 3 Ultra is fast and genuinely good Compared it with 3 frontier models: DeepSeek V4, MiniMax M3, and Qwen 3.7 Max on 2 prompts very impressive results
0:17

Nemotron 3 Ultra is fast and genuinely good Compared it with 3 frontier models: DeepSeek V4, MiniMax M3, and Qwen 3.7 Max on 2 prompts very impressive results

GMI Cloud

219,575 Aufrufe • vor 3 Tagen

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.
0:13

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.

Nat Friedman

1,764,005 Aufrufe • vor 3 Jahren

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema - e/acc

527,894 Aufrufe • vor 1 Jahr

Happy Friday! We just put DeepSeek-V4-Pro up on It’s the world’s largest open source model at 1.6T parameters, and you can run it for free running on NVIDIA Blackwell GPUs. Try the NVIDIA NIM API →
0:18

Happy Friday! We just put DeepSeek-V4-Pro up on It’s the world’s largest open source model at 1.6T parameters, and you can run it for free running on NVIDIA Blackwell GPUs. Try the NVIDIA NIM API →

NVIDIA AI

201,449 Aufrufe • vor 1 Monat

Laika AI x Inference Labs Excited to announce our partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.
0:10

Laika AI x Inference Labs Excited to announce our partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.

Laika AI

13,727 Aufrufe • vor 1 Jahr

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants
0:11

Another demo of the iPhone 17 Pro’s on-device LLM performance This time with Ling mini 2.0 by InclusionAI, a 16B MoE model with 1.4B active parameters running at ~120tk/s Thanks to Awni Hannun for the MLX DWQ 2-bit quants

Adrien Grondin

46,205 Aufrufe • vor 8 Monaten

Introducing a 100% free coding agent with DeepSeek v4 Pro Choose any model, all free: - DeepSeek v4 Pro/Flash - Kimi K2.6 - MiniMax M2.7 npm i -g freebuff
0:18

Introducing a 100% free coding agent with DeepSeek v4 Pro Choose any model, all free: - DeepSeek v4 Pro/Flash - Kimi K2.6 - MiniMax M2.7 npm i -g freebuff

James Grugett

392,085 Aufrufe • vor 27 Tagen

Got continuous batching working with SSMs in mlx-lm. Here's four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).
0:18

Got continuous batching working with SSMs in mlx-lm. Here's four OpenCode agents simultaneously running Nvidia's Nemotron Nano on 64GB M4 Max. This is a nice model for smaller machines since it's MoE + hybrid attention (small cache).

Awni Hannun

35,078 Aufrufe • vor 5 Monaten

Real-time Moondream inference using our new inference engine
0:18

Real-time Moondream inference using our new inference engine

vik

144,409 Aufrufe • vor 2 Monaten

The world’s fastest inference for Llama 4 Scout is now on Poe! At over 2,600 tokens per second, this bot allows for near-instant interactions. (1/2)
0:18

The world’s fastest inference for Llama 4 Scout is now on Poe! At over 2,600 tokens per second, this bot allows for near-instant interactions. (1/2)

Poe

17,286 Aufrufe • vor 1 Jahr

Want to run Deepseek R1 ? Text-generation-inference v3.1.0 is out and supports it out of the box. Both on AMD and Nvidia !
0:14

Want to run Deepseek R1 ? Text-generation-inference v3.1.0 is out and supports it out of the box. Both on AMD and Nvidia !

Nicolas Patry

28,859 Aufrufe • vor 1 Jahr

Today, we're excited to announce a partnership with Manta Network (🔱,🔱), the Modular L2 solution that is transforming the landscape of ZK. Nesa is bringing private AI inference to the Manta ecosystem through a specialized collaboration. For the first time, developers on Manta can access Nesa's full library of AI models on-chain, and enjoy lightning fast, end-to-end private AI inference without ever leaving the Manta ecosystem. This means that dapps and protocols can now fuse with AI via smart contract on Manta. This integration is set to redefine the future of decentralized technology with AI. Stay tuned for more updates on how Nesa and Manta Network will be shaping the future of crypto together.
0:14

Today, we're excited to announce a partnership with Manta Network (🔱,🔱), the Modular L2 solution that is transforming the landscape of ZK. Nesa is bringing private AI inference to the Manta ecosystem through a specialized collaboration. For the first time, developers on Manta can access Nesa's full library of AI models on-chain, and enjoy lightning fast, end-to-end private AI inference without ever leaving the Manta ecosystem. This means that dapps and protocols can now fuse with AI via smart contract on Manta. This integration is set to redefine the future of decentralized technology with AI. Stay tuned for more updates on how Nesa and Manta Network will be shaping the future of crypto together.

Nesa

35,179 Aufrufe • vor 1 Jahr

My AI broke the world record on Tempest yesterday! But I still hold the human record :-) [on Extreme difficulty settings] Here's a little demo reel of the Tempest AI doing inference and training at the same time up on the hardest Tempest levels. This is all running on our Dell Technologies 7875 Workstation, with the 9995WX CPU handling 2000 fps of Tempest while the dual Blackwell RTX6000 GPUs do inference and training.
0:19

My AI broke the world record on Tempest yesterday! But I still hold the human record :-) [on Extreme difficulty settings] Here's a little demo reel of the Tempest AI doing inference and training at the same time up on the hardest Tempest levels. This is all running on our Dell Technologies 7875 Workstation, with the 9995WX CPU handling 2000 fps of Tempest while the dual Blackwell RTX6000 GPUs do inference and training.

Dave W Plummer

37,372 Aufrufe • vor 3 Monaten

Deepseek V4 Flash is now free via Nous Portal for a limited time thanks to Novita AI!
0:11

Deepseek V4 Flash is now free via Nous Portal for a limited time thanks to Novita AI!

Nous Research

516,855 Aufrufe • vor 25 Tagen

1. Model agnostic. 2. Inference agnostic. & now, 3. Platform agnostic. Cline for JetBrains is here. (install it below)
0:15

1. Model agnostic. 2. Inference agnostic. & now, 3. Platform agnostic. Cline for JetBrains is here. (install it below)

Cline

72,472 Aufrufe • vor 8 Monaten

NVIDIA Nemotron 3 Nano Omni, a new multimodal reasoning model, is now live on Jetson AI Lab and unifies vision, audio, and language into a single reasoning loop. 🙌 Power your NemoClaws by running this model with Ollama, vLLM and other inference frameworks on NVIDIA Jetson hardware. Try it ➡️
0:13

NVIDIA Nemotron 3 Nano Omni, a new multimodal reasoning model, is now live on Jetson AI Lab and unifies vision, audio, and language into a single reasoning loop. 🙌 Power your NemoClaws by running this model with Ollama, vLLM and other inference frameworks on NVIDIA Jetson hardware. Try it ➡️

NVIDIA Robotics

15,828 Aufrufe • vor 1 Monat