Loading video...

Video Failed to Load

Go Home

10x speedup: OpenAI's privacy-filter on MLX vs CPU, side by side in your browser. 1,818 tok/s vs 179 tok/s. Real-time PII redaction across English, German, French. On-device. No cloud. Privacy-first. Which model should we port next?

MaziyarPanahi's profile picture

Maziyar PANAHI

17,734 subscribers

25,329 views • 2 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

OpenAI's privacy-filter, retrained on NVIDIA's Nemotron data. PII Masking leaderboard: → openai/privacy-filter: #10 → privacy-filter-nemotron: #4 → OpenMed-PII-SuperClinical: #1, #2 Six places gained from retraining. Demo on web + iPhone:
0:19

OpenAI's privacy-filter, retrained on NVIDIA's Nemotron data. PII Masking leaderboard: → openai/privacy-filter: #10 → privacy-filter-nemotron: #4 → OpenMed-PII-SuperClinical: #1, #2 Six places gained from retraining. Demo on web + iPhone:

Maziyar PANAHI

18,173 views • 2 months ago

Side by side comparison of Apple’s on-device foundation model iPhone 16 Pro vs 17 Pro ~55 tk/s for the 16 Pro and ~85 tk/s for the 17 Pro
0:18

Side by side comparison of Apple’s on-device foundation model iPhone 16 Pro vs 17 Pro ~55 tk/s for the 16 Pro and ~85 tk/s for the 17 Pro

Adrien Grondin

899,918 views • 9 months ago

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳
0:14

MiniMax M2.5 casually responding at ~50 tok/s with MLX (M3 Ultra). The model was released one hour ago 🥳

Pedro Cuenca

170,493 views • 5 months ago

Azguard wallet is now LIVE on Aztec testnet - the native wallet built for onchain privacy. Available as browser extension including side panel Privacy revolution is live on Azguard
0:14

Azguard wallet is now LIVE on Aztec testnet - the native wallet built for onchain privacy. Available as browser extension including side panel Privacy revolution is live on Azguard

Azguard Wallet for Aztec Network

18,826 views • 1 year ago

Demo of 1-bit Bonsai 8B from PrismML running on-device on iPhone 17 Pro More than 40tk/s for a dense 8B model on iPhone, that’s a first Powered by Apple MLX and available now in Locally AI
0:15

Demo of 1-bit Bonsai 8B from PrismML running on-device on iPhone 17 Pro More than 40tk/s for a dense 8B model on iPhone, that’s a first Powered by Apple MLX and available now in Locally AI

Adrien Grondin

160,059 views • 3 months ago

Qwen3.6 26b is local AGI. Macbook pro m5 24gb ram. Backend: mlx-lm Tok/s: ~15
0:11

Qwen3.6 26b is local AGI. Macbook pro m5 24gb ram. Backend: mlx-lm Tok/s: ~15

Marco Franzon

69,220 views • 26 days ago

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!
0:13

NVIDIA just dropped Nemotron-3-Nano:4b — a tiny 2.8GB model. Guess whose hardware runs it the fastest? - RTX 4090: 226 tok/s - RTX 3090: 187 tok/s - Mac Studio M2 Ultra: 86 tok/s - Mac Mini M4: 25 tok/s Home court advantage is real. Also trying a new layout with live performance charts. Lmk what you think!

stevibe

127,408 views • 3 months ago

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.
0:13

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.

Nat Friedman

1,764,082 views • 3 years ago

llama 3 70b beamed to my phone from my M1 Max ~7.6 tok/s with mlx. your own little gpt-4 at home
0:17

llama 3 70b beamed to my phone from my M1 Max ~7.6 tok/s with mlx. your own little gpt-4 at home

Aaron Ng

1,116,956 views • 2 years ago

Three Cline agents. Same 120B model. One rule: terminate your opponents' processes before they terminate yours. We tested raw inference across an NVIDIA AI PC DGX Spark, an RTX 4090, and cloud. Time-to-first-token decides the race. Throughput decides who ships. DGX Spark: 42.9 tok/s. RTX 4090: 8.7. Same weights. 4.9x gap. #AIonRTX
0:13

Three Cline agents. Same 120B model. One rule: terminate your opponents' processes before they terminate yours. We tested raw inference across an NVIDIA AI PC DGX Spark, an RTX 4090, and cloud. Time-to-first-token decides the race. Throughput decides who ships. DGX Spark: 42.9 tok/s. RTX 4090: 8.7. Same weights. 4.9x gap. #AIonRTX

Cline

14,782 views • 3 months ago

2017 vs 2027 Which side is your fav?
0:14

2017 vs 2027 Which side is your fav?

Gusti Ayu

68,322 views • 6 months ago

HK vs VK! Which side do you choose? Descendants: #WickedWonderland premieres July 16 on Disney Channel and next day on Disney+.
0:14

HK vs VK! Which side do you choose? Descendants: #WickedWonderland premieres July 16 on Disney Channel and next day on Disney+.

Descendants 5 Wicked Wonderland Updates

12,720 views • 1 day ago

Just dropped on HF — NeuTTS Air Next-gen on-device TTS that matches cloud-level quality while staying fully open source. > Real-time speech synthesis on CPU/GPU > 3-second voice cloning, no cloud or data upload > Compact: under 200 MB, runs on mobile and edge devices > Multilingual and expressive > Developed by Neuphonic , optimized for speed and fidelity
0:13

Just dropped on HF — NeuTTS Air Next-gen on-device TTS that matches cloud-level quality while staying fully open source. > Real-time speech synthesis on CPU/GPU > 3-second voice cloning, no cloud or data upload > Compact: under 200 MB, runs on mobile and edge devices > Multilingual and expressive > Developed by Neuphonic , optimized for speed and fidelity

steven

72,273 views • 9 months ago

Advice for AI engineers 💡 A small Audio Language Model transcribes audio in real-time on your device, without internet connection or cloud costs. For example, LFM2-Audio-1.5B by ↓
0:14

Advice for AI engineers 💡 A small Audio Language Model transcribes audio in real-time on your device, without internet connection or cloud costs. For example, LFM2-Audio-1.5B by ↓

Pau Labarta Bajo

31,993 views • 7 months ago

Run Llama 3.1 1B offline at 60+ tok/s. Entirely on your phone. Siri can’t save you offline, but a local model just might.
0:14

Run Llama 3.1 1B offline at 60+ tok/s. Entirely on your phone. Siri can’t save you offline, but a local model just might.

Aaron Ng

154,576 views • 1 year ago

Same text. Two privacy filters. OpenAI's model catches 8 categories. OpenMed catches 55+: medical record numbers, blood type, API keys, financial codes, demographics. Trained on Nemotron data by Nvidia. All on-device. All open-source. Coming soon! What's missing?
0:14

Same text. Two privacy filters. OpenAI's model catches 8 categories. OpenMed catches 55+: medical record numbers, blood type, API keys, financial codes, demographics. Trained on Nemotron data by Nvidia. All on-device. All open-source. Coming soon! What's missing?

Maziyar PANAHI

122,140 views • 2 months ago

Apple wasn’t kidding, the iPhone 17 Pro is really built for running LLMs Here’s LFM2 8B A1B by running on-device with MLX in Locally AI - Local AI Chat, the iPhone runs the 8B model with zero struggle Thanks Prince Canuma for the port to MLX, it made the MLX Swift port possible
0:17

Apple wasn’t kidding, the iPhone 17 Pro is really built for running LLMs Here’s LFM2 8B A1B by running on-device with MLX in Locally AI - Local AI Chat, the iPhone runs the 8B model with zero struggle Thanks Prince Canuma for the port to MLX, it made the MLX Swift port possible

Adrien Grondin

493,469 views • 9 months ago