Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon

adrgrondin's profile picture

Adrien Grondin

8,516 subscribers

1,045,563 görüntüleme • 6 ay önce •via X (Twitter)

Sağlık & İyilikEğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Google's Gemma 4 26B A4B running on MacBook Pro M5 Max 100% native Swift Achieving over 100tk/s with Apple MLX framework
0:14

Google's Gemma 4 26B A4B running on MacBook Pro M5 Max 100% native Swift Achieving over 100tk/s with Apple MLX framework

Adrien Grondin

351,153 görüntüleme • 5 ay önce

Demo of 1-bit Bonsai 8B from PrismML running on-device on iPhone 17 Pro More than 40tk/s for a dense 8B model on iPhone, that’s a first Powered by Apple MLX and available now in Locally AI
0:15

Demo of 1-bit Bonsai 8B from PrismML running on-device on iPhone 17 Pro More than 40tk/s for a dense 8B model on iPhone, that’s a first Powered by Apple MLX and available now in Locally AI

Adrien Grondin

161,153 görüntüleme • 6 ay önce

Apple wasn’t kidding, the iPhone 17 Pro is really built for running LLMs Here’s LFM2 8B A1B by running on-device with MLX in Locally AI - Local AI Chat, the iPhone runs the 8B model with zero struggle Thanks Prince Canuma for the port to MLX, it made the MLX Swift port possible
0:17

Apple wasn’t kidding, the iPhone 17 Pro is really built for running LLMs Here’s LFM2 8B A1B by running on-device with MLX in Locally AI - Local AI Chat, the iPhone runs the 8B model with zero struggle Thanks Prince Canuma for the port to MLX, it made the MLX Swift port possible

Adrien Grondin

493,561 görüntüleme • 1 yıl önce

Here’s Apple’s foundation model running on iPhone 17 Pro It’s just so fast Apple was not joking, the A19 Pro chip is really good for running LLMs on-device
0:11

Here’s Apple’s foundation model running on iPhone 17 Pro It’s just so fast Apple was not joking, the A19 Pro chip is really good for running LLMs on-device

Adrien Grondin

1,566,031 görüntüleme • 1 yıl önce

With iOS 27 Apple introduced its most powerful on-device model: AFM 3 Core Advanced It’s a 20 billion parameters model activating only between 1 to 4 billion parameters depending on the request Here’s the model running on iPhone 17 Pro
0:15

With iOS 27 Apple introduced its most powerful on-device model: AFM 3 Core Advanced It’s a 20 billion parameters model activating only between 1 to 4 billion parameters depending on the request Here’s the model running on iPhone 17 Pro

Adrien Grondin

472,118 görüntüleme • 22 gün önce

Claude fable 5 is cooked Claude fable 5 Vs Gemma 4 26b a4b qat MTP Gemma 4 26b running locally on my 8GB vram single RTX 4060 built this using three.js in a single session and 3 prompts. no cloud, no subscription 100% private unlimited use. how long until it can catch up completely?
0:19

Claude fable 5 is cooked Claude fable 5 Vs Gemma 4 26b a4b qat MTP Gemma 4 26b running locally on my 8GB vram single RTX 4060 built this using three.js in a single session and 3 prompts. no cloud, no subscription 100% private unlimited use. how long until it can catch up completely?

Alok

30,802 görüntüleme • 3 ay önce

Meet Gemma 3n, a model that runs on as little as 2GB of RAM 🤯 It shares the same architecture as Gemini Nano, and is engineered for incredible performance. We added audio understanding, so now it’s multimodal, fast and lean, and runs on-device (no cloud connection required!)
0:15

Meet Gemma 3n, a model that runs on as little as 2GB of RAM 🤯 It shares the same architecture as Gemini Nano, and is engineered for incredible performance. We added audio understanding, so now it’s multimodal, fast and lean, and runs on-device (no cloud connection required!)

Google AI

228,990 görüntüleme • 1 yıl önce

Gemma 4 analyzes the video. Generates key questions. Calls Falcon Perception. "Find all the people." 156 found. "Detect only white cars." 8 found. A 26B model is running agentic multi-QA vision orchestration. The models are running locally on a MacBook with MLX. No API.
0:16

Gemma 4 analyzes the video. Generates key questions. Calls Falcon Perception. "Find all the people." 156 found. "Detect only white cars." 8 found. A 26B model is running agentic multi-QA vision orchestration. The models are running locally on a MacBook with MLX. No API.

Maziyar PANAHI

159,049 görüntüleme • 5 ay önce

Gemma 4 just dropped. I had it captioning video in real-time within an hour. Running locally on a MacBook. No cloud. No API. Real-time scene understanding. Oh and SAM3 is segmenting every object in the same frame. Same laptop.
0:16

Gemma 4 just dropped. I had it captioning video in real-time within an hour. Running locally on a MacBook. No cloud. No API. Real-time scene understanding. Oh and SAM3 is segmenting every object in the same frame. Same laptop.

Maziyar PANAHI

197,303 görüntüleme • 6 ay önce

APPLE JUST GOT CAUGHT IN 4K💀 Everyone is roasting Apple for removing the new Photographic Styles from older iPhone. Yet iPhone Air on iOS 27.2 is running the SAME Photographic Style as the iPhone 18 Pro Max. Hardware limitation? Or just another Apple paywall? 👀
0:18

APPLE JUST GOT CAUGHT IN 4K💀 Everyone is roasting Apple for removing the new Photographic Styles from older iPhone. Yet iPhone Air on iOS 27.2 is running the SAME Photographic Style as the iPhone 18 Pro Max. Hardware limitation? Or just another Apple paywall? 👀

Yash

76,882 görüntüleme • 14 gün önce

Introducing MLX-Swift-TS An SDK for running time series foundation models fully on-device on Apple Silicon. When I joined Datadog, Inc. , I was introduced to Toto, our time series foundation model, and got excited about zero-shot forecasting across different domains. While building a health copilot app, I realized there wasn’t a simple way to run models like these locally on device. So I built one. MLX-Swift-TS exposes a common TimeSeriesForecaster interface for loading and running multiple time series architectures directly in Swift using MLX. No server required. The attached video shows on-device forecasting running inside a native Swift app. Huge thanks to Awni Hannun and the MLX team for building MLX and its Swift API, Prince Canuma for inspiration on MLX SDK patterns, and Ameet Talwalkar and the Datadog team for Toto.
0:17

Introducing MLX-Swift-TS An SDK for running time series foundation models fully on-device on Apple Silicon. When I joined Datadog, Inc. , I was introduced to Toto, our time series foundation model, and got excited about zero-shot forecasting across different domains. While building a health copilot app, I realized there wasn’t a simple way to run models like these locally on device. So I built one. MLX-Swift-TS exposes a common TimeSeriesForecaster interface for loading and running multiple time series architectures directly in Swift using MLX. No server required. The attached video shows on-device forecasting running inside a native Swift app. Huge thanks to Awni Hannun and the MLX team for building MLX and its Swift API, Prince Canuma for inspiration on MLX SDK patterns, and Ameet Talwalkar and the Datadog team for Toto.

Kunal Batra

12,585 görüntüleme • 7 ay önce

Gemma 4 is here! Our most intelligent open models to date, are built on the same world-class research and tech as Gemini 3, and are sized to run and fine-tune efficiently on local hardware. Check out what Google Gemma 4 brings to devs: 💎 Advanced Reasoning: Deep logic tasks, complex multi-step planning, and beyond 💎 Longer context: Seamlessly analyze entire codebases with context windows of 128K tokens for our edge models and 256K tokens for our largest models 💎 Vision and audio: Rich, multimodal interactions out of the box 💎 140+ languages: Trained on 140+ languages 💎 Apache 2.0 license: industry-standard open-source license
0:19

Gemma 4 is here! Our most intelligent open models to date, are built on the same world-class research and tech as Gemini 3, and are sized to run and fine-tune efficiently on local hardware. Check out what Google Gemma 4 brings to devs: 💎 Advanced Reasoning: Deep logic tasks, complex multi-step planning, and beyond 💎 Longer context: Seamlessly analyze entire codebases with context windows of 128K tokens for our edge models and 256K tokens for our largest models 💎 Vision and audio: Rich, multimodal interactions out of the box 💎 140+ languages: Trained on 140+ languages 💎 Apache 2.0 license: industry-standard open-source license

Google for Developers

270,290 görüntüleme • 6 ay önce

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,724 görüntüleme • 1 yıl önce

Gemma 4 looks at a parking lot. Decides what to ask. Calls SAM 3.1. "Segment all vehicles." 64 found. "Now just the white ones." 23 found. One model reasoning and orchestrating. One model executing. Both running locally on a MacBook. MLX. No cloud. No API.
0:14

Gemma 4 looks at a parking lot. Decides what to ask. Calls SAM 3.1. "Segment all vehicles." 64 found. "Now just the white ones." 23 found. One model reasoning and orchestrating. One model executing. Both running locally on a MacBook. MLX. No cloud. No API.

Maziyar PANAHI

594,966 görüntüleme • 6 ay önce

Run Gemma 4 26b MTP on 8 GB VRAM GPUs at 25+ tokens/second. Flags included! local llm space is moving at terminal velocity. only 3 days ago google released gemma 4 26b a4b qat quants. more efficient than before, ran on 8gb vram at 20 tok/sec. and now just a few hours ago, mainline llama.cpp merged a massive update and we just shattered our own record. decode throughput went 25-40% up on the same 8 GB VRAM setup! Before MTP: 20 tps -> After MTP: 28 tps! llama.cpp just officially merged PR #23398 ("add Gemma4 MTP"), bringing native Multi-Token Prediction (MTP) support to Gemma 4 models. By running speculative drafting on the same 8GB VRAM RTX 4060 setup, my decode throughput on a 64k context instantly leaped to a blistering 25–27 tokens/sec thats 25-30% increase with the same hardware. Here is the architectural catch you need to know: Unlike the Qwen 3.5 and 3.6 series, which bake the MTP heads directly into the base GGUF, the Gemma 4 MTP head is not built in. You must download a separate, specialized MTP drafter GGUF (the assistant model) to act as the speculator. (I've dropped the download link in the replies). copy and try the exact flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --spec-type draft-mtp --spec-draft-n-max 6 --spec-draft-p-min 0.7 --spec-draft-model gemma-4-26b-A4B-it-assistant-Q4_0.gguf -c 64000 -v n-max 4 and p-min 0.7 is also worth checking out. benchmark on your setup and workflow. if you have a single 8 gb vram nvidia rtx 4060, 3060, 3070, 2080, 2070, grab the MTP drafter GGUF link in the comments and try it yourself. Check it out even if you have asmaller or a larger gpu, such as a single rtx 3090, 4090, 3060, 2060. MTP works for all gemma 4 sizes such as gemma 4 12b, gemma 4 31b etc. but remember to grab the correct mtp draft assistant models respectively. what are you benchmarking today
0:15

Run Gemma 4 26b MTP on 8 GB VRAM GPUs at 25+ tokens/second. Flags included! local llm space is moving at terminal velocity. only 3 days ago google released gemma 4 26b a4b qat quants. more efficient than before, ran on 8gb vram at 20 tok/sec. and now just a few hours ago, mainline llama.cpp merged a massive update and we just shattered our own record. decode throughput went 25-40% up on the same 8 GB VRAM setup! Before MTP: 20 tps -> After MTP: 28 tps! llama.cpp just officially merged PR #23398 ("add Gemma4 MTP"), bringing native Multi-Token Prediction (MTP) support to Gemma 4 models. By running speculative drafting on the same 8GB VRAM RTX 4060 setup, my decode throughput on a 64k context instantly leaped to a blistering 25–27 tokens/sec thats 25-30% increase with the same hardware. Here is the architectural catch you need to know: Unlike the Qwen 3.5 and 3.6 series, which bake the MTP heads directly into the base GGUF, the Gemma 4 MTP head is not built in. You must download a separate, specialized MTP drafter GGUF (the assistant model) to act as the speculator. (I've dropped the download link in the replies). copy and try the exact flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --spec-type draft-mtp --spec-draft-n-max 6 --spec-draft-p-min 0.7 --spec-draft-model gemma-4-26b-A4B-it-assistant-Q4_0.gguf -c 64000 -v n-max 4 and p-min 0.7 is also worth checking out. benchmark on your setup and workflow. if you have a single 8 gb vram nvidia rtx 4060, 3060, 3070, 2080, 2070, grab the MTP drafter GGUF link in the comments and try it yourself. Check it out even if you have asmaller or a larger gpu, such as a single rtx 3090, 4090, 3060, 2060. MTP works for all gemma 4 sizes such as gemma 4 12b, gemma 4 31b etc. but remember to grab the correct mtp draft assistant models respectively. what are you benchmarking today

Alok

200,913 görüntüleme • 4 ay önce

I wanted to try Maple-Preview by DeepGrove on iPhone so I simply ported the model to MLX Swift using Bionic Maple runs at ~80tk/s on iPhone 17 Pro with my quick port, it’s a 20B-A1B model at 2-bit The model is close to Qwen3.5 35-A2B in many benchs and runs easily on iPhone
0:15

I wanted to try Maple-Preview by DeepGrove on iPhone so I simply ported the model to MLX Swift using Bionic Maple runs at ~80tk/s on iPhone 17 Pro with my quick port, it’s a 20B-A1B model at 2-bit The model is close to Qwen3.5 35-A2B in many benchs and runs easily on iPhone

Adrien Grondin

13,376 görüntüleme • 1 ay önce

Wdym it stopped charging in a properly chilled AC room with the ceiling fan running at full speed 😭 Stuff like this is why I will never consider an iPhone as my primary device (atleast until they fix this). (And this is the iPhone 17 Pro, the one with a vapour chamber 🥀)
0:12

Wdym it stopped charging in a properly chilled AC room with the ceiling fan running at full speed 😭 Stuff like this is why I will never consider an iPhone as my primary device (atleast until they fix this). (And this is the iPhone 17 Pro, the one with a vapour chamber 🥀)

Aryan Gupta

143,730 görüntüleme • 3 ay önce

As promised, instead of for $15,000 GPUs, we max optimized a model that can fit comfortably in any GPU with 8GB VRAM running Windows 200 tok/s, *128k context, text/image/audio. Anyone who wants to join the local AI community with 5 year old laptop can run this with recipe below, and this test was done on 3070ti laptop I forgot I had. We pulled every instruct model released since May that fits 8 or 12 GB, and ran the six that mattered through the same 20 tasks: JSON extraction, code, a table, arithmetic, a fake-treaty honesty check, tool calls, language, a persona, a fact planted deep in a long prompt, and three photos. Gemma 4 E4B, Gemma 4 12B, Qwen3.5-9B, Qwen3.5-4B, Ornith-1.5-9B, the Qwen3.8-9B distill. E4B is not the top scorer. Qwen3.5-9B and Ornith beat it on long context and on reading fine print in a photo. It won the pick for the things a newcomer on an old card actually needs: Apache 2.0, text + image + audio in, 3 GB of headroom on an 8 GB card, Weights Google trained to be 4-bit (QAT), and Google's own speculative drafter shipped next to them. Nothing else in the tier has all five, and the drafter alone is worth 2–3×.
0:10

As promised, instead of for $15,000 GPUs, we max optimized a model that can fit comfortably in any GPU with 8GB VRAM running Windows 200 tok/s, *128k context, text/image/audio. Anyone who wants to join the local AI community with 5 year old laptop can run this with recipe below, and this test was done on 3070ti laptop I forgot I had. We pulled every instruct model released since May that fits 8 or 12 GB, and ran the six that mattered through the same 20 tasks: JSON extraction, code, a table, arithmetic, a fake-treaty honesty check, tool calls, language, a persona, a fact planted deep in a long prompt, and three photos. Gemma 4 E4B, Gemma 4 12B, Qwen3.5-9B, Qwen3.5-4B, Ornith-1.5-9B, the Qwen3.8-9B distill. E4B is not the top scorer. Qwen3.5-9B and Ornith beat it on long context and on reading fine print in a photo. It won the pick for the things a newcomer on an old card actually needs: Apache 2.0, text + image + audio in, 3 GB of headroom on an 8 GB card, Weights Google trained to be 4-bit (QAT), and Google's own speculative drafter shipped next to them. Nothing else in the tier has all five, and the drafter alone is worth 2–3×.

Killy

127,634 görüntüleme • 1 ay önce

Iceman has now spent 27 consecutive days at #1 on Apple Music USA, and is 1 day from entering the top 17 longest running #1 albums of all time on the platform in the US🧊
0:19

Iceman has now spent 27 consecutive days at #1 on Apple Music USA, and is 1 day from entering the top 17 longest running #1 albums of all time on the platform in the US🧊

Akademiks TV

18,100 görüntüleme • 3 ay önce