Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

I got a fully maxed out MacBook, mostly so I can run local models fast. And omg here is mixtral running on ollama... almost cannot believe how fast it is. This is with no internet!! A model that beats GPT-3.5 running locally! What!

Charlie Holtz

35,043 subscribers

196,086 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von sanket patel

sanket patelvor 2 Jahren

@ollama Will it keep my lap nice and warm

Profilbild von Charlie Holtz

Charlie Holtzvor 2 Jahren

@ollama this is actually how @anotherjesse heats his office in the winter (not even joking)

Profilbild von UNPLUGGED PERFORMANCE

UNPLUGGED PERFORMANCEvor 2 Jahren

Get superior performance with Unplugged Performance's UP-05 forged wheels. Available for all S, 3, X, and Y Tesla Models.

Profilbild von Tim Carambat

Tim Carambatvor 2 Jahren

@ollama Now plug that into AnythingLLM and get the speed + vector database, and a lot more in a single package. Supports any @ollama model! I need to get an M3, my intel with Ollama just ain't cutting it.

Profilbild von Vikr

Vikrvor 2 Jahren

@ollama could you do /set verbose i want know how much tokens/s did M3 got

Profilbild von Charlie Holtz

Charlie Holtzvor 2 Jahren

@ollama 36 t/s!

Profilbild von Josh Marino

Josh Marinovor 2 Jahren

@ollama That just means robots running local neural networks will be here far faster than people expect! Especially since AI chips are advancing faster than printer tech did in the 90’s!

Profilbild von Josh Whiton

Josh Whitonvor 2 Jahren

@DanielleFong @ollama I feel like this is one of the only reasons anyone is getting a fully maxed out MacBook these days.

Profilbild von Mbongeni Ndlovu

Mbongeni Ndlovuvor 2 Jahren

@ollama That’s why I maxed out my MacBook too

Profilbild von Unemployed Capital Allocator

Unemployed Capital Allocatorvor 2 Jahren

@ollama How does it do at ingesting longer prompts? If you put in half the context window as prompt, is it much slower?

Profilbild von Charlie Holtz

Charlie Holtzvor 2 Jahren

@ollama interesting question! just tested with ~2000 token input and it still was very fast, 31 tokens/second:

Ähnliche Videos

Qwen 3.6 35B running at over 100 tokens per second on my $5,399 MacBook Pro M5 Max. This is the best local AI model I have ever run. 128GB of unified memory. No cloud. No API costs. No rate limits. Just raw local inference at speeds I didn't think were possible on a laptop. This model is more intelligent than GPT 5 on benchmarks. Running locally. On a MacBook. For free after the hardware cost. I said local AI would never compete with frontier. I'm starting to rethink that. The gap is closing faster than anyone expected.

Qwen 3.6 35B running at over 100 tokens per second on my $5,399 MacBook Pro M5 Max. This is the best local AI model I have ever run. 128GB of unified memory. No cloud. No API costs. No rate limits. Just raw local inference at speeds I didn't think were possible on a laptop. This model is more intelligent than GPT 5 on benchmarks. Running locally. On a MacBook. For free after the hardware cost. I said local AI would never compete with frontier. I'm starting to rethink that. The gap is closing faster than anyone expected.

BridgeMind

62,800 Aufrufe • vor 3 Monaten

Gemma 4 running on my iPhone works without internet, is blazing fast and can translate Japanese from a pill bottle. Local AI models running on a phone feels like magic.

Gemma 4 running on my iPhone works without internet, is blazing fast and can translate Japanese from a pill bottle. Local AI models running on a phone feels like magic.

NetworkChuck

556,557 Aufrufe • vor 5 Monaten

I'm shocked by how easy it is to run PydanticAI with a model running logically. Here we have: ollama running llama3.2 locally, being used by Pydantic AI extracting structured data from an input.

I'm shocked by how easy it is to run PydanticAI with a model running logically. Here we have: ollama running llama3.2 locally, being used by Pydantic AI extracting structured data from an input.

Samuel Colvin

18,084 Aufrufe • vor 1 Jahr

Last week I spent $20,000 on Mac Studios so I could run OpenClaw locally Fully powered by local AI models running 24/7 Now my agent: • Is doing tasks 24/7/365 • Doesn't cost any tokens • Is fully private In this video I cover how it works and how you can do the same thing:

Last week I spent $20,000 on Mac Studios so I could run OpenClaw locally Fully powered by local AI models running 24/7 Now my agent: • Is doing tasks 24/7/365 • Doesn't cost any tokens • Is fully private In this video I cover how it works and how you can do the same thing:

Alex Finn

573,816 Aufrufe • vor 6 Monaten

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀

Daniel San

101,567 Aufrufe • vor 2 Jahren

Behold... GPT-OSS (20B) running 100% locally in your browser on WebGPU. This shouldn't be possible — but with Transformers.js v4 and ONNX Runtime Web, it is! A new class of AI apps is emerging. Zero-install, infinite distribution. Simply visit a website and run models locally.

Behold... GPT-OSS (20B) running 100% locally in your browser on WebGPU. This shouldn't be possible — but with Transformers.js v4 and ONNX Runtime Web, it is! A new class of AI apps is emerging. Zero-install, infinite distribution. Simply visit a website and run models locally.

Xenova

311,512 Aufrufe • vor 6 Monaten

Qwen3.5-35B-A3B running locally on an M4 chip at 49.5 tokens per second. A 35B model. On a laptop. In real time. LOCAL AI IS GETTING SCARY FAST.

Qwen3.5-35B-A3B running locally on an M4 chip at 49.5 tokens per second. A 35B model. On a laptop. In real time. LOCAL AI IS GETTING SCARY FAST.

0xMarioNawfal

478,363 Aufrufe • vor 6 Monaten

I’ve been using the new MacBook Pro M5 since launch and it’s a beast, especially compared to my old M1! It’s good for running the smaller AI models locally but also everything you do is just so fast. Here’s Gemma 3 QAT 12B running with MLX:

I’ve been using the new MacBook Pro M5 since launch and it’s a beast, especially compared to my old M1! It’s good for running the smaller AI models locally but also everything you do is just so fast. Here’s Gemma 3 QAT 12B running with MLX:

Adrien Grondin

101,950 Aufrufe • vor 9 Monaten

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,558,786 Aufrufe • vor 6 Monaten

I seriously cannot believe this is a 0.6B LLM! 🤯 Qwen just released Qwen3, a series of hybrid reasoning models that allow you to control how much "thinking" the model does for a given task. They can even run locally in your browser on WebGPU with 🤗 Transformers.js!

I seriously cannot believe this is a 0.6B LLM! 🤯 Qwen just released Qwen3, a series of hybrid reasoning models that allow you to control how much "thinking" the model does for a given task. They can even run locally in your browser on WebGPU with 🤗 Transformers.js!

Xenova

74,954 Aufrufe • vor 1 Jahr

The value produced by models is getting so much better so fast that old hardware is actually getting *more* expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

The value produced by models is getting so much better so fast that old hardware is actually getting more expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

Dwarkesh Patel

78,277 Aufrufe • vor 5 Monaten

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇

Daniel San

584,100 Aufrufe • vor 1 Jahr

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

BridgeMind

57,304 Aufrufe • vor 4 Monaten

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,211 Aufrufe • vor 1 Jahr

This is the world that the Trump-Musk regime intends to advance so fast on the Chinese model with Musk's billionaire mate David Sacks running AI policy. This is what the MAGA hijack is really all about and my goodness how so many have fallen for it. Time to wake up FAST. Excellent video.

This is the world that the Trump-Musk regime intends to advance so fast on the Chinese model with Musk's billionaire mate David Sacks running AI policy. This is what the MAGA hijack is really all about and my goodness how so many have fallen for it. Time to wake up FAST. Excellent video.

David Icke

507,204 Aufrufe • vor 1 Jahr

Most people think a 0.8B model is a toy. 3rd test. This time: OCR I fed Qwen 3.5 0.8B three images: a Kindle page, my terrible handwriting x 2. Mostly correct. Running locally. From a model smaller than your browser tab.

Most people think a 0.8B model is a toy. 3rd test. This time: OCR I fed Qwen 3.5 0.8B three images: a Kindle page, my terrible handwriting x 2. Mostly correct. Running locally. From a model smaller than your browser tab.

stevibe

125,292 Aufrufe • vor 6 Monaten

Small language models give me warm, fuzzy feelings! Google dropped this 270M model (Gemma), and it’s absolutely wild how coherent and fast it is on my phone. Locally installed, chatting without internet (PocketPal app).

Small language models give me warm, fuzzy feelings! Google dropped this 270M model (Gemma), and it’s absolutely wild how coherent and fast it is on my phone. Locally installed, chatting without internet (PocketPal app).

Paras Chopra

60,437 Aufrufe • vor 1 Jahr