Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool... show more

Tanmay

7,012 subscribers

11,682 Aufrufe • vor 3 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

785,842 Aufrufe • vor 6 Monaten

Run Orpheus — the state-of-the-art TTS — on your local PC! Orpheus-FastAPI is a high-performance TTS app with OpenAI-compatible API, running efficiently with LM Studio backend. And Death wrote this 1-click launcher for us! Cross platform (Mac, Linux, Windows)

Run Orpheus — the state-of-the-art TTS — on your local PC! Orpheus-FastAPI is a high-performance TTS app with OpenAI-compatible API, running efficiently with LM Studio backend. And Death wrote this 1-click launcher for us! Cross platform (Mac, Linux, Windows)

cocktail peanut

30,930 Aufrufe • vor 1 Jahr

i spent the afternoon experimenting with Gemma 4's vision capabilities made an app that uses roboflow RF-DETR for a first pass of object detections and Gemma to summarize the scene in one sentence for fun i asked Gemma to "describe what you see as if you were a medieval bard" all made with free local AI models (running via webgpu in the browser thanks to transformers js) lots more possibilities to explore with this

i spent the afternoon experimenting with Gemma 4's vision capabilities made an app that uses roboflow RF-DETR for a first pass of object detections and Gemma to summarize the scene in one sentence for fun i asked Gemma to "describe what you see as if you were a medieval bard" all made with free local AI models (running via webgpu in the browser thanks to transformers js) lots more possibilities to explore with this

AA

701,613 Aufrufe • vor 6 Monaten

Personal Computer is now available to all users in a new Perplexity Mac app. Personal Computer is an advanced version of Perplexity Computer. It operates on any Mac, running tasks across your local files, native Mac apps, the web, and Perplexity’s secure servers.

Personal Computer is now available to all users in a new Perplexity Mac app. Personal Computer is an advanced version of Perplexity Computer. It operates on any Mac, running tasks across your local files, native Mac apps, the web, and Perplexity’s secure servers.

Perplexity

384,613 Aufrufe • vor 4 Monaten

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Prince Canuma

82,349 Aufrufe • vor 5 Monaten

$I just crammed the updated Gemma 4 26B A4B QAT (MoE) with 180k context into an 8GB RTX 4060 (8 GB VRAM + 16 GB RAM only!!) and optimized the batch size. 23 tokens/sec decode, 300 tokens/sec prefill Yesterday I showed you a Gemma 4 31B dense model running flawlessly on an RTX 4090. Today, we're breaking the VRAM bank on a budget card using Unsloth’s new Gemma 4 26B (A4B) QAT quants. Following Google’s chat template update that boosted agentic benchmarks by +10%, I pushed this model to its absolute limits. Here is how you squeeze 250k context out of 8GB of VRAM. # The Setup & The Optimization - Hardware: Nvidia RTX 4060 (8GB VRAM) + 16GB System RAM - Environment: CUDA 13.0 build of llama.cpp - Model: gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf - Prompt: 28,000 tokens of prompt for each run If you read my L2 cache breakdown (attached in replies), you know the 4060’s 24MB cache maxes out at `-b 1024 -ub 1024`. Push past that, and prefill crashes. I locked those flags in for every test below to ensure maximum GEMM throughput. # 1. The Raw Context Push (Unquantized KV Cache) First, I wanted to see how far pure 8GB VRAM + 16GB RAM could stretch without touching the KV cache: - 80k Context: Prefill 385 t/s | Decode 25.5 t/s - 120k Context: Prefill 270 t/s | Decode 24 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 120000 --port 8080 -ub 1024 -b 1024 Without KV quantization, 120k is your hard ceiling. push past that prefill throughput drops off a cliff, making the model practically unusable for large agentic workloads. # 2. The Q8 KV Cache Lifeline To survive 250k context on a budget card, you have to quantize the KV cache. I enabled 8 bit KV cache (`-ctk q8_0 -ctv q8_0`) and re ran: - 180k Context: Prefill 280 t/s | Decode 22.8 t/s - 250k Context: Prefill 115 t/s | Decode 20 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 180000 --port 8080 -b 1024 -ub 1024 -ctk q8_0 -ctv q8_0 Result: Q8 KV cache brings 250k context back from the dead. Decode speed stabilizes at a highly usable 20 t/s. You are trading a very small bit amount of reasoning precision for an extra 130,000 tokens of context window. if you own a single rtx 3050, 3060, 3070, 4050, 4060, 5050 or 5060, you must try this model and optimize your batch size for higher prefill. Hugging Face links to the updated Unsloth's QAT quants and performance graph are in the replies below. What model are you running on your 6GB, 8GB or 12GB cards right now? Let's see your setups.$

I just crammed the updated Gemma 4 26B A4B QAT (MoE) with 180k context into an 8GB RTX 4060 (8 GB VRAM + 16 GB RAM only!!) and optimized the batch size. 23 tokens/sec decode, 300 tokens/sec prefill Yesterday I showed you a Gemma 4 31B dense model running flawlessly on an RTX 4090. Today, we're breaking the VRAM bank on a budget card using Unsloth’s new Gemma 4 26B (A4B) QAT quants. Following Google’s chat template update that boosted agentic benchmarks by +10%, I pushed this model to its absolute limits. Here is how you squeeze 250k context out of 8GB of VRAM. # The Setup & The Optimization - Hardware: Nvidia RTX 4060 (8GB VRAM) + 16GB System RAM - Environment: CUDA 13.0 build of llama.cpp - Model: gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf - Prompt: 28,000 tokens of prompt for each run If you read my L2 cache breakdown (attached in replies), you know the 4060’s 24MB cache maxes out at `-b 1024 -ub 1024`. Push past that, and prefill crashes. I locked those flags in for every test below to ensure maximum GEMM throughput. # 1. The Raw Context Push (Unquantized KV Cache) First, I wanted to see how far pure 8GB VRAM + 16GB RAM could stretch without touching the KV cache: - 80k Context: Prefill 385 t/s | Decode 25.5 t/s - 120k Context: Prefill 270 t/s | Decode 24 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 120000 --port 8080 -ub 1024 -b 1024 Without KV quantization, 120k is your hard ceiling. push past that prefill throughput drops off a cliff, making the model practically unusable for large agentic workloads. # 2. The Q8 KV Cache Lifeline To survive 250k context on a budget card, you have to quantize the KV cache. I enabled 8 bit KV cache (`-ctk q8_0 -ctv q8_0`) and re ran: - 180k Context: Prefill 280 t/s | Decode 22.8 t/s - 250k Context: Prefill 115 t/s | Decode 20 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 180000 --port 8080 -b 1024 -ub 1024 -ctk q8_0 -ctv q8_0 Result: Q8 KV cache brings 250k context back from the dead. Decode speed stabilizes at a highly usable 20 t/s. You are trading a very small bit amount of reasoning precision for an extra 130,000 tokens of context window. if you own a single rtx 3050, 3060, 3070, 4050, 4060, 5050 or 5060, you must try this model and optimize your batch size for higher prefill. Hugging Face links to the updated Unsloth's QAT quants and performance graph are in the replies below. What model are you running on your 6GB, 8GB or 12GB cards right now? Let's see your setups.

Alok

36,617 Aufrufe • vor 2 Monaten

Google's Gemma 4 26B A4B QAT hits 25+ tokens/sec and 320+ tokens/sec prefill on 8 GB VRAM (RTX 4060) + 16 GB RAM using TurboQuant Prefill just went from 200 → 320+ tok/s on the same 8GB card. 1.6x, no new hardware, no new quant, just a KV cache trick stacked on top of the Gemma 4 26B MoE setup from a few days ago. A few days ago I posted Gemma 4 26B A4B hitting 28 tok/s decode on 8GB VRAM using native MTP. prefill was stuck around 200 tok/s. fair callout by the community. So today I tested something I'd already been meaning to try: TheTom/llama-cpp-turboquant, the TurboQuant KV cache fork by Tom Turney (Tom Turney). (github link in the comments) thanks to him, the fork just got resynced to mainline, so MTP + TurboQuant now run together cleanly (I didnt see any meaningful gains by using MTP with this setup though but you can try). The flags (No MTP): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -cnv -c 64000 --cache-type-k q8_0 --cache-type-v turbo3 Results on the same RTX 4060 8GB, tested with a 27k token prompt at 64k context loaded: Prefill: 200 tok/s → 320+ tok/s Decode: stayed above 25 tok/s (without MTP) Why it works: TurboQuant uses walsh hadamard rotation + polar quantization on the KV cache. keys are sensitive to compression, values aren't much, so it splits the difference: K stays at q8_0, V drops to turbo3 (~3 bits). bonus from the memory savings: same 8GB card can now stretch to 100-120k context with minimal decode penalty. It should now be snappier with any agent harness such as hermes agent without compromise on intelligence. If you're already running Gemma 4 on a small card, this stacks on top for free. Try --cache-type-k q8_0 --cache-type-v turbo3 on your setup and report back what your prefill/decode split looks like. unsloth model gguf and llama.cpp turboquant fork links in the comments. what's your prefill number before vs after?

Google's Gemma 4 26B A4B QAT hits 25+ tokens/sec and 320+ tokens/sec prefill on 8 GB VRAM (RTX 4060) + 16 GB RAM using TurboQuant Prefill just went from 200 → 320+ tok/s on the same 8GB card. 1.6x, no new hardware, no new quant, just a KV cache trick stacked on top of the Gemma 4 26B MoE setup from a few days ago. A few days ago I posted Gemma 4 26B A4B hitting 28 tok/s decode on 8GB VRAM using native MTP. prefill was stuck around 200 tok/s. fair callout by the community. So today I tested something I'd already been meaning to try: TheTom/llama-cpp-turboquant, the TurboQuant KV cache fork by Tom Turney (Tom Turney). (github link in the comments) thanks to him, the fork just got resynced to mainline, so MTP + TurboQuant now run together cleanly (I didnt see any meaningful gains by using MTP with this setup though but you can try). The flags (No MTP): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -cnv -c 64000 --cache-type-k q8_0 --cache-type-v turbo3 Results on the same RTX 4060 8GB, tested with a 27k token prompt at 64k context loaded: Prefill: 200 tok/s → 320+ tok/s Decode: stayed above 25 tok/s (without MTP) Why it works: TurboQuant uses walsh hadamard rotation + polar quantization on the KV cache. keys are sensitive to compression, values aren't much, so it splits the difference: K stays at q8_0, V drops to turbo3 (~3 bits). bonus from the memory savings: same 8GB card can now stretch to 100-120k context with minimal decode penalty. It should now be snappier with any agent harness such as hermes agent without compromise on intelligence. If you're already running Gemma 4 on a small card, this stacks on top for free. Try --cache-type-k q8_0 --cache-type-v turbo3 on your setup and report back what your prefill/decode split looks like. unsloth model gguf and llama.cpp turboquant fork links in the comments. what's your prefill number before vs after?

Alok

119,821 Aufrufe • vor 3 Monaten

The iPad's external display support is underrated. I connected my iPad to a Studio Display and ran Workbench. Now my headless Mac mini (back in Minneapolis) is running full-screen on a 27" display in Ohio. Remote Mac on a Studio Display through an iPad. Pretty wild. 🤯

The iPad's external display support is underrated. I connected my iPad to a Studio Display and ran Workbench. Now my headless Mac mini (back in Minneapolis) is running full-screen on a 27" display in Ohio. Remote Mac on a Studio Display through an iPad. Pretty wild. 🤯

Matt Ronge

64,918 Aufrufe • vor 5 Monaten

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

Alok

62,381 Aufrufe • vor 3 Monaten

If you’re running AI agents on a headless Mac mini, being able to check in from an iPad is ridiculously useful. Here’s demo of using my iPad with Astropad Workbench to connect to my Mac Mini

If you’re running AI agents on a headless Mac mini, being able to check in from an iPad is ridiculously useful. Here’s demo of using my iPad with Astropad Workbench to connect to my Mac Mini

Matt Ronge

21,435 Aufrufe • vor 1 Monat

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Splash is here! A month ago, DFlash 2 ran Qwen3.8-27B at 70 tok/s on an M5 Max. Today, Splash runs it at 144 tok/s on the same Mac ⚡ Getting that speed used to take work. Splash is built around the model, with the DFlash 2 draft already in it. Two commands and you're running.

Zhijian Liu

92,333 Aufrufe • vor 12 Tagen

What happens when you put Gemma 4 into the hands of developers across 20+ countries? So much innovation! Like this app that turns an image prompt into a working web app in seconds, running on-device! Want to get involved? Join or host an event!

What happens when you put Gemma 4 into the hands of developers across 20+ countries? So much innovation! Like this app that turns an image prompt into a working web app in seconds, running on-device! Want to get involved? Join or host an event!

Google Gemma

37,158 Aufrufe • vor 2 Monaten

Another example of the multiple TTS parallel pipelines pattern. Here's a voice AI agent that speaks both English and Arabic, using a specific model/voice for each language. These are PlayAI voices. The STT, TTS, and LLM inference is all running on GroqInc. (The LLM is LLama 4 Maverick.)

Another example of the multiple TTS parallel pipelines pattern. Here's a voice AI agent that speaks both English and Arabic, using a specific model/voice for each language. These are PlayAI voices. The STT, TTS, and LLM inference is all running on GroqInc. (The LLM is LLama 4 Maverick.)

kwindla

13,538 Aufrufe • vor 1 Jahr

Let's be honest Claude Code was not working good with local LLMs. So, I Tried using OpenCode with LM Studio using Qwen 3.5 9B model on my RTX 5080 GPU. It's very usable and much faster than Claude Code. I'm actually planning to use this setup to build an entire vibe coded SaaS

Let's be honest Claude Code was not working good with local LLMs. So, I Tried using OpenCode with LM Studio using Qwen 3.5 9B model on my RTX 5080 GPU. It's very usable and much faster than Claude Code. I'm actually planning to use this setup to build an entire vibe coded SaaS

Tushar Verma

137,948 Aufrufe • vor 6 Monaten

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2

Nidhi Singh

16,289 Aufrufe • vor 3 Monaten

I freaked out when my WiFi router suddenly died. then realized my autonomous Hermes agent is running fully local, nothing stopped. Hermes Agent + Gemma 4 26B A4B QAT MoE, 100% local on my laptop, building my side projects while I scroll my phone zero API calls. zero cost. 100% private. fully offline. This might be the most satisfying thing I’ve watched in a while. last post: showed Hermes + local Gemma 4 26B pull off backtest a trading strategy. this time I asked it to develop something i'd use myself everyday: # A full unpacked extension with: - React side panel UI - Local llama.cpp backend (offline AI) - Live tab sync + status tracking - Auto context extraction via Readability.js Vision on Demand → captures viewport screenshots as compressed JPEGs Deterministic action system -> model outputs tokens -> directly controls page scrolling It planned everything first. Then started executing step by step. all i did was say 'ok'. only once. # What’s wild: - It reports back after every phase - Auto compresses context when nearing limits - Actualy, stays on track llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 --cache-type-k q8_0 --cache-type-v q8_0 --port 8080 # Performance on a single NVIDIA RTX 4060 (8GB VRAM) + 16 GB DDR4 RAM Gaming Laptop: - 300 tokens/sec prefill - 25+ tokens/sec decode More than usable for real dev workflows. This isn’t AI demo territory anymore. This is autonomous local software actually building things.

I freaked out when my WiFi router suddenly died. then realized my autonomous Hermes agent is running fully local, nothing stopped. Hermes Agent + Gemma 4 26B A4B QAT MoE, 100% local on my laptop, building my side projects while I scroll my phone zero API calls. zero cost. 100% private. fully offline. This might be the most satisfying thing I’ve watched in a while. last post: showed Hermes + local Gemma 4 26B pull off backtest a trading strategy. this time I asked it to develop something i'd use myself everyday: # A full unpacked extension with: - React side panel UI - Local llama.cpp backend (offline AI) - Live tab sync + status tracking - Auto context extraction via Readability.js Vision on Demand → captures viewport screenshots as compressed JPEGs Deterministic action system -> model outputs tokens -> directly controls page scrolling It planned everything first. Then started executing step by step. all i did was say 'ok'. only once. # What’s wild: - It reports back after every phase - Auto compresses context when nearing limits - Actualy, stays on track llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 --cache-type-k q8_0 --cache-type-v q8_0 --port 8080 # Performance on a single NVIDIA RTX 4060 (8GB VRAM) + 16 GB DDR4 RAM Gaming Laptop: - 300 tokens/sec prefill - 25+ tokens/sec decode More than usable for real dev workflows. This isn’t AI demo territory anymore. This is autonomous local software actually building things.

Alok

56,717 Aufrufe • vor 3 Monaten

Vibe coded this dotnet uninstall tool to help me manage the many version of dotnet I have on my Mac using a single file C# app and Spectre. #dotnet

Vibe coded this dotnet uninstall tool to help me manage the many version of dotnet I have on my Mac using a single file C# app and Spectre. #dotnet

David Fowler

17,723 Aufrufe • vor 1 Jahr

VERY COOL!! Thanks jordan for the tip, I can now test my iOS app on web I asked Claude Code to set up serve-sim, a web-based iOS simulator that can stream the iOS app my Claude Code on VPS built live to me, because it's on some Mac Mini in the cloud without even a GUI (it's headless) I cannot actually try it on the Mac itself! Before Claude would make me a page with screenshots but I had no idea how the app felt, now with this it works This is hosted on the Mac Mini via SSH (it shows localhost but it's a tunnel via SSH to the Mac Mini), bit laggy but nice!

VERY COOL!! Thanks jordan for the tip, I can now test my iOS app on web I asked Claude Code to set up serve-sim, a web-based iOS simulator that can stream the iOS app my Claude Code on VPS built live to me, because it's on some Mac Mini in the cloud without even a GUI (it's headless) I cannot actually try it on the Mac itself! Before Claude would make me a page with screenshots but I had no idea how the app felt, now with this it works This is hosted on the Mac Mini via SSH (it shows localhost but it's a tunnel via SSH to the Mac Mini), bit laggy but nice!

@levelsio

429,509 Aufrufe • vor 2 Monaten

Here's a first look at Android's upcoming all-in-one Terminal app, which downloads, configures, runs, and interfaces with an instance of Debian running a virtual machine! Currently, the Debian images are hosted on a Googler's GitHub, but Google plans to host these images themselves soon. For more info on this Terminal app, see the replies👇 (Thanks to an anonymous dev for sharing this!)

Here's a first look at Android's upcoming all-in-one Terminal app, which downloads, configures, runs, and interfaces with an instance of Debian running a virtual machine! Currently, the Debian images are hosted on a Googler's GitHub, but Google plans to host these images themselves soon. For more info on this Terminal app, see the replies👇 (Thanks to an anonymous dev for sharing this!)

Mishaal Rahman

54,662 Aufrufe • vor 1 Jahr