Загрузка видео...

Не удалось загрузить видео

На главную

A catalogue is useless if you can’t run it. ⬟ The Playground is executable. Models are live, not listed. 100k+ models available for real inference. No local setup required. Browse less. Execute more.

15,378 просмотров • 7 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.

Alok

62,631 просмотров • 1 месяц назад

SEVEN RTX 3090S IN A WATER TANK FOR AI SERVER it is a private AI server with the power bill moved into your room. not a clean Mac mini. not a quiet box under a monitor. loose vertical GPUs sit inside a transparent tank. bubbles rise through distilled water. ALLIED CONTROL is printed on the side. it looks closer to a lab accident than a normal workstation. but the logic is obvious: seven RTX 3090s = seven 24GB cards. that is the used-market shortcut for people who want local inference without paying cloud tax on every run. put Ollama, llama.cpp, vLLM, Open WebUI, Tailscale, Qwen, DeepSeek, or Llama on top. now the box can handle client files, code agents, scraping jobs, evals, transcription, and boring overnight work. not because it beats frontier cloud models. because it changes the bill shape. no rate limit. no per-token anxiety. no sensitive client context leaving the building. no monthly stack quietly turning into rent. the ugly part is physical. seven 3090s can pull serious power, dump serious heat, and punish lazy cooling. distilled water is the weird visual, not a setup tip. real immersion rigs live or die on coolant chemistry, insulation, pumps, maintenance, and whether the room can handle the heat. local AI PCs are becoming less like gaming builds and more like small private data centers. the early question is not: can it run ChatGPT? it is: what work is repetitive, private, expensive in the cloud, and worth owning in hardware?

kocer

591,487 просмотров • 1 месяц назад