Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Qwen3.6-27B-Base looping so hard, I've run out of context and scrollback. Left like this for a good 30 minutes wasting electricity on my spendy UK tariff ⚡️ 🤑 Unsloth AI's Q8 UD XL quant of Qwen and llama.cpp - fp16 kv cache - driven by opencode 1/2

Alex Ellis

31,101 subscribers

14,661 просмотров • 18 дней назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Gemma 4 12B QAT (dense) achieves 1000+ tokens/sec prefill on 8GB VRAM with 120k context Gemma 4 12B QAT (dense), TurboQuant (Without MTP), RTX 4060 8GB VRAM: Prefill: 1000+ tok/s (42% increase) Decode: 25+ tok/s (25% increase) Context: 120k (150% increase) prefill was 700 tok/sec and decode 20 tok/sec with only 48k context without turbo quant (older test with mtp link in the comments) llama.cpp TurboQuant flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -c 120000 --cache-type-k q8_0 --cache-type-v turbo3 -ngl 99 --port 8080 tested with a 27k prompt, 120k context loaded. -ngl 99 here isn't a typo, full 12B dense, every layer on GPU, on an 8GB card. that's the part worth sitting with. The model has vision, audio input, thinking/reasoning and fits your 8GB card. TurboQuant's KV cache savings are what free up the room to do that at 120k context. side by side with yesterday: 26B A4B MoE got 320+ tok/s prefill. this dense 12B is clearing 1000+ rig: RTX 4060 8GB · i7H · 16GB RAM same two flags as yesterday, different model size: --cache-type-k q8_0 --cache-type-v turbo3 thanks to TheTom/llama-cpp-turboquant, TurboQuant fork of llama.cpp by Tom Turney (Tom Turney) to make this work. unsloth's model quant huggingface and the llama.cpp fork github link in the comments Do you prefer a dense or a MoE for your 8GB card?

Gemma 4 12B QAT (dense) achieves 1000+ tokens/sec prefill on 8GB VRAM with 120k context Gemma 4 12B QAT (dense), TurboQuant (Without MTP), RTX 4060 8GB VRAM: Prefill: 1000+ tok/s (42% increase) Decode: 25+ tok/s (25% increase) Context: 120k (150% increase) prefill was 700 tok/sec and decode 20 tok/sec with only 48k context without turbo quant (older test with mtp link in the comments) llama.cpp TurboQuant flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -c 120000 --cache-type-k q8_0 --cache-type-v turbo3 -ngl 99 --port 8080 tested with a 27k prompt, 120k context loaded. -ngl 99 here isn't a typo, full 12B dense, every layer on GPU, on an 8GB card. that's the part worth sitting with. The model has vision, audio input, thinking/reasoning and fits your 8GB card. TurboQuant's KV cache savings are what free up the room to do that at 120k context. side by side with yesterday: 26B A4B MoE got 320+ tok/s prefill. this dense 12B is clearing 1000+ rig: RTX 4060 8GB · i7H · 16GB RAM same two flags as yesterday, different model size: --cache-type-k q8_0 --cache-type-v turbo3 thanks to TheTom/llama-cpp-turboquant, TurboQuant fork of llama.cpp by Tom Turney (Tom Turney) to make this work. unsloth's model quant huggingface and the llama.cpp fork github link in the comments Do you prefer a dense or a MoE for your 8GB card?

Alok

34,500 просмотров • 29 дней назад

gemma-4-12B-agentic-fable5-composer2.5 V2 is out. the agentic upgrade to the model trained on Fable 5's reasoning. Running it now with TurboQuant llama.cpp on a single RTX 4060( 8 GB VRAM) at 30 tokens/second with full 25000 context and reasoning: # The benchmarks v2 is built for coding + agentic work. writing code, running commands, using tools, debugging, multi step technical tasks. The clearest signal is tau2 bench telecom, an agentic tool use benchmark whose diagnose → fix → verify loop mirrors real terminal/debugging work: tau2 bench telecom numbers: base Gemma 4 12B: ~15% this finetune: ~55%. (Self reported) thats a huge jump # TheTom/llama-cpp-turboquant flags: llama-server.exe -m gemma4-v2-Q4_K_M.gguf -ngl 99 -c 25000 --cache-type-k q8_0 --cache-type-v turbo3 --port 8080 Flag breakdown: -ngl 99 → full GPU offload -c 25000 → 25K context --cache-type-k q8_0 --cache-type-v turbo3 → mixed-precision KV cache — K at 8-bit, V at ~3-bit via TurboQuant (Walsh Hadamard rotated polar quant, Google's own KV-compression research). Not even merged into mainline llama.cpp. running it off a fork. No API. No cloud. Just llama.cpp. well, a fork of it and any 6gb+ GPU. If you tried yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF, check this out and share your experience with the models

gemma-4-12B-agentic-fable5-composer2.5 V2 is out. the agentic upgrade to the model trained on Fable 5's reasoning. Running it now with TurboQuant llama.cpp on a single RTX 4060( 8 GB VRAM) at 30 tokens/second with full 25000 context and reasoning: # The benchmarks v2 is built for coding + agentic work. writing code, running commands, using tools, debugging, multi step technical tasks. The clearest signal is tau2 bench telecom, an agentic tool use benchmark whose diagnose → fix → verify loop mirrors real terminal/debugging work: tau2 bench telecom numbers: base Gemma 4 12B: ~15% this finetune: ~55%. (Self reported) thats a huge jump # TheTom/llama-cpp-turboquant flags: llama-server.exe -m gemma4-v2-Q4_K_M.gguf -ngl 99 -c 25000 --cache-type-k q8_0 --cache-type-v turbo3 --port 8080 Flag breakdown: -ngl 99 → full GPU offload -c 25000 → 25K context --cache-type-k q8_0 --cache-type-v turbo3 → mixed-precision KV cache — K at 8-bit, V at ~3-bit via TurboQuant (Walsh Hadamard rotated polar quant, Google's own KV-compression research). Not even merged into mainline llama.cpp. running it off a fork. No API. No cloud. Just llama.cpp. well, a fork of it and any 6gb+ GPU. If you tried yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF, check this out and share your experience with the models

Alok

144,654 просмотров • 27 дней назад

you're paying $20/mo for something your $500 GPU can already do. Gemma 4 26B A4B QAT MoE + Hermes Agent running on a single RTX 4060 (8GB VRAM). Built a vision capable, 100% free, 100% local, private AI assistant that lives in my Chrome browser. No API keys. No cloud. No subscriptions. 100% vibe coded. 0% handholding. It has full context of whatever's on my screen can answer questions, summarize pages, extract data, and see images. Same local model handles everything, no external calls, ever. keep reading for the model and hermes agent tips i learnt while building this locally. Here's the exact setup for anyone running local LLMs on 6-8 GB VRAM: llama.cpp server flags (on my NVIDIA RTX 4060 8gb VRAM): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --cache-type-k q8_0 --cache-type-v q8_0 -c 150000 --port 8080 Throughput with quantization: Prefill: 200-250 tokens/sec Decode: 20-25 tokens/sec reduce context if oom on 6 gb vram card. Key learnings: - Quantize KV cache to q8 for faster prefill/decode. Prefill goes from 100-150 (unquantized) to 200-250 tok/s (q8). - But watch out, once actual context grows past ~50k tokens on high entropy workloads, q8 KV quantization can cause hallucinations. Low entropy workloads are mostly unaffected. If you see it happening, drop the quantization. This is common across all local models. - In Hermes Agent settings -> Memory & Context, bump compression threshold from default 0.5 to 0.7. Default triggers way too frequent context compression and eats time. Up next: add persistent memory, web search, tool calling, streaming output and whatever you suggest. Running a 26B MoE with vision + 150k context window on 8GB VRAM would've sounded impossible 6 months ago. Works the same on the NVIDIA RTX 3060 Ti, 3070, 4060 Ti, 5060, 2080, or any 8GB card. VRAM is the only requirement. Local AI agents are closer than people think. You just need to know where the knobs are. Model's Unsloth quant hugging face link in the comments. Have you tried Hermes agent by Nous Research yet? What are you building with local LLMs? Drop it below, let's see what this community is shipping.

you're paying $20/mo for something your $500 GPU can already do. Gemma 4 26B A4B QAT MoE + Hermes Agent running on a single RTX 4060 (8GB VRAM). Built a vision capable, 100% free, 100% local, private AI assistant that lives in my Chrome browser. No API keys. No cloud. No subscriptions. 100% vibe coded. 0% handholding. It has full context of whatever's on my screen can answer questions, summarize pages, extract data, and see images. Same local model handles everything, no external calls, ever. keep reading for the model and hermes agent tips i learnt while building this locally. Here's the exact setup for anyone running local LLMs on 6-8 GB VRAM: llama.cpp server flags (on my NVIDIA RTX 4060 8gb VRAM): -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --cache-type-k q8_0 --cache-type-v q8_0 -c 150000 --port 8080 Throughput with quantization: Prefill: 200-250 tokens/sec Decode: 20-25 tokens/sec reduce context if oom on 6 gb vram card. Key learnings: - Quantize KV cache to q8 for faster prefill/decode. Prefill goes from 100-150 (unquantized) to 200-250 tok/s (q8). - But watch out, once actual context grows past ~50k tokens on high entropy workloads, q8 KV quantization can cause hallucinations. Low entropy workloads are mostly unaffected. If you see it happening, drop the quantization. This is common across all local models. - In Hermes Agent settings -> Memory & Context, bump compression threshold from default 0.5 to 0.7. Default triggers way too frequent context compression and eats time. Up next: add persistent memory, web search, tool calling, streaming output and whatever you suggest. Running a 26B MoE with vision + 150k context window on 8GB VRAM would've sounded impossible 6 months ago. Works the same on the NVIDIA RTX 3060 Ti, 3070, 4060 Ti, 5060, 2080, or any 8GB card. VRAM is the only requirement. Local AI agents are closer than people think. You just need to know where the knobs are. Model's Unsloth quant hugging face link in the comments. Have you tried Hermes agent by Nous Research yet? What are you building with local LLMs? Drop it below, let's see what this community is shipping.

Alok

36,031 просмотров • 15 дней назад

🚨ODD SCORING ALERT🚨 This is from tonight's #RedSox and Washington Nationals game, from the first inning. Here, Curtis Mead is picked off and decides to run way left of second base, and the throw from Willson Contreras hits him. For scoring purposes, this is a pickoff and a stolen base for Mead, because he took off on the pickoff and never stops or hesitates going to second, so even though the throw hits Mead, by rule it has to be a stolen base. Rules wise, this is a legal move by Mead. He isn't out of the base path because you can establish your own path, and if you don't want to run in a straight line, you don't have to (otherwise there would be no rounding of bases). You cannot be out of the basepath until a tag is attempted (and then you get three feet). #NATITUDE #DirtyWater

🚨ODD SCORING ALERT🚨 This is from tonight's #RedSox and Washington Nationals game, from the first inning. Here, Curtis Mead is picked off and decides to run way left of second base, and the throw from Willson Contreras hits him. For scoring purposes, this is a pickoff and a stolen base for Mead, because he took off on the pickoff and never stops or hesitates going to second, so even though the throw hits Mead, by rule it has to be a stolen base. Rules wise, this is a legal move by Mead. He isn't out of the base path because you can establish your own path, and if you don't want to run in a straight line, you don't have to (otherwise there would be no rounding of bases). You cannot be out of the basepath until a tag is attempted (and then you get three feet). #NATITUDE #DirtyWater

MLB Scoring Changes

144,151 просмотров • 17 дней назад

Each week I do two back to back long runs. I’m in the THICK of training for my next 100miler. This week, the 2 back-to-back runs equaled nearly 12hrs of running, 5:45hrs Tuesday and 6hrs Wednesday. I was in Phoenix for Thanksgiving with family so I was able to run in the desert /warm conditions just east of the city - good training for my HURT 100miler in January (on Oahu). Also very technical! Back to back long runs allow me to do two hard endurance efforts on consecutive days without the same body fatigue/impact I’d get from just doing a single day 12hr run. First 2 pics are from Day 1 run, second 2 (pic/video) from day 2. Was definitely tired by the end of day 2. But felt good most the run. Need to dial in my calories better. Had to carry all my water for nearly 6hrs for day 1. Very heavy pack (13 pounds in a Salomon 12L). Day 2 had a water refill halfway through, still good amount of weight carried, but way less than day 1 (took 48oz + calories).

Each week I do two back to back long runs. I’m in the THICK of training for my next 100miler. This week, the 2 back-to-back runs equaled nearly 12hrs of running, 5:45hrs Tuesday and 6hrs Wednesday. I was in Phoenix for Thanksgiving with family so I was able to run in the desert /warm conditions just east of the city - good training for my HURT 100miler in January (on Oahu). Also very technical! Back to back long runs allow me to do two hard endurance efforts on consecutive days without the same body fatigue/impact I’d get from just doing a single day 12hr run. First 2 pics are from Day 1 run, second 2 (pic/video) from day 2. Was definitely tired by the end of day 2. But felt good most the run. Need to dial in my calories better. Had to carry all my water for nearly 6hrs for day 1. Very heavy pack (13 pounds in a Salomon 12L). Day 2 had a water refill halfway through, still good amount of weight carried, but way less than day 1 (took 48oz + calories).

Candice

20,552 просмотров • 1 год назад

Dove Cameron via recent TikTok ❤️‍🩹: “I'm a little drunk right now but i have to say i haven't been this good mentally and emotionally in so long and i feel so lucky and so happy finally and i can't wait to share everything ive been working on with you. The last 2 years have been some of the best years for my mental health, and even if it seems i've been a little MIA, I have not left you. thank you for sticking with me. I love you”

Dove Cameron via recent TikTok ❤️‍🩹: “I'm a little drunk right now but i have to say i haven't been this good mentally and emotionally in so long and i feel so lucky and so happy finally and i can't wait to share everything ive been working on with you. The last 2 years have been some of the best years for my mental health, and even if it seems i've been a little MIA, I have not left you. thank you for sticking with me. I love you”

Dove Cameron Charts

254,083 просмотров • 9 месяцев назад

How to get startup ideas that generate $? 🤔 That's the topic of my new video! 🎬🍿 After watching it, you'll never run out of startup ideas... ✅ And you'll understand WHAT is a good idea 👉 Watch it on YouTube (link in the first comments) I spent over $2000 in gear for this video, so I hope you'll like it 😄

How to get startup ideas that generate $? 🤔 That's the topic of my new video! 🎬🍿 After watching it, you'll never run out of startup ideas... ✅ And you'll understand WHAT is a good idea 👉 Watch it on YouTube (link in the first comments) I spent over $2000 in gear for this video, so I hope you'll like it 😄

Nico

37,506 просмотров • 2 лет назад

I've rewritten this several times, but I can't find any better way to say it... so here it goes. One of the things I'm scared of most, is time. Particularly, time that is wasted. There's a sundial that hangs on my wall with an inscription that says, "I have done nothing good today, I have lost a day." X has always been a place where I wanted to put out things that didn't waste anyone's time. Whatever post I put out, whatever thought, whatever story, comment,... whatever it was, was worth the time for you to read it, and for me to create it. Because I know, in my bones, that the time we have is precious and thought of wasting even a minute of yours or mine doing something worthless, is genuinely one of my greatest fears. As we were walking down this mountain together, my son looks up at me and says, "Dad... God made me VERY good at hiking." I laughed and say, "Yeah he did!" A couple minutes go by and he stops, he turns at looks at me, and goes: "Dad, God made you VERY good at work and work is very important..." That got me. I often worry that my work pulls me too far away from my kids. Even more than that... ...I know that what I do on X does cause and will cause life to be harder for my kids than if I chose to remain silent Walking off that mountain, one of my deepest fears about wasting time was answered by the very person I was afraid I might be hurting. As a dad, that means everything.

I've rewritten this several times, but I can't find any better way to say it... so here it goes. One of the things I'm scared of most, is time. Particularly, time that is wasted. There's a sundial that hangs on my wall with an inscription that says, "I have done nothing good today, I have lost a day." X has always been a place where I wanted to put out things that didn't waste anyone's time. Whatever post I put out, whatever thought, whatever story, comment,... whatever it was, was worth the time for you to read it, and for me to create it. Because I know, in my bones, that the time we have is precious and thought of wasting even a minute of yours or mine doing something worthless, is genuinely one of my greatest fears. As we were walking down this mountain together, my son looks up at me and says, "Dad... God made me VERY good at hiking." I laughed and say, "Yeah he did!" A couple minutes go by and he stops, he turns at looks at me, and goes: "Dad, God made you VERY good at work and work is very important..." That got me. I often worry that my work pulls me too far away from my kids. Even more than that... ...I know that what I do on X does cause and will cause life to be harder for my kids than if I chose to remain silent Walking off that mountain, one of my deepest fears about wasting time was answered by the very person I was afraid I might be hurting. As a dad, that means everything.

Matt Van Swol

100,167 просмотров • 8 месяцев назад

MY REVIEW OF Youknowwazgood Dog shit. All of it smoked funky as hell, I could tell this wasn’t properly grown, dried or cured. After the homie blamed me, my rolling, my papers and my environment for the cause, i knew he was on some bullshit. I left some of the Cranberry Z out to smoke to test any and all theory, shit dried out in less than 2 hours, to dust. I tried to communicate with him, he doesn’t do returns he said, i even offered to fly it out personally, that wouldn’t do. So instead of wasting your money with him or his dumbass reps who are clearly not giving a fuck about the quality or how it misrepresents his brand and all brands for that matter. There’s a few big shops on X that sell his work, if I were you, I’d stay clear, if they sell you boof one time, they’ll do it again💯🫡 But remember, apparently this was my fault and my environment caused this so be sure to slap some reviews up for wherever your jars land. And to the homie Youknowwazgood, shame on you. If you think it ends here, good luck.

Don Hushlelini

30,483 просмотров • 4 месяцев назад

🚨!!Foil Nami Playmat Giveaway!!🚨 We have 9 days left until season 2 of One Piece drops so we would like to giveaway an official One Piece Nami playmat that was customized and foiled out by us. All you have to do is: 1. Follow us 2. Repost 3. Like the Post 4. Optional: Checkout our Dark Magician Girl giveway on TikTok! Sephiroth, Dark Magician Girl, Nami, Vivi, Boa, and Bonney are all available in our shop now. We’ll pick a random winner Sunday 3/8, Good luck to everyone!

🚨!!Foil Nami Playmat Giveaway!!🚨 We have 9 days left until season 2 of One Piece drops so we would like to giveaway an official One Piece Nami playmat that was customized and foiled out by us. All you have to do is: 1. Follow us 2. Repost 3. Like the Post 4. Optional: Checkout our Dark Magician Girl giveway on TikTok! Sephiroth, Dark Magician Girl, Nami, Vivi, Boa, and Bonney are all available in our shop now. We’ll pick a random winner Sunday 3/8, Good luck to everyone!

Matified

43,690 просмотров • 4 месяцев назад

Carol Vorderman, "Never, ever, in my lifetime, have I seen what is patently a corrupt government like this one. Where they are taking from the poor and they are giving to the already rich" CV has authored a book, "Out of order" where she calls out this behaviour, and she's going on tour in September. More at this link: Whether it's the lies of Brexit driven by Boris Johnson, or relaxing Treasury rules and approving £ billions during covid, most of which we now know to be fraud, approved by then Chancellor Rishi Sunak. Our economy is being trashed with Brexit red tape, our country is being robbed by now PM's bad decision making, and we are being sold a NI tax cut as a good thing, when our services like schools and the NHS are both literally crumbling. The Conservative party of 14 years keep saying they have a plan, yet every one of their plans have made things worse for our country. And their constant voting to repeatedly dump raw sewage in our rivers and seas, what a disgrace. Our political landscape is in desperate need of decent people speaking out. A few celebs have had the courage to do so, and no doubt CV's book tour will be a wonderful and welcoming night out. Do check it out 👍

Carol Vorderman, "Never, ever, in my lifetime, have I seen what is patently a corrupt government like this one. Where they are taking from the poor and they are giving to the already rich" CV has authored a book, "Out of order" where she calls out this behaviour, and she's going on tour in September. More at this link: Whether it's the lies of Brexit driven by Boris Johnson, or relaxing Treasury rules and approving £ billions during covid, most of which we now know to be fraud, approved by then Chancellor Rishi Sunak. Our economy is being trashed with Brexit red tape, our country is being robbed by now PM's bad decision making, and we are being sold a NI tax cut as a good thing, when our services like schools and the NHS are both literally crumbling. The Conservative party of 14 years keep saying they have a plan, yet every one of their plans have made things worse for our country. And their constant voting to repeatedly dump raw sewage in our rivers and seas, what a disgrace. Our political landscape is in desperate need of decent people speaking out. A few celebs have had the courage to do so, and no doubt CV's book tour will be a wonderful and welcoming night out. Do check it out 👍

Farrukh

541,553 просмотров • 2 лет назад

LLM Artifacts Connected to Andrej Karpathy's LLM Knowledge base idea, I've been building out a fun way to generate dynamic artifacts from these knowledge bases with the goal of discovering and revealing meaningful and deeper insights. LLM KBs are hard to consume for humans, as I think they are more built for agents. So the question is, what form would be useful for humans to take actions and make important decisions? That's what I am trying to figure out with these artifacts. The artifact example shows a pulse on HN discussions around AI-related stories. The insights can go deeper, of course, but this is already super fun and thought-provoking, like some of my favorite podcasts. The format and depth matter a lot. The aggregation skills of agents are outstanding if you tune the prompts and skill carefully. I built this artifact generator in a few minutes through an agent skill, but I feel like there are so many ways that LLM-generated information can be used and consumed. Like generating deeper insights and analysis, and things that are just not feasible for humans today. The generated artifact (including its data and design) serves as reusable templates or can be updated in real-time via auomations, which is something I am also working on. It is truly an insane way to monitor and track information. Better than a newsletter. Better than newspapers. There is something about this that gets me really excited about the future of AI agents for knowledge generation and discovery. Lots of hidden gems everywhere just waiting to be discovered and acted on if the information is presented correctly. This is not perfect. The format, style/prose can be improved, but this is easy to customize via skill. You can personalize it to your liking. I feel like these dynamic artifacts are going to emerge as a strong new medium to stay on the cutting edge of things, both for agents and humans. My target is research, of course. This was just a basic example. Besides animation, I am also targeting other components like voice, videos, images, slides, etc. This space is full of opportunities to explore. Skill for this coming soon.

LLM Artifacts Connected to Andrej Karpathy's LLM Knowledge base idea, I've been building out a fun way to generate dynamic artifacts from these knowledge bases with the goal of discovering and revealing meaningful and deeper insights. LLM KBs are hard to consume for humans, as I think they are more built for agents. So the question is, what form would be useful for humans to take actions and make important decisions? That's what I am trying to figure out with these artifacts. The artifact example shows a pulse on HN discussions around AI-related stories. The insights can go deeper, of course, but this is already super fun and thought-provoking, like some of my favorite podcasts. The format and depth matter a lot. The aggregation skills of agents are outstanding if you tune the prompts and skill carefully. I built this artifact generator in a few minutes through an agent skill, but I feel like there are so many ways that LLM-generated information can be used and consumed. Like generating deeper insights and analysis, and things that are just not feasible for humans today. The generated artifact (including its data and design) serves as reusable templates or can be updated in real-time via auomations, which is something I am also working on. It is truly an insane way to monitor and track information. Better than a newsletter. Better than newspapers. There is something about this that gets me really excited about the future of AI agents for knowledge generation and discovery. Lots of hidden gems everywhere just waiting to be discovered and acted on if the information is presented correctly. This is not perfect. The format, style/prose can be improved, but this is easy to customize via skill. You can personalize it to your liking. I feel like these dynamic artifacts are going to emerge as a strong new medium to stay on the cutting edge of things, both for agents and humans. My target is research, of course. This was just a basic example. Besides animation, I am also targeting other components like voice, videos, images, slides, etc. This space is full of opportunities to explore. Skill for this coming soon.

elvis

31,190 просмотров • 2 месяцев назад

Some personal/professional news: I'm launching "When the Facts Change - economics and politics in a fast-changing world". Click on the link in my X biography to read my first post. I'll obviously continue to write my weekly "Economics Agenda" column for the Telegraph, and co-present the "Planet Normal" podcast each week with the brilliant Allison Pearson I've been encouraged by lots of people, not least on X, to do more writing and broadcasting. So here goes .... "When the Facts Change" will initially be limited to written posts. But, as I find my feet, I'll add audio and video content too. So please check out my launch post via the link in my X biog and feel free to share this news. 🧵 1/2

Some personal/professional news: I'm launching "When the Facts Change - economics and politics in a fast-changing world". Click on the link in my X biography to read my first post. I'll obviously continue to write my weekly "Economics Agenda" column for the Telegraph, and co-present the "Planet Normal" podcast each week with the brilliant Allison Pearson I've been encouraged by lots of people, not least on X, to do more writing and broadcasting. So here goes .... "When the Facts Change" will initially be limited to written posts. But, as I find my feet, I'll add audio and video content too. So please check out my launch post via the link in my X biog and feel free to share this news. 🧵 1/2

Liam Halligan

402,233 просмотров • 1 год назад

Eleven years ago I founded my first startup. One of the first things I did was to get a desk at the startup coworking space at Google Campus in Old Street. Through that community, I met so many of the people who would shape my career and learned so much about what the startup ethos is truly about. As we have scaled Attio, we've partnered with Google Cloud to build on top of the mind blowing technologies they've created like Spanner, Colossus and Gemini to power a new generation of CRM for our customers. For all these reasons, Google is deeply woven into my startup journey and that makes today's announcement so special to me. I'm beyond excited to announce our $52 million Series B, led by Michael McBride and the GV, along with the continued support of our incredible partners from Redpoint , Point Nine , Balderton Capital and 01A. We're entering a software renaissance driven by AI and we're unbelievably excited about continuing the hard work of delivering the next generation of CRM. We're building a truly AI-native platform that can run code, integrate anywhere and bring unified context for the next generation of autonomous workflows. None of this would have been possible without the support of the more than 5,000 customers who have already chosen Attio as the backbone of their GTM stack. Your belief in our vision is what has made this possible and I can't wait to share more about some of the incredible functionality that's still to come. I'm incredibly grateful for the continued support of alex bard , Patrick Chase, Ricardo Sequerra Amram, Daniel Waterhouse and dick costolo whose belief in Attio from the earliest days has been so essential to our progress so far and to my co-founder Nicolas Sharp, for the many sleepless nights and challenging decisions that got us this far. We've got so much still to build, and we're always hiring, so if you are interested in being part of reinventing the core of the GTM stack, please take a look at For now though, it's back to building!

Eleven years ago I founded my first startup. One of the first things I did was to get a desk at the startup coworking space at Google Campus in Old Street. Through that community, I met so many of the people who would shape my career and learned so much about what the startup ethos is truly about. As we have scaled Attio, we've partnered with Google Cloud to build on top of the mind blowing technologies they've created like Spanner, Colossus and Gemini to power a new generation of CRM for our customers. For all these reasons, Google is deeply woven into my startup journey and that makes today's announcement so special to me. I'm beyond excited to announce our $52 million Series B, led by Michael McBride and the GV, along with the continued support of our incredible partners from Redpoint , Point Nine , Balderton Capital and 01A. We're entering a software renaissance driven by AI and we're unbelievably excited about continuing the hard work of delivering the next generation of CRM. We're building a truly AI-native platform that can run code, integrate anywhere and bring unified context for the next generation of autonomous workflows. None of this would have been possible without the support of the more than 5,000 customers who have already chosen Attio as the backbone of their GTM stack. Your belief in our vision is what has made this possible and I can't wait to share more about some of the incredible functionality that's still to come. I'm incredibly grateful for the continued support of alex bard , Patrick Chase, Ricardo Sequerra Amram, Daniel Waterhouse and dick costolo whose belief in Attio from the earliest days has been so essential to our progress so far and to my co-founder Nicolas Sharp, for the many sleepless nights and challenging decisions that got us this far. We've got so much still to build, and we're always hiring, so if you are interested in being part of reinventing the core of the GTM stack, please take a look at For now though, it's back to building!

Alexander Christie

16,506 просмотров • 10 месяцев назад

Went to Nottingham today and my god the place looks so sad and run down. The 1st thing was the horrendous smell of weed as we parked in the Trinity Square car park and then on the 2nd floor of the stairs to get out someone had literally shit up the actual wall. It was absolutely disgusting and I genuinely was nearly sick. We couldn’t stop heaving the smell was like a sewage works and baby poo. What sort of human actually pulls their pants down and shits up a wall. 💩 Then the city centre and Christmas market was so so depressingly bad. Such a shame because Nottingham has so much amazing history and architecture but it’s like they’ve given up with the city and let it just go to ruin. And then to top it off I was served this at Five Guys UK and charged £24.10 for the pleasure haha funny but annoying because it’s bloody expensive 😔

Went to Nottingham today and my god the place looks so sad and run down. The 1st thing was the horrendous smell of weed as we parked in the Trinity Square car park and then on the 2nd floor of the stairs to get out someone had literally shit up the actual wall. It was absolutely disgusting and I genuinely was nearly sick. We couldn’t stop heaving the smell was like a sewage works and baby poo. What sort of human actually pulls their pants down and shits up a wall. 💩 Then the city centre and Christmas market was so so depressingly bad. Such a shame because Nottingham has so much amazing history and architecture but it’s like they’ve given up with the city and let it just go to ruin. And then to top it off I was served this at Five Guys UK and charged £24.10 for the pleasure haha funny but annoying because it’s bloody expensive 😔

Daniel Kerley

842,861 просмотров • 7 месяцев назад

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Alok

292,096 просмотров • 1 месяц назад

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,838,219 просмотров • 2 месяцев назад

Bit of end of cycle / start of year testing AKA gym yap Front Squat 140kg: Very happy with this given I've only been back squatting for 9 weeks, and that's the longest cycle I've been able to string together in like 5+ years. I can technically call this a PB since it was beltless, sleeveless and less fat bodyweight so over the moon here. Backsquats of any nature keep tweaking my lower back so gonna stick with fronts for the rest of the year. Bench 140kg: Ive done this a lot in the last 10 years, but I was always 105kg bodyweight or more. So to do it at 97kg is a big win. Pressing power has been feeling horrible for the last year or so and I couldn't figure out a good programme, so happy that it's finally clicked again. Deadlift 200kg: Tweaked my lower back 3 times last year and put a lot of effort into fixing it up and strengthening it. 200 was easy so I did attempt 220kg but as soon as it left the ground either I would have failed or had a 10 second grind on my hands so decided to just not risk it. Goal for last year was to get healthy enough to be able to train the deadlift again and I achieved that so despite being far from my best of 262.5kg I've very pleased Ive been able to get to a position where I can train it again without headaches or lower back tweaks. OHP 80kg: Haven't trained OHP in a year or more, and stopped doing behind the neck press 3 months ago when I started benching more, so just wanted to do this for fun. Bit off my best of 95kg but its funny how much strength you retain without even doing a vertical press. Cutting down to 90kg again over the next 2-3 months then will see where I can take these over the rest of the year. No real numbers in mind, just happy to be able to train hard.

Bit of end of cycle / start of year testing AKA gym yap Front Squat 140kg: Very happy with this given I've only been back squatting for 9 weeks, and that's the longest cycle I've been able to string together in like 5+ years. I can technically call this a PB since it was beltless, sleeveless and less fat bodyweight so over the moon here. Backsquats of any nature keep tweaking my lower back so gonna stick with fronts for the rest of the year. Bench 140kg: Ive done this a lot in the last 10 years, but I was always 105kg bodyweight or more. So to do it at 97kg is a big win. Pressing power has been feeling horrible for the last year or so and I couldn't figure out a good programme, so happy that it's finally clicked again. Deadlift 200kg: Tweaked my lower back 3 times last year and put a lot of effort into fixing it up and strengthening it. 200 was easy so I did attempt 220kg but as soon as it left the ground either I would have failed or had a 10 second grind on my hands so decided to just not risk it. Goal for last year was to get healthy enough to be able to train the deadlift again and I achieved that so despite being far from my best of 262.5kg I've very pleased Ive been able to get to a position where I can train it again without headaches or lower back tweaks. OHP 80kg: Haven't trained OHP in a year or more, and stopped doing behind the neck press 3 months ago when I started benching more, so just wanted to do this for fun. Bit off my best of 95kg but its funny how much strength you retain without even doing a vertical press. Cutting down to 90kg again over the next 2-3 months then will see where I can take these over the rest of the year. No real numbers in mind, just happy to be able to train hard.

Reisshub

19,040 просмотров • 6 месяцев назад