Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

GPT 5.6 Sol turned my Daylight DC-1 into Tom Riddle's diary from Harry Potter. My prompts fade, OCR with Apple's Vision framework + Gemma 4 31B + Ollama respond. Everything locally on my Macbook. Watch till the end and volume up! Magical and inspired by Maxime Rivest 🧙‍♂️🦙🐧 work 🙏

ivanfioravanti's profile picture

Ivan Fioravanti ᯅ

40,963 subscribers

21,088 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Fable turned my remarkable into Tom Riddle's diary from Harry Potter. My prompts fade, a LLM respond. Magical!
0:24

Fable turned my remarkable into Tom Riddle's diary from Harry Potter. My prompts fade, a LLM respond. Magical!

Maxime Rivest 🧙‍♂️🦙🐧

2,111,835 Aufrufe • vor 2 Monaten

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer
0:22

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

23,088 Aufrufe • vor 4 Monaten

I made $40K on YouTube last month and never turned on a camera, it was 7 ChatGPT prompts Here's what I typed > Prompt 1: generate 10 viral video ideas in my niche from the latest trends > Prompt 2: write a full script with a hook, story arc, and CTA that maximizes watch time > Prompt 3: build me a faceless channel, script it, voice it, edit it, make it run without me > Prompt 4: generate a clickworthy, SEO-optimized title, description, and tags > Prompt 5: write a 15-second hook people physically can't stop watching > Prompt 6: create a 30-day growth and monetization plan from zero > Prompt 7: turn the video into a blog post, a Reel, a TikTok, and a thread 7 prompts, 1 laptop, 0 cameras the channel never needed my face, it needed my prompts
1:36

I made $40K on YouTube last month and never turned on a camera, it was 7 ChatGPT prompts Here's what I typed > Prompt 1: generate 10 viral video ideas in my niche from the latest trends > Prompt 2: write a full script with a hook, story arc, and CTA that maximizes watch time > Prompt 3: build me a faceless channel, script it, voice it, edit it, make it run without me > Prompt 4: generate a clickworthy, SEO-optimized title, description, and tags > Prompt 5: write a 15-second hook people physically can't stop watching > Prompt 6: create a 30-day growth and monetization plan from zero > Prompt 7: turn the video into a blog post, a Reel, a TikTok, and a thread 7 prompts, 1 laptop, 0 cameras the channel never needed my face, it needed my prompts

Woody

70,247 Aufrufe • vor 3 Monaten

The exact blueprint that ranked my sites #1 on Google, is now available! 🚀 👉 Watch over my shoulder as I reveal my entire SEO system. Everything I've learned ranking 20+ sites to #1: finding untapped niches, validating keywords, creating content that ranks, and building powerful backlinks. You'll learn my exact process: 🔍 How to find profitable niches others miss 💰 The science behind keywords that actually convert ✍️ My proven content creation framework 🔗 Step-by-step backlink building strategies that work in 2024 📈 And the exact templates my team uses to scale No theory. No fluff. Just the battle-tested SEO blueprint that's worked 20+ times. By the end of this course, you'll have everything you need to stop hoping for traffic and start generating it systematically. Get it with 32% discount for the next 16 hours.
1:26

The exact blueprint that ranked my sites #1 on Google, is now available! 🚀 👉 Watch over my shoulder as I reveal my entire SEO system. Everything I've learned ranking 20+ sites to #1: finding untapped niches, validating keywords, creating content that ranks, and building powerful backlinks. You'll learn my exact process: 🔍 How to find profitable niches others miss 💰 The science behind keywords that actually convert ✍️ My proven content creation framework 🔗 Step-by-step backlink building strategies that work in 2024 📈 And the exact templates my team uses to scale No theory. No fluff. Just the battle-tested SEO blueprint that's worked 20+ times. By the end of this course, you'll have everything you need to stop hoping for traffic and start generating it systematically. Get it with 32% discount for the next 16 hours.

Danny Postma

59,977 Aufrufe • vor 1 Jahr

Mini tutorial for Google DeepMind 's VEO 2 Prompt Buddy. (Works well with other AI video tools as well). I’ve created my VEO 2 Prompt Buddy Custom GPT to help me go beyond my own brain and prompts. It can describe your images, enhance your prompts, be creative and more. It will provide you 4 different sections to choose from. I’ve also added a camera combinations section inspired by the Hailuo I2V-01-Director model for testing purposes. It’s currently in BETA mode in my GPT because, sometimes, multiple camera movements don’t work optimally in VEO 2 right now. Output sections: 1. PRO – Provides detailed, professional, and cinematically accurate results. 2. OUT OF THE BOX – Creates something more creative and unexpected from your idea. 3. ULTRA DETAILED – Expands your idea with extreme precision and even more detail. 4. CAMERA COMBINATIONS – Generates a version with cinematic camera transitions. This hot sauce is FREE for a limited time! Hope you have fun with this one. Link down below: --- All results are text-to-video
1:21

Mini tutorial for Google DeepMind 's VEO 2 Prompt Buddy. (Works well with other AI video tools as well). I’ve created my VEO 2 Prompt Buddy Custom GPT to help me go beyond my own brain and prompts. It can describe your images, enhance your prompts, be creative and more. It will provide you 4 different sections to choose from. I’ve also added a camera combinations section inspired by the Hailuo I2V-01-Director model for testing purposes. It’s currently in BETA mode in my GPT because, sometimes, multiple camera movements don’t work optimally in VEO 2 right now. Output sections: 1. PRO – Provides detailed, professional, and cinematically accurate results. 2. OUT OF THE BOX – Creates something more creative and unexpected from your idea. 3. ULTRA DETAILED – Expands your idea with extreme precision and even more detail. 4. CAMERA COMBINATIONS – Generates a version with cinematic camera transitions. This hot sauce is FREE for a limited time! Hope you have fun with this one. Link down below: --- All results are text-to-video

Allar Haltsonen

58,652 Aufrufe • vor 1 Jahr

I made a page-flipping sketchbook inspired by Matthew Yu's concept. Books and pages are some of the most interesting UIs to explore. AI gives you the basics unless you hand it a solid open-source to work from. So I asked for drag to paginate, which took many fixes for shadows, clipping, alpha masks and logic. Then a magnifying glass that has to move out of the way when you flip a page. For images I used gpt image 2 and it's very good at these Singapore sketches. I also love adding minimal background textures like japanese paper, water painting and ornaments. That's where the personality comes from. FYI I'm using Claude Code on the desktop app. My Codex is utterly out of limits after GPT 5.6 Sol ate all my $200/m plan 2-3x faster than 5.5. From testing, Opus 5 at Extra is more efficient. Let me know if I should open-source this or give the prompts.
0:21

I made a page-flipping sketchbook inspired by Matthew Yu's concept. Books and pages are some of the most interesting UIs to explore. AI gives you the basics unless you hand it a solid open-source to work from. So I asked for drag to paginate, which took many fixes for shadows, clipping, alpha masks and logic. Then a magnifying glass that has to move out of the way when you flip a page. For images I used gpt image 2 and it's very good at these Singapore sketches. I also love adding minimal background textures like japanese paper, water painting and ornaments. That's where the personality comes from. FYI I'm using Claude Code on the desktop app. My Codex is utterly out of limits after GPT 5.6 Sol ate all my $200/m plan 2-3x faster than 5.5. From testing, Opus 5 at Extra is more efficient. Let me know if I should open-source this or give the prompts.

Meng To

133,571 Aufrufe • vor 1 Monat

GPT 5.6 SOL IS HERE! How to run your personal + business life with GPT 5.6 Sol + Codex (full 49 min masterclass) We tested it for 30 days and the video it's the CLEAREST look at the FUTURE of work: Here's what's possible once you set it up: 1. Your inbox becomes cards every morning, each with a summary and a reply drafted in your own voice. Y 2. Your Slack, meeting notes, and company updates can turn into one daily feed with a clear next action. It learns what you care about over time and rewrites its own prompts to get sharper. 3. You can give your agent its own email address, so your other tools and even your team's Slack bot email it directly and it just handles things. 4. You can have it watch you do a task once and turn it into a skill it repeats forever. 5. You can set a long goal and walk away. You can have it run for 20 hours straight, and fine-tune your own models, something that was out of reach for non-engineers 12 months ago. How to start: Open Codex, give it access to your computer, and ask it to suggest things it could do for you based on how you already work. Full episode on The Startup Ideas Podcast (SIP) 🧃 (thanks Dan Shipper for sharing your entire workflow and review of GPT 5.6) Start with one boring task, get it working, and build from there. You'll learn exactly how to make something similar. GPT 5.6 Sol is impressive. Sol (according to openAI benchmarks) is the best coding model out right now. It set a new state of the art on Terminal-Bench 2.1 at 88.8%, and its "ultra mode" hits 91.9%, beating Claude Opus 4.8, Fable 5, and even Mythos 5 this masterclass is 100% free, like always. For more The Startup Ideas Podcast (SIP) 🧃 Watch
49:16

GPT 5.6 SOL IS HERE! How to run your personal + business life with GPT 5.6 Sol + Codex (full 49 min masterclass) We tested it for 30 days and the video it's the CLEAREST look at the FUTURE of work: Here's what's possible once you set it up: 1. Your inbox becomes cards every morning, each with a summary and a reply drafted in your own voice. Y 2. Your Slack, meeting notes, and company updates can turn into one daily feed with a clear next action. It learns what you care about over time and rewrites its own prompts to get sharper. 3. You can give your agent its own email address, so your other tools and even your team's Slack bot email it directly and it just handles things. 4. You can have it watch you do a task once and turn it into a skill it repeats forever. 5. You can set a long goal and walk away. You can have it run for 20 hours straight, and fine-tune your own models, something that was out of reach for non-engineers 12 months ago. How to start: Open Codex, give it access to your computer, and ask it to suggest things it could do for you based on how you already work. Full episode on The Startup Ideas Podcast (SIP) 🧃 (thanks Dan Shipper for sharing your entire workflow and review of GPT 5.6) Start with one boring task, get it working, and build from there. You'll learn exactly how to make something similar. GPT 5.6 Sol is impressive. Sol (according to openAI benchmarks) is the best coding model out right now. It set a new state of the art on Terminal-Bench 2.1 at 88.8%, and its "ultra mode" hits 91.9%, beating Claude Opus 4.8, Fable 5, and even Mythos 5 this masterclass is 100% free, like always. For more The Startup Ideas Podcast (SIP) 🧃 Watch

GREG ISENBERG

156,617 Aufrufe • vor 2 Monaten

Thrilled to unveil what we've been cooking the last few weeks: the Gravity AI assistant. Local, free, owned by you. Now available for free on macOS -> GravityAI 🌠 Gravity ensures absolute privacy by keeping all conversations on your device. No compromises, your data is exclusively yours. Plus, with LLaVa and OCR, Gravity gains vision – just snap a screenshot, and it's ready to chat about anything on your screen. Everything has been built to optimize the power of Apple silicon: I used Mistral AI as a base LLM and a in-app instance of ollama. It is insanely fast. Gravity will continue to evolve from here. For example, we're working on speech recognition and speaker diarization to passively learn from your meetings and messages in the background, no integrations needed. Incredibly proud of our journey so far and a big shoutout to my co-founder Tye Daniel for grinding to make this happen (we were up till 4am making sure this ships today) We're hiring folks with talent and passion, and we welcome you to join us on this exciting journey!
0:40

Thrilled to unveil what we've been cooking the last few weeks: the Gravity AI assistant. Local, free, owned by you. Now available for free on macOS -> GravityAI 🌠 Gravity ensures absolute privacy by keeping all conversations on your device. No compromises, your data is exclusively yours. Plus, with LLaVa and OCR, Gravity gains vision – just snap a screenshot, and it's ready to chat about anything on your screen. Everything has been built to optimize the power of Apple silicon: I used Mistral AI as a base LLM and a in-app instance of ollama. It is insanely fast. Gravity will continue to evolve from here. For example, we're working on speech recognition and speaker diarization to passively learn from your meetings and messages in the background, no integrations needed. Incredibly proud of our journey so far and a big shoutout to my co-founder Tye Daniel for grinding to make this happen (we were up till 4am making sure this ships today) We're hiring folks with talent and passion, and we welcome you to join us on this exciting journey!

Sulaiman Khan Ghori

71,430 Aufrufe • vor 2 Jahren

My trust and respect for Elon will only be restored when he comes through on at least one of these promises—just one: 1. Robotaxi expansion to hundreds of thousands by end of 2026 (on record from his CNBC interview with David) 2. Being able to enroll my Tesla into the company ride-share fleet 3. FSD unsupervised feature complete (end of 2020 on record) 4. CyberCab fleet up and running Just one of these four coming true by year's end and I will be a fan again, completely respectful as I have been for 14 years— is that too much to ask for ?!
2:15

My trust and respect for Elon will only be restored when he comes through on at least one of these promises—just one: 1. Robotaxi expansion to hundreds of thousands by end of 2026 (on record from his CNBC interview with David) 2. Being able to enroll my Tesla into the company ride-share fleet 3. FSD unsupervised feature complete (end of 2020 on record) 4. CyberCab fleet up and running Just one of these four coming true by year's end and I will be a fan again, completely respectful as I have been for 14 years— is that too much to ask for ?!

Jimmy

73,878 Aufrufe • vor 4 Monaten

GPT 5.6 Sol just saved me €650 a year and demonstrated just how good this model is as an agent in @ChatGPTapp Codex. This is not a clickbait, let me explain. In France, insurance companies tend to hide all their prices behind quote forms that take at least 5min to complete for a single configuration on a single provider's website. And that take an other 5min to understand. If you want to compare 10 companies across 5 configurations each, it takes at least 4h +. (It's such a painful process that entire businesses exist just to compare insurance offers.) Since I have two cars, it would normally take me 8h so an entire day to have a real large view of my best option. Companies know this and use the friction to maintain overpriced offers. So I asked @ChatGPTapp with GPT 5.6 Sol, using Chrome tabs, to go through all those annoying forms. I provided him all my contrats with my current insurance companies so he have context. For some insurers, you even have to speak with a representative just to get a quote (which is absurd in 2026), so it emailed the companies, exchanged the required information, and obtained the prices. It then ran a complete benchmark, read all the terms and conditions, and recommended three options from three different companies. I picked one, and it completed the subscription with my new insurance company. And that's how I ended up with better insurance coverage for less money. For 4% of my weekly quota in 20x plan. (i think it's fair) All of that happened while I was walking my dog for 50 minutes, he was working on my computer all by it's on. Yes, computer use existed before OpenAI GPT 5.6 Sol, but this is a completely different level in the way it handles these kinds of tasks. I think this story shows the new era of AI we're entering, good model is not only for one single task as coding or answering question, AI now can do things for you, like in your daily live. I love being able to hand my computer over to GPT 5.6 Sol. PS: The only annoying part was that some companies still require "Verify you're human" checks. In the age of AI agents, websites really need to be ready for robot access.
0:20

GPT 5.6 Sol just saved me €650 a year and demonstrated just how good this model is as an agent in @ChatGPTapp Codex. This is not a clickbait, let me explain. In France, insurance companies tend to hide all their prices behind quote forms that take at least 5min to complete for a single configuration on a single provider's website. And that take an other 5min to understand. If you want to compare 10 companies across 5 configurations each, it takes at least 4h +. (It's such a painful process that entire businesses exist just to compare insurance offers.) Since I have two cars, it would normally take me 8h so an entire day to have a real large view of my best option. Companies know this and use the friction to maintain overpriced offers. So I asked @ChatGPTapp with GPT 5.6 Sol, using Chrome tabs, to go through all those annoying forms. I provided him all my contrats with my current insurance companies so he have context. For some insurers, you even have to speak with a representative just to get a quote (which is absurd in 2026), so it emailed the companies, exchanged the required information, and obtained the prices. It then ran a complete benchmark, read all the terms and conditions, and recommended three options from three different companies. I picked one, and it completed the subscription with my new insurance company. And that's how I ended up with better insurance coverage for less money. For 4% of my weekly quota in 20x plan. (i think it's fair) All of that happened while I was walking my dog for 50 minutes, he was working on my computer all by it's on. Yes, computer use existed before OpenAI GPT 5.6 Sol, but this is a completely different level in the way it handles these kinds of tasks. I think this story shows the new era of AI we're entering, good model is not only for one single task as coding or answering question, AI now can do things for you, like in your daily live. I love being able to hand my computer over to GPT 5.6 Sol. PS: The only annoying part was that some companies still require "Verify you're human" checks. In the age of AI agents, websites really need to be ready for robot access.

Defend Intelligence (Anis Ayari)

78,748 Aufrufe • vor 2 Monaten

Day 8 of building the game I've wanted to play for 15 years with AI. I absolutely love GPT-6. I'm switching from Claude completely. Today it finally happened: a full animation set for a non-humanoid character. My wolf is rigged and animated. All of last week I was trying approaches in parallel: Mesh2Motion, Cascadeur, Animate Anything, a pile of smaller tools. Everything kept almost working and then falling apart. Then GPT-6, and it's done. To be fair, I came in with a worked-out workflow: 1. Generate the model optimized from the start. Tripo P2.0 with multiview input, around 10k polygons, and the model came out great. 2. My agent turned the same side reference I used for the 3D gen into motion references with Seedance 2.5. Takes vary, so I picked the best ones. I only need 4 animations. 3. I decided the timing of each animation upfront, so they drop into the game as is. 4. Handed all of it to GPT-6 with Blender MCP. A couple of hours and a couple of feedback rounds later I have working animations I'm taking into the game. I fought this with Fable and the automation always came out crooked. This time it just worked. If you've been stuck on animations for a creature like this, try that route. And if anyone wants my workflow MD for GPT-6, I'll share it.
0:38

Day 8 of building the game I've wanted to play for 15 years with AI. I absolutely love GPT-6. I'm switching from Claude completely. Today it finally happened: a full animation set for a non-humanoid character. My wolf is rigged and animated. All of last week I was trying approaches in parallel: Mesh2Motion, Cascadeur, Animate Anything, a pile of smaller tools. Everything kept almost working and then falling apart. Then GPT-6, and it's done. To be fair, I came in with a worked-out workflow: 1. Generate the model optimized from the start. Tripo P2.0 with multiview input, around 10k polygons, and the model came out great. 2. My agent turned the same side reference I used for the 3D gen into motion references with Seedance 2.5. Takes vary, so I picked the best ones. I only need 4 animations. 3. I decided the timing of each animation upfront, so they drop into the game as is. 4. Handed all of it to GPT-6 with Blender MCP. A couple of hours and a couple of feedback rounds later I have working animations I'm taking into the game. I fought this with Fable and the automation always came out crooked. This time it just worked. If you've been stuck on animations for a creature like this, try that route. And if anyone wants my workflow MD for GPT-6, I'll share it.

Stefan 3D AI

90,850 Aufrufe • vor 19 Tagen

BREAKING: GPT-5.6 Sol is out—AND Codex has been merged into ChatGPT Desktop as ChatGPT Codex. This combo model and desktop app harness are the gold-standard for knowledge work in AI. 5.6 is powerful, fast, half the price of Fable, and my default for almost everything. We’ve been testing it internally Every 📧 for about a month across coding, writing, design, and knowledge work. Here’s our day-zero vibe check: - An A-tier coder—but it’s not Fable. Sol scored 56/100 on our Senior Engineer benchmark compared to a 91 for Fable. I think the 56/100 undersells it, it's an excellent implementor, and very smart. But Fable just writes conceptually cleaner code and works better at the top end of task complexity. PRO-TIP: Use GPT-5.6 as Fable's subagent for the most goated combo in AI coding. - The best writer of the frontier models. It’s clearer and more concise than Fable or Opus 4.8, without the overexplaining or weird private language. It can one-shot marketing emails, help you workshop taglines, and explain complex concepts clearly. It's also super fast, which makes it easy to collaborate with. - Design is better, but not top-tier. It has noticeably more taste than 5.5, but Fable and Opus 4.8 are still playing at a different level. See examples in the video and vibe check below. - The real leap is knowledge work. Sol is the first model I’ve trusted to run whole loops of knowledge work—not just help with individual tasks. I use it to process email, surface decisions from meetings and Slack, find job candidates, scan Facebook Marketplace for furniture, and log my meals. It has shifted my job from doing the work to tending the system that does it. - The merged app is fine. I was extremely worried about this because I love the Codex app. OpenAI was caught in an interesting position: How to make an agent orchestration app for regular ChatGPT consumers, coders, and businesses all in one app. They now split the interface between ChatGPT Work and ChatGPT Codex. They're basically the same except Work hides code. And "Chat" has been demoted to 2nd tier status for quick questions in either one. It's not a big leap, but it's not a huge setback either. And it remains my favorite of the desktop agent orchestration apps. Verdict: If I really had to put my finger on it, I'd say Fable has way more big model smell. But that means it's a skill in itself to get value out of it—99% of people are still not there yet. GPT-5.6 is almost as powerful, but is easy to use, fast, and relatively cheap. It should give you an early sense of where model work is going. Full Every 📧 Vibe Check:
13:34

BREAKING: GPT-5.6 Sol is out—AND Codex has been merged into ChatGPT Desktop as ChatGPT Codex. This combo model and desktop app harness are the gold-standard for knowledge work in AI. 5.6 is powerful, fast, half the price of Fable, and my default for almost everything. We’ve been testing it internally Every 📧 for about a month across coding, writing, design, and knowledge work. Here’s our day-zero vibe check: - An A-tier coder—but it’s not Fable. Sol scored 56/100 on our Senior Engineer benchmark compared to a 91 for Fable. I think the 56/100 undersells it, it's an excellent implementor, and very smart. But Fable just writes conceptually cleaner code and works better at the top end of task complexity. PRO-TIP: Use GPT-5.6 as Fable's subagent for the most goated combo in AI coding. - The best writer of the frontier models. It’s clearer and more concise than Fable or Opus 4.8, without the overexplaining or weird private language. It can one-shot marketing emails, help you workshop taglines, and explain complex concepts clearly. It's also super fast, which makes it easy to collaborate with. - Design is better, but not top-tier. It has noticeably more taste than 5.5, but Fable and Opus 4.8 are still playing at a different level. See examples in the video and vibe check below. - The real leap is knowledge work. Sol is the first model I’ve trusted to run whole loops of knowledge work—not just help with individual tasks. I use it to process email, surface decisions from meetings and Slack, find job candidates, scan Facebook Marketplace for furniture, and log my meals. It has shifted my job from doing the work to tending the system that does it. - The merged app is fine. I was extremely worried about this because I love the Codex app. OpenAI was caught in an interesting position: How to make an agent orchestration app for regular ChatGPT consumers, coders, and businesses all in one app. They now split the interface between ChatGPT Work and ChatGPT Codex. They're basically the same except Work hides code. And "Chat" has been demoted to 2nd tier status for quick questions in either one. It's not a big leap, but it's not a huge setback either. And it remains my favorite of the desktop agent orchestration apps. Verdict: If I really had to put my finger on it, I'd say Fable has way more big model smell. But that means it's a skill in itself to get value out of it—99% of people are still not there yet. GPT-5.6 is almost as powerful, but is easy to use, fast, and relatively cheap. It should give you an early sense of where model work is going. Full Every 📧 Vibe Check:

Dan Shipper

146,190 Aufrufe • vor 2 Monaten

THE WAY YOU LOOK AT AI GAME DEVELOPMENT WILL CHANGE AFTER YOU WATCH THIS VIDEO "The AI Slop is Coming for Games." That's right. Introducing... Slopet League Everything you see here was built by Opus 5 and GPT 5.6 Sol in _____🤫. This is not an html page, it is a functional game with a fair bit of polish working inside a real game engine. This has to be some kind of trick. NO. It downloaded a map. NO. Imported assets. Only the Octane meshes, I have other cars that were generated that look just as good. The balls, the map, the effects, the physics, animations, etc were all generated from scratch by Opus 5. GPT 5.6 Sol handled the car meshes. This is only the beginning. I made this in about a day and a half, and most of that time I was waiting on Opus to finish generation. Maybe 25 prompts. I will share my process, and many photos, proofs, evidence of this as we go. This is just a taste, I'm extremely busy this week and working on real life stuff so I'm not going to put any more time into this until next week, but believe me I'm gonna polish this into something beautiful. If you're interested, please share, bookmark, follow, and comment below. I want the world to see just how powerful the language models are TODAY. It's only going to get better from here. Buckle up folks. Oh, and What a save!
1:39

THE WAY YOU LOOK AT AI GAME DEVELOPMENT WILL CHANGE AFTER YOU WATCH THIS VIDEO "The AI Slop is Coming for Games." That's right. Introducing... Slopet League Everything you see here was built by Opus 5 and GPT 5.6 Sol in _____🤫. This is not an html page, it is a functional game with a fair bit of polish working inside a real game engine. This has to be some kind of trick. NO. It downloaded a map. NO. Imported assets. Only the Octane meshes, I have other cars that were generated that look just as good. The balls, the map, the effects, the physics, animations, etc were all generated from scratch by Opus 5. GPT 5.6 Sol handled the car meshes. This is only the beginning. I made this in about a day and a half, and most of that time I was waiting on Opus to finish generation. Maybe 25 prompts. I will share my process, and many photos, proofs, evidence of this as we go. This is just a taste, I'm extremely busy this week and working on real life stuff so I'm not going to put any more time into this until next week, but believe me I'm gonna polish this into something beautiful. If you're interested, please share, bookmark, follow, and comment below. I want the world to see just how powerful the language models are TODAY. It's only going to get better from here. Buckle up folks. Oh, and What a save!

am.will

131,731 Aufrufe • vor 2 Monaten

This is Farzapedia. I had an LLM take 2,500 entries from my diary, Apple Notes, and some iMessage convos to create a personal Wikipedia for me. It made 400 detailed articles for my friends, my startups, research areas, and even my favorite animes and their impact on me complete with backlinks. But, this Wiki was not built for me! I built it for my agent! The structure of the wiki files and how it's all backlinked is very easily crawlable by any agent + makes it a truly useful knowledge base. I can spin up Claude Code on the wiki and starting at index.md (a catalog of all my articles) the agent does a really good job at drilling into the specific pages on my wiki it needs context on when I have a query. For example, when trying to cook up a new landing page I may ask: "I'm trying to design this landing page for a new idea I have. Please look into the images and films that inspired me recently and give me ideas for new copy and aesthetics". In my diary I kept track of everything from: learnings, people, inspo, interesting links, images. So the agent reads my wiki and pulls up my "Philosophy" articles from notes on a Studio Ghibli documentary, "Competitor" articles with YC companies whose landing pages I screenshotted, and pics of 1970s Beatles merch I saved years ago. And it delivers a great answer. I built a similar system to this a year ago with RAG but it was ass. A knowledge base that lets an agent find what it needs via a file system it actually understands just works better. The most magical thing now is as I add new things to my wiki (articles, images of inspo, meeting notes) the system will likely update 2-3 different articles where it feels that context belongs, or, just creates a new article. It's like this super genius librarian for your brain that's always filing stuff for your perfectly and also let's you easily query the knowledge for tasks useful to you (ex. design, product, writing, etc) and it never gets tired. I might spend next week productizing this, if that's of interest to you DM me + tell me your usecase!
2:24

This is Farzapedia. I had an LLM take 2,500 entries from my diary, Apple Notes, and some iMessage convos to create a personal Wikipedia for me. It made 400 detailed articles for my friends, my startups, research areas, and even my favorite animes and their impact on me complete with backlinks. But, this Wiki was not built for me! I built it for my agent! The structure of the wiki files and how it's all backlinked is very easily crawlable by any agent + makes it a truly useful knowledge base. I can spin up Claude Code on the wiki and starting at index.md (a catalog of all my articles) the agent does a really good job at drilling into the specific pages on my wiki it needs context on when I have a query. For example, when trying to cook up a new landing page I may ask: "I'm trying to design this landing page for a new idea I have. Please look into the images and films that inspired me recently and give me ideas for new copy and aesthetics". In my diary I kept track of everything from: learnings, people, inspo, interesting links, images. So the agent reads my wiki and pulls up my "Philosophy" articles from notes on a Studio Ghibli documentary, "Competitor" articles with YC companies whose landing pages I screenshotted, and pics of 1970s Beatles merch I saved years ago. And it delivers a great answer. I built a similar system to this a year ago with RAG but it was ass. A knowledge base that lets an agent find what it needs via a file system it actually understands just works better. The most magical thing now is as I add new things to my wiki (articles, images of inspo, meeting notes) the system will likely update 2-3 different articles where it feels that context belongs, or, just creates a new article. It's like this super genius librarian for your brain that's always filing stuff for your perfectly and also let's you easily query the knowledge for tasks useful to you (ex. design, product, writing, etc) and it never gets tired. I might spend next week productizing this, if that's of interest to you DM me + tell me your usecase!

Farza 🇵🇰🇺🇸

2,137,617 Aufrufe • vor 5 Monaten

I’ve been using GPT-5.6 Sol internally for the past two months, I've spent probably 25+ billion tokens. Here’s my review and comparison to Fable 5: > Let's start with the analogy because everyone seems to be giving theirs - GPT-5.6 is likely the last version of the GPT-5 training run series. It's kind of like an athlete at their peak. Through years of experience in the game, they've become the most reliable player and has the highest game IQ. But, there's no more room to grow. Fable on the other hand, being essentially the first version of a new training run, is the first round draft pick rookie. Raw talent mixed with the energy only a young person would have results in some incredible plays we didn't think possible, but also mistakes due to lack of experience. But that rookie will only improve and likely will be better than the veteran ever was because it's a new game and a new era. > GPT-5.6 is genuinely better at long, sustained work. With /goal, I've had it running complex projects for days with almost no intervention. It built a Minecraft-style game, kept adding features and mobs after the core game worked, and only stopped because I stopped the run. I never felt as though I had to jump in and guide it back to the right path. > It keeps finding useful work when you give it a concrete finish line. I had it recreate Excel with a loop. It inspected the real desktop excel app with Computer Use, comparing that against its own build, and closing the gaps. I stopped it after six days after it had built an incredible amount of functionality. > It's faster than other models in two different ways. The raw generation speed is higher, something OpenAI has been putting effort into. But it also takes a shorter path to solutions. It wanders less, changes less code, and generally knows how to get things done directly. In daily use, it feels about 2-3x times faster than Fable. That's my impression, not a controlled benchmark. The difference is large enough that I notice it constantly. > It works well across a wide range of tasks. I use it for one-line edits, quick questions, browser chores, and multi-day builds without changing my prompting style. Speaking of browser control, its the best ever I've used. To the point where I actually use it often. If a task lives on a website, GPT-5.6 usually opens the browser and does it there instead of asking for an API key or forcing everything through the terminal. When I switched back to GPT-5.5, it went straight to the command line even when the browser was clearly the better tool. > And it can handle real browser work, not just toy demos. During a data import, I had it monitor Supabase and resize instances as the load changed. It stayed on the dashboard, adjusted capacity, and checked the result without an API or a custom script. > I also gave it a full Google Workspace migration. It moved Forward Future from to preserved the old aliases, and configured MX, SPF, and DKIM. Before a consequential save, it stopped, explained exactly what would change, and waited for confirmation. > The reasoning setting matters a lot. Light is good for questions and small edits. High and Extra High are the sweet spots for serious work. Ultra usually takes longer than the extra thinking is worth and burns tokens. > I love that 5.6 is split into 3 sizes. Not only can you control speed and cost that way, but you still also have the thinking effort setting for each of them. Very precise controls. I just wish Codex automatically routed my prompts for me. > Its personality is blunt and a little bland. Claude feels warmer and more natural to talk to. GPT-5.6 is more clinical, but I like that for work. It gives me enough explanation and rarely pads the answer. I usually have to ask Fable to explain things more simply and/or more concise. > Its front-end taste has improved, but the default is predictable. Left alone, it turns websites into PowerPoint decks with huge statements and hard section breaks. The good news is that it takes design direction well and can revise without destroying the parts that already work. > It still makes confident mistakes. I asked it to rebuild parts of a system, and it told me the job was finished. Later, I found out it wasn't. Bits of its internal process also leak into the answer occasionally. > Claude Fable is more naturally autonomous on large, open-ended projects. GPT-5.6 is easier to reach for. I don't need to invent a huge project to justify using it. It works just as well for a small edit or browser chore. > GPT-5.6 is also cheaper. Sol costs $5 per million input tokens and $30 per million output tokens. Fable costs $10 and $50. Cached input is cheaper too. Still, cost per finished task matters more than cost per token. > GPT-5.6 isn't the best at everything, and it still needs supervision. But it generates faster, wanders less, works at almost any scale, and wastes less of my time. It's the model I have the most confidence in to get the job done right the first time. I put together a full breakdown with all the tests, prompts, and examples on a site. You can read it here:
7:46

I’ve been using GPT-5.6 Sol internally for the past two months, I've spent probably 25+ billion tokens. Here’s my review and comparison to Fable 5: > Let's start with the analogy because everyone seems to be giving theirs - GPT-5.6 is likely the last version of the GPT-5 training run series. It's kind of like an athlete at their peak. Through years of experience in the game, they've become the most reliable player and has the highest game IQ. But, there's no more room to grow. Fable on the other hand, being essentially the first version of a new training run, is the first round draft pick rookie. Raw talent mixed with the energy only a young person would have results in some incredible plays we didn't think possible, but also mistakes due to lack of experience. But that rookie will only improve and likely will be better than the veteran ever was because it's a new game and a new era. > GPT-5.6 is genuinely better at long, sustained work. With /goal, I've had it running complex projects for days with almost no intervention. It built a Minecraft-style game, kept adding features and mobs after the core game worked, and only stopped because I stopped the run. I never felt as though I had to jump in and guide it back to the right path. > It keeps finding useful work when you give it a concrete finish line. I had it recreate Excel with a loop. It inspected the real desktop excel app with Computer Use, comparing that against its own build, and closing the gaps. I stopped it after six days after it had built an incredible amount of functionality. > It's faster than other models in two different ways. The raw generation speed is higher, something OpenAI has been putting effort into. But it also takes a shorter path to solutions. It wanders less, changes less code, and generally knows how to get things done directly. In daily use, it feels about 2-3x times faster than Fable. That's my impression, not a controlled benchmark. The difference is large enough that I notice it constantly. > It works well across a wide range of tasks. I use it for one-line edits, quick questions, browser chores, and multi-day builds without changing my prompting style. Speaking of browser control, its the best ever I've used. To the point where I actually use it often. If a task lives on a website, GPT-5.6 usually opens the browser and does it there instead of asking for an API key or forcing everything through the terminal. When I switched back to GPT-5.5, it went straight to the command line even when the browser was clearly the better tool. > And it can handle real browser work, not just toy demos. During a data import, I had it monitor Supabase and resize instances as the load changed. It stayed on the dashboard, adjusted capacity, and checked the result without an API or a custom script. > I also gave it a full Google Workspace migration. It moved Forward Future from to preserved the old aliases, and configured MX, SPF, and DKIM. Before a consequential save, it stopped, explained exactly what would change, and waited for confirmation. > The reasoning setting matters a lot. Light is good for questions and small edits. High and Extra High are the sweet spots for serious work. Ultra usually takes longer than the extra thinking is worth and burns tokens. > I love that 5.6 is split into 3 sizes. Not only can you control speed and cost that way, but you still also have the thinking effort setting for each of them. Very precise controls. I just wish Codex automatically routed my prompts for me. > Its personality is blunt and a little bland. Claude feels warmer and more natural to talk to. GPT-5.6 is more clinical, but I like that for work. It gives me enough explanation and rarely pads the answer. I usually have to ask Fable to explain things more simply and/or more concise. > Its front-end taste has improved, but the default is predictable. Left alone, it turns websites into PowerPoint decks with huge statements and hard section breaks. The good news is that it takes design direction well and can revise without destroying the parts that already work. > It still makes confident mistakes. I asked it to rebuild parts of a system, and it told me the job was finished. Later, I found out it wasn't. Bits of its internal process also leak into the answer occasionally. > Claude Fable is more naturally autonomous on large, open-ended projects. GPT-5.6 is easier to reach for. I don't need to invent a huge project to justify using it. It works just as well for a small edit or browser chore. > GPT-5.6 is also cheaper. Sol costs $5 per million input tokens and $30 per million output tokens. Fable costs $10 and $50. Cached input is cheaper too. Still, cost per finished task matters more than cost per token. > GPT-5.6 isn't the best at everything, and it still needs supervision. But it generates faster, wanders less, works at almost any scale, and wastes less of my time. It's the model I have the most confidence in to get the job done right the first time. I put together a full breakdown with all the tests, prompts, and examples on a site. You can read it here:

Matthew Berman

188,148 Aufrufe • vor 2 Monaten

Not long ago, I turned down an offer from a major news network. I read through the terms and realized it came with too many constraints, it was more of a control document than a partnership. So I walked away and chose a smaller outlet instead. The pay was not the same, but they treated me with integrity and gave me the freedom I cared about. No one ever controlled my speech. My mom thought I was making a mistake. She warned me that people who cling to idealism and independence in the media don’t usually end up financially secure. My dad saw it differently. He told me to go for it, decline the fancy offer and go with what I believed, he said that I had the drive to carve my own path and shouldn’t let myself get absorbed into something that didn’t fit. He’s always been in my corner. He's my best friend. I trusted that instinct and went all in. That decision gave me the space to chase my own vision in journalism, and eventually I stepped out completely on my own, no overhead, no constraints. It’s intense, almost nonstop work 24/7, but the payoff is real - I love what I do, I respect the people I work with and I get to stay true to myself. I sleep so well at night! I might not be the richest journalist out there, but I’d argue I’m one of the luckiest. I love my colleagues and I love my friends in the industry. Journalists are often hated today because of the rise of propaganda machines, but some are doing far braver work than many soldiers out there! So support the good ones, you all know who they are. Who do you want to see on the show next?
1:03

Not long ago, I turned down an offer from a major news network. I read through the terms and realized it came with too many constraints, it was more of a control document than a partnership. So I walked away and chose a smaller outlet instead. The pay was not the same, but they treated me with integrity and gave me the freedom I cared about. No one ever controlled my speech. My mom thought I was making a mistake. She warned me that people who cling to idealism and independence in the media don’t usually end up financially secure. My dad saw it differently. He told me to go for it, decline the fancy offer and go with what I believed, he said that I had the drive to carve my own path and shouldn’t let myself get absorbed into something that didn’t fit. He’s always been in my corner. He's my best friend. I trusted that instinct and went all in. That decision gave me the space to chase my own vision in journalism, and eventually I stepped out completely on my own, no overhead, no constraints. It’s intense, almost nonstop work 24/7, but the payoff is real - I love what I do, I respect the people I work with and I get to stay true to myself. I sleep so well at night! I might not be the richest journalist out there, but I’d argue I’m one of the luckiest. I love my colleagues and I love my friends in the industry. Journalists are often hated today because of the rise of propaganda machines, but some are doing far braver work than many soldiers out there! So support the good ones, you all know who they are. Who do you want to see on the show next?

ELIZABETH LANE

53,119 Aufrufe • vor 5 Monaten

Yesterday my father Ned Lang (#1 dad in the world!) drove 6 hours from Upstate NY to the Washington DC Jail Gulag - and was almost DENIED ENTRY!! Every Wednesday my dedicated Dad drives down from our hometown in Narrowsburg NY, and we meet for an hour & pray. Its the highlight of my week. When I have to leave the visiting room and watch him walk away into freedom, I am completely destroyed. I always feel so safe around my big, strong loving father. He's been my rock throughout this struggle. God has blessed me with a tenacious fighter of a father. The horribly mismanaged facility almost didn't him inside, and my lawyer had to call the General Consul of the DC Jail, who told actually him "Just make sure Mr. Lang doesn't post on his Twitter, we can't handle anymore bad publicity right now" !! These tyrants who have tortured us and our families for 4 Christmas' actually want us to stay silent and do them a favor!! After my dad finally got inside to visit, Gods grace was shown: wonder working & mighty!! I went back up to my unit C2a and BAPTIZED 3 MEN into the body Christ!!! Including my J6 brother Joshua Dale who is at court right now, being sentenced. I pray he doesn't come back to my cell tonight and returns home to his wife and children in Tennessee. There are 25 January 6 Political Prisoners left with me inside the DC Jail Gulag and another 200 of us spread throughout the Federal Prison system across America - including 50 in halfway houses!! All of us will be missing our FOURTH Christmas away from our loved ones. In this video my dad made outside the DC Jail yesterday after our visit, you can feel the desperation in his voice, as a father with a heartfelt petition to President Trump: 'Leave No Man Behind in the Gulag - FREE THEM ALL ON DAY ONE!!' I love you dad, Merry Christmas. I hope you and Lexi enjoy your KINGSIZE MyPillows I bought this year for your presents. (Spoiler alert!!) Family is everything. Fight for your families America. #MAGA
1:41

Yesterday my father Ned Lang (#1 dad in the world!) drove 6 hours from Upstate NY to the Washington DC Jail Gulag - and was almost DENIED ENTRY!! Every Wednesday my dedicated Dad drives down from our hometown in Narrowsburg NY, and we meet for an hour & pray. Its the highlight of my week. When I have to leave the visiting room and watch him walk away into freedom, I am completely destroyed. I always feel so safe around my big, strong loving father. He's been my rock throughout this struggle. God has blessed me with a tenacious fighter of a father. The horribly mismanaged facility almost didn't him inside, and my lawyer had to call the General Consul of the DC Jail, who told actually him "Just make sure Mr. Lang doesn't post on his Twitter, we can't handle anymore bad publicity right now" !! These tyrants who have tortured us and our families for 4 Christmas' actually want us to stay silent and do them a favor!! After my dad finally got inside to visit, Gods grace was shown: wonder working & mighty!! I went back up to my unit C2a and BAPTIZED 3 MEN into the body Christ!!! Including my J6 brother Joshua Dale who is at court right now, being sentenced. I pray he doesn't come back to my cell tonight and returns home to his wife and children in Tennessee. There are 25 January 6 Political Prisoners left with me inside the DC Jail Gulag and another 200 of us spread throughout the Federal Prison system across America - including 50 in halfway houses!! All of us will be missing our FOURTH Christmas away from our loved ones. In this video my dad made outside the DC Jail yesterday after our visit, you can feel the desperation in his voice, as a father with a heartfelt petition to President Trump: 'Leave No Man Behind in the Gulag - FREE THEM ALL ON DAY ONE!!' I love you dad, Merry Christmas. I hope you and Lexi enjoy your KINGSIZE MyPillows I bought this year for your presents. (Spoiler alert!!) Family is everything. Fight for your families America. #MAGA

Jake Lang - January 6 Political Prisoner 🇺🇸

42,725 Aufrufe • vor 1 Jahr

For science, AI sovereignty and physics-grounded reasoning are non-negotiable. But how can we teach a small LLM like Gemma-4-E4B physics? One way is to use Agent Skills, but this has so far been limited to closed frontier models. mistral․rs now implements Agent Skills natively: the first self-hosted inference engine that does this as part of the local inference substrate, where we can use small models to solve complex scientific and other tasks in a flexible and scalable way. We are in a period of uncertainty about frontier models - access, pricing, deprecation, abrupt restriction. The good news is that when the entire stack runs locally we can build AI that is entirely your own: You own the weights, the skills, the execution loop, the data - all of it runs on your hardware and is reproducible and durable. While virtually all local inference engines expose a model behind an OpenAI-compatible endpoint, everything agentic is then assembled around it by an external orchestrator that injects context, manages tools, mounts files, and brokers execution. mistral․rs is natively agentic and moves that machinery into the server itself, allowing us to build complex agentic workflows and run them locally, on open-source models. With this new feature you can now upload Agent Skills bundles to /v1/skills, reference them from Responses API requests by identity, and run them inside a native agentic loop with persistent Python sessions, figure capture, sandboxed shell execution, file inputs mounted directly into the working session; plug-and-play and completely compatible with your existing code/workflow. A model with a native skill substrate can act, observe consequences, and can modify what it is able to do. The skill is retained procedural capability of the system. Attached is a short video of all of it: skills, code execution, the full agentic loop carried by Gemma-4-E4B; running entirely on my MacBook Pro. You can install and run a server with this capability in two lines in your terminal, with any quantization you need. Nice work by the Google Gemma team Logan Kilpatrick Demis Hassabis and Eric Buehler with mistral․rs!
1:08

For science, AI sovereignty and physics-grounded reasoning are non-negotiable. But how can we teach a small LLM like Gemma-4-E4B physics? One way is to use Agent Skills, but this has so far been limited to closed frontier models. mistral․rs now implements Agent Skills natively: the first self-hosted inference engine that does this as part of the local inference substrate, where we can use small models to solve complex scientific and other tasks in a flexible and scalable way. We are in a period of uncertainty about frontier models - access, pricing, deprecation, abrupt restriction. The good news is that when the entire stack runs locally we can build AI that is entirely your own: You own the weights, the skills, the execution loop, the data - all of it runs on your hardware and is reproducible and durable. While virtually all local inference engines expose a model behind an OpenAI-compatible endpoint, everything agentic is then assembled around it by an external orchestrator that injects context, manages tools, mounts files, and brokers execution. mistral․rs is natively agentic and moves that machinery into the server itself, allowing us to build complex agentic workflows and run them locally, on open-source models. With this new feature you can now upload Agent Skills bundles to /v1/skills, reference them from Responses API requests by identity, and run them inside a native agentic loop with persistent Python sessions, figure capture, sandboxed shell execution, file inputs mounted directly into the working session; plug-and-play and completely compatible with your existing code/workflow. A model with a native skill substrate can act, observe consequences, and can modify what it is able to do. The skill is retained procedural capability of the system. Attached is a short video of all of it: skills, code execution, the full agentic loop carried by Gemma-4-E4B; running entirely on my MacBook Pro. You can install and run a server with this capability in two lines in your terminal, with any quantization you need. Nice work by the Google Gemma team Logan Kilpatrick Demis Hassabis and Eric Buehler with mistral․rs!

Markus J. Buehler

10,229 Aufrufe • vor 3 Monaten

A single RTX 4090 (24 GB VRAM) can run the updated gemma 4 31B (dense) model with a 190,000 context window at 33 tokens/second. The VRAM barrier is dying. Google quietly updated Gemma 4, and Unsloth immediately compiled the new quants. I built llama.cpp from source on Ubuntu 22 to benchmark it. Google's stealth update 2 days ago enabled uniform Flash Attention 4 on Hopper to boost prefill and patched the chat template to improve tool calling. The agentic reasoning gains on the benchmark charts are massive: TB2 (Agents): +4.5% (to 25.8%) Tau2 (Telecom): +10.1% (to 62.7%) Running on Ubuntu 22, CUDA 13.0 with a single NVIDIA GeForce RTX 4090. Here is the exact step by step benchmarking process with a massive 28k tokens prompt and the commands I used to squeeze out maximum context without killing my throughput: # 1. The Baseline (Unquantized KV Cache) I started with full GPU offload (-ngl 99) and pushed the context to 40k. llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -c 40000 -fa on --port 8080 -v VRAM: 23.8 GB (maxed out on card) Throughput: Prefill: 2198.81 t/s | Decode: 35.77 t/s (with 28k tokens prompt) # 2. The CPU Split Trap I tried stretching to 80k context by offloading layers to the CPU (-ngl 52). llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 80000 -ngl 52 -fa on --port 8080 -v Throughput: Prefill: 1212.73 t/s | Decode: 5 t/s (with 28k tokens prompt) # 3. The KV Quantization Breakthrough Instead of spilling layers to the CPU, I kept the model fully on card (-ngl 99) but enabled 8-bit KV cache quantization to free up VRAM. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 100000 --cache-type-k q8_0 --cache-type-v q8_0 -ngl 99 --port 8080 -v VRAM: 23.9 GB Throughput: Prefill: 2139.68 t/s | Decode: 32 t/s (with 28k tokens prompt) Result: 100k tokens of context on a single GPU with practically zero speed loss (and minimal intelligence loss). # 4. The Limit Test (Q4 KV Cache) To find the absolute breaking point, I dropped the KV cache to 4 bit (q4_0) and set -c 190000. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 190000 --cache-type-k q4_0 --cache-type-v q4_0 -ngl 99 --port 8080 -v VRAM: 23.8 GB Throughput: Prefill: 2206.66 t/s | Decode: 33 t/s (with 28k tokens prompt) (Note: Pushing it to 220k required dropping to -ngl 58 again, which immediately penalized decode down to 17 t/s). # The Tradeoff: For Max Reasoning: Keep your KV cache unquantized (f16). You get pristine reasoning but hit a strict 40k context ceiling. For Massive Document Retrieval: If you need to feed the model giant codebases, use --cache-type-k q4_0. Getting 190k context at 33 tokens/second on a consumer desktop with a 31b dense model is a cheat code. If you’re rocking a single 3090 or 4090 and slept on Gemma 4 earlier, this update is your cue to dust off the terminal. Hugging Face links to the Unsloth QAT quants are in the replies below.
0:20

A single RTX 4090 (24 GB VRAM) can run the updated gemma 4 31B (dense) model with a 190,000 context window at 33 tokens/second. The VRAM barrier is dying. Google quietly updated Gemma 4, and Unsloth immediately compiled the new quants. I built llama.cpp from source on Ubuntu 22 to benchmark it. Google's stealth update 2 days ago enabled uniform Flash Attention 4 on Hopper to boost prefill and patched the chat template to improve tool calling. The agentic reasoning gains on the benchmark charts are massive: TB2 (Agents): +4.5% (to 25.8%) Tau2 (Telecom): +10.1% (to 62.7%) Running on Ubuntu 22, CUDA 13.0 with a single NVIDIA GeForce RTX 4090. Here is the exact step by step benchmarking process with a massive 28k tokens prompt and the commands I used to squeeze out maximum context without killing my throughput: # 1. The Baseline (Unquantized KV Cache) I started with full GPU offload (-ngl 99) and pushed the context to 40k. llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -c 40000 -fa on --port 8080 -v VRAM: 23.8 GB (maxed out on card) Throughput: Prefill: 2198.81 t/s | Decode: 35.77 t/s (with 28k tokens prompt) # 2. The CPU Split Trap I tried stretching to 80k context by offloading layers to the CPU (-ngl 52). llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 80000 -ngl 52 -fa on --port 8080 -v Throughput: Prefill: 1212.73 t/s | Decode: 5 t/s (with 28k tokens prompt) # 3. The KV Quantization Breakthrough Instead of spilling layers to the CPU, I kept the model fully on card (-ngl 99) but enabled 8-bit KV cache quantization to free up VRAM. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 100000 --cache-type-k q8_0 --cache-type-v q8_0 -ngl 99 --port 8080 -v VRAM: 23.9 GB Throughput: Prefill: 2139.68 t/s | Decode: 32 t/s (with 28k tokens prompt) Result: 100k tokens of context on a single GPU with practically zero speed loss (and minimal intelligence loss). # 4. The Limit Test (Q4 KV Cache) To find the absolute breaking point, I dropped the KV cache to 4 bit (q4_0) and set -c 190000. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 190000 --cache-type-k q4_0 --cache-type-v q4_0 -ngl 99 --port 8080 -v VRAM: 23.8 GB Throughput: Prefill: 2206.66 t/s | Decode: 33 t/s (with 28k tokens prompt) (Note: Pushing it to 220k required dropping to -ngl 58 again, which immediately penalized decode down to 17 t/s). # The Tradeoff: For Max Reasoning: Keep your KV cache unquantized (f16). You get pristine reasoning but hit a strict 40k context ceiling. For Massive Document Retrieval: If you need to feed the model giant codebases, use --cache-type-k q4_0. Getting 190k context at 33 tokens/second on a consumer desktop with a 31b dense model is a cheat code. If you’re rocking a single 3090 or 4090 and slept on Gemma 4 earlier, this update is your cue to dust off the terminal. Hugging Face links to the Unsloth QAT quants are in the replies below.

Alok

76,069 Aufrufe • vor 2 Monaten

When I reflect on the story of the Israelites, it’s hard to believe how quickly they took for granted everything God had done for them. He delivered them from slavery with mighty signs and wonders, parted the Red Sea, provided manna from heaven every morning, and guided them with a pillar of cloud by day and fire by night. Yet time and again, they complained, doubted, and turned to their own ways. Then I look at my own life… and I realize I’m not so different. How often has God shown up for me—answered prayers, opened doors, carried me through storms—only for me to slip into worry, control, or ingratitude the moment things feel uncertain? Lord, help me never to live in Your grace only to die in my own greed or self-reliance. Teach me to wait on You, to trust You fully, and to never take Your faithfulness for granted. I surrender control. I choose gratitude. Amen. 🙏
1:50

When I reflect on the story of the Israelites, it’s hard to believe how quickly they took for granted everything God had done for them. He delivered them from slavery with mighty signs and wonders, parted the Red Sea, provided manna from heaven every morning, and guided them with a pillar of cloud by day and fire by night. Yet time and again, they complained, doubted, and turned to their own ways. Then I look at my own life… and I realize I’m not so different. How often has God shown up for me—answered prayers, opened doors, carried me through storms—only for me to slip into worry, control, or ingratitude the moment things feel uncertain? Lord, help me never to live in Your grace only to die in my own greed or self-reliance. Teach me to wait on You, to trust You fully, and to never take Your faithfulness for granted. I surrender control. I choose gratitude. Amen. 🙏

NancyH

10,233 Aufrufe • vor 4 Monaten