Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Congratulations to the Cohere team on releasing North Mini Code 1.0 🔥 We provide day-0 support on MLX thanks to our close collaboration with the Cohere team. North Mini Code 1.0 is a 30B param MoE model with 3B active. It runs at ~66 tok/s in BF16, truly impressive... show more

Prince Canuma

22,202 subscribers

35,367 Aufrufe • vor 4 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Congratulations to the Cohere team on the release of Cohere Transcribe Arabic! 🎉 Runs natively on mlx-audio (Python + Swift) from day-0 🚀 What's inside: → 2B params, Conformer encoder-decoder (audio-in, text-out) → Large Conformer encoder for acoustic representations + lightweight Transformer decoder for token generation → Built for Arabic dialects and Arabic-English code-switching → Auto-resampling to 16kHz + stereo→mono handling built into preprocessing → Apache 2.0 — fully open for community research Currently topping the Open Universal Arabic ASR Leaderboard 🥇 Get started today: 🐍 Python uv pip install -U mlx-audio 🍎 Swift

Congratulations to the Cohere team on the release of Cohere Transcribe Arabic! 🎉 Runs natively on mlx-audio (Python + Swift) from day-0 🚀 What's inside: → 2B params, Conformer encoder-decoder (audio-in, text-out) → Large Conformer encoder for acoustic representations + lightweight Transformer decoder for token generation → Built for Arabic dialects and Arabic-English code-switching → Auto-resampling to 16kHz + stereo→mono handling built into preprocessing → Apache 2.0 — fully open for community research Currently topping the Open Universal Arabic ASR Leaderboard 🥇 Get started today: 🐍 Python uv pip install -U mlx-audio 🍎 Swift

Prince Canuma

16,680 Aufrufe • vor 3 Monaten

Nothing beats open-source! We now have a fully open-source coding agent that can compete with any proprietary solution out there: • 500+ available models • It doesn't train on your code • It doesn't charge a penny for tokens • It's open. You can change if you want. Kilo Code is the #1 app on OpenRouter right now, and the community is releasing new features on GitHub at rocket speed (about 20 releases in the last 30 days). I installed it in Visual Studio Code, but you can install it in pretty much any IDE. I need to spend more time with it, but here is a very interesting design choice: Most coding agents support two modes: an "agentic" mode for writing code and an "ask" mode for talking to the model without making any changes. Kilo Code supports 5 different modes: 1. Architect mode - Research, design, and planning 2. Debug mode - Problem solving and troubleshooting 3. Code mode - To write code. 4. Ask mode - To talk to the model. 5. Orchestrator mode - Combine every agent to solve problems. On paper, I really like this. I'll need more time to try out the specialized agents and see whether niching becomes an advantage. Here is the link to the website: Thanks to the Kilo (acq. by Anaconda) for partnering with me on this post.

Nothing beats open-source! We now have a fully open-source coding agent that can compete with any proprietary solution out there: • 500+ available models • It doesn't train on your code • It doesn't charge a penny for tokens • It's open. You can change if you want. Kilo Code is the #1 app on OpenRouter right now, and the community is releasing new features on GitHub at rocket speed (about 20 releases in the last 30 days). I installed it in Visual Studio Code, but you can install it in pretty much any IDE. I need to spend more time with it, but here is a very interesting design choice: Most coding agents support two modes: an "agentic" mode for writing code and an "ask" mode for talking to the model without making any changes. Kilo Code supports 5 different modes: 1. Architect mode - Research, design, and planning 2. Debug mode - Problem solving and troubleshooting 3. Code mode - To write code. 4. Ask mode - To talk to the model. 5. Orchestrator mode - Combine every agent to solve problems. On paper, I really like this. I'll need more time to try out the specialized agents and see whether niching becomes an advantage. Here is the link to the website: Thanks to the Kilo (acq. by Anaconda) for partnering with me on this post.

Santiago

73,304 Aufrufe • vor 11 Monaten

This is literally the easiest way to build an AI agent. Zero code. You only need to run a couple of commands. (I've said this before, but Google ADK is my favorite way to build code-first agents.) Here is what you need to do: 1. Open your terminal and create an empty folder 2. Run `uv init` inside the folder (if you don't have uv installed, you are really missing out!) 3. Install the google-adk library using the following command: `uv add google-adk`. 4. At this point, you can use the ask to create your agent with a single command: `uv run adk create --type=config my_agent`. This will ask you a couple of questions, including your GEMINI_API_KEY (you can create one online). 5. The command above will create a my_agent/root_agent.yaml file. Open it and add the following two lines at the end of the file so your agent can use Google to search for stuff: tools: - name: google_search 6. You have everything you need now. Run the adk web interface and start playing with your agent: `uv run adk web` That's it! Literally zero code. The agent works off the YAML configuration file.

This is literally the easiest way to build an AI agent. Zero code. You only need to run a couple of commands. (I've said this before, but Google ADK is my favorite way to build code-first agents.) Here is what you need to do: 1. Open your terminal and create an empty folder 2. Run `uv init` inside the folder (if you don't have uv installed, you are really missing out!) 3. Install the google-adk library using the following command: `uv add google-adk`. 4. At this point, you can use the ask to create your agent with a single command: `uv run adk create --type=config my_agent`. This will ask you a couple of questions, including your GEMINI_API_KEY (you can create one online). 5. The command above will create a my_agent/root_agent.yaml file. Open it and add the following two lines at the end of the file so your agent can use Google to search for stuff: tools: - name: google_search 6. You have everything you need now. Run the adk web interface and start playing with your agent: `uv run adk web` That's it! Literally zero code. The agent works off the YAML configuration file.

Santiago

140,922 Aufrufe • vor 10 Monaten

Which models can your machine actually run? Magnitude is a 100% free, open source desktop app that: - profiles your hardware and runs sample calculations - predicts tok/s for every model before you download - recommends the best models, from fast to smart Pick your models and it handles the rest: - downloads and tunes the models for your hardware - connects agents like pi, opencode, hermes in one click - runs models on demand as your agent works Works on whatever hardware you already own: MacBook (Intel or Apple Silicon), Mac Mini, Mac Studio, DGX Spark, Strix Halo, any NVIDIA/AMD GPU, or just a CPU Download it on macOS, Windows, or Linux: Open source:

Which models can your machine actually run? Magnitude is a 100% free, open source desktop app that: - profiles your hardware and runs sample calculations - predicts tok/s for every model before you download - recommends the best models, from fast to smart Pick your models and it handles the rest: - downloads and tunes the models for your hardware - connects agents like pi, opencode, hermes in one click - runs models on demand as your agent works Works on whatever hardware you already own: MacBook (Intel or Apple Silicon), Mac Mini, Mac Studio, DGX Spark, Strix Halo, any NVIDIA/AMD GPU, or just a CPU Download it on macOS, Windows, or Linux: Open source:

Tom Greenwald

49,258 Aufrufe • vor 25 Tagen

My Hermes Install Step by Step: 1. Install Ubuntu Linux on a machine – Once installed make sure you have the latest updates 2. Install both Google Chrome and Microsoft Edge browser and log in to your accounts to synch bookmarks/favorites 3. Set up an Ollama account at – I have the annual Pro Plan (if I can ever come up with the funds I will probably spring for the Max plan but Pro is simply awesome) 4. Install Ollama and then run “ollama run glm-5.1:cloud” – It will then have you authenticate to your account and add your machine 5. Install Hermes (watch my video for explanation around this as you may get interrupted during install) 6. Once Hermes is installed you will be prompted to configure. I chose the default quick configure. For model provider select Ollama, provide your Ollama API key, then for model choose desired model. At this point in time I recommend glm-5.1 7. Download Obsidian as your Hermes second brain, set up a vault then tell Hermes to integrate it and provide Hermes the vault location 8. Start building and have fun!

My Hermes Install Step by Step: 1. Install Ubuntu Linux on a machine – Once installed make sure you have the latest updates 2. Install both Google Chrome and Microsoft Edge browser and log in to your accounts to synch bookmarks/favorites 3. Set up an Ollama account at – I have the annual Pro Plan (if I can ever come up with the funds I will probably spring for the Max plan but Pro is simply awesome) 4. Install Ollama and then run “ollama run glm-5.1:cloud” – It will then have you authenticate to your account and add your machine 5. Install Hermes (watch my video for explanation around this as you may get interrupted during install) 6. Once Hermes is installed you will be prompted to configure. I chose the default quick configure. For model provider select Ollama, provide your Ollama API key, then for model choose desired model. At this point in time I recommend glm-5.1 7. Download Obsidian as your Hermes second brain, set up a vault then tell Hermes to integrate it and provide Hermes the vault location 8. Start building and have fun!

Mike Gannotti

109,010 Aufrufe • vor 5 Monaten

Pi was built when there were already agent harnesses around. Here’s why Mario Zechner(Mario Zechner), found them suboptimal and built Pi, a minimalist self-modifying agent: #1 - Mario initially was a believer in Claude Code: "I was a believer in Claude code because they were the first that packaged agentic search up in a really compelling package. And at the time that fit my workflow really well. Everything around the LLM was kind of nice and tidy and easy to understand. I was super happy. I was proselytising Claude code." #2 - Reverse engineering Claude Code highlighted the degradation that Mario felt as a user: "I personally like simple tools that are stable and that I can rely on. Even if they have non-deterministic parts, all the deterministic parts should be as stable as possible. That was just not the experience with Claude Code around summer 2025. They would take away your control of the context. They would inject stuff behind your back, which is bad. Then, your workflows stopped working because there's now a system reminder that you don't even see in the UI that would modify the behaviour of the model. They would also do this to the system prompt. I built a little service where I can track the progression or evolution of the system, prompt and tool definitions and, with every release, it was messing with stuff. That just messed with my workflows and I don't appreciate that." #3 - PI was built with an appreciation for simple and reliable tools: "If I commit to a development tool, I want it to be a stable, reliable thing like a hammer. I don't want my hammer to break a different spot every day. That's terrible. We need somebody who goes the full velocity kind of way. But I don't want to work with a tool like that."

Pi was built when there were already agent harnesses around. Here’s why Mario Zechner(Mario Zechner), found them suboptimal and built Pi, a minimalist self-modifying agent: #1 - Mario initially was a believer in Claude Code: "I was a believer in Claude code because they were the first that packaged agentic search up in a really compelling package. And at the time that fit my workflow really well. Everything around the LLM was kind of nice and tidy and easy to understand. I was super happy. I was proselytising Claude code." #2 - Reverse engineering Claude Code highlighted the degradation that Mario felt as a user: "I personally like simple tools that are stable and that I can rely on. Even if they have non-deterministic parts, all the deterministic parts should be as stable as possible. That was just not the experience with Claude Code around summer 2025. They would take away your control of the context. They would inject stuff behind your back, which is bad. Then, your workflows stopped working because there's now a system reminder that you don't even see in the UI that would modify the behaviour of the model. They would also do this to the system prompt. I built a little service where I can track the progression or evolution of the system, prompt and tool definitions and, with every release, it was messing with stuff. That just messed with my workflows and I don't appreciate that." #3 - PI was built with an appreciation for simple and reliable tools: "If I commit to a development tool, I want it to be a stable, reliable thing like a hammer. I don't want my hammer to break a different spot every day. That's terrible. We need somebody who goes the full velocity kind of way. But I don't want to work with a tool like that."

The Pragmatic Engineer

63,020 Aufrufe • vor 5 Monaten

AMD'S NEW MINI PC RUNS CLAUDE CODE OFFLINE FOR $0/MONTH AND REPLACES $200 OPENAI + $200 CLAUDE CODE SUBSCRIPTIONS amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on a single piece of silicon with 128gb of unified memory the gmktec evo-x2 runs qwen3 coder 30b at 40 to 50 tokens per second and llama 3.3 70b at 20 to 25 tokens per second locally. on linux you get 110gb of usable vram out of 128gb setup takes 90 minutes. wipe windows for linux, install ollama with one command, pull the model, point claude code at localhost. same interface, same agent loop, nothing leaves your network a heavy ai user pays $200 for claude code max, $200 for chatgpt pro and $20 for cursor. that's $5,040 a year and the box pays itself off in 5 months after break-even every month is $420 that used to leave your account. three year math is roughly $13,000 not paid to ai companies plus zero rate limits on overnight agents bookmark this and read the article below

AMD'S NEW MINI PC RUNS CLAUDE CODE OFFLINE FOR $0/MONTH AND REPLACES $200 OPENAI + $200 CLAUDE CODE SUBSCRIPTIONS amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on a single piece of silicon with 128gb of unified memory the gmktec evo-x2 runs qwen3 coder 30b at 40 to 50 tokens per second and llama 3.3 70b at 20 to 25 tokens per second locally. on linux you get 110gb of usable vram out of 128gb setup takes 90 minutes. wipe windows for linux, install ollama with one command, pull the model, point claude code at localhost. same interface, same agent loop, nothing leaves your network a heavy ai user pays $200 for claude code max, $200 for chatgpt pro and $20 for cursor. that's $5,040 a year and the box pays itself off in 5 months after break-even every month is $420 that used to leave your account. three year math is roughly $13,000 not paid to ai companies plus zero rate limits on overnight agents bookmark this and read the article below

starmex

33,516 Aufrufe • vor 4 Monaten

27B model in 8.45 GB. 60 tok/s on a single 3090. And it built this voxel pagoda from a single prompt. Qwen3.8-27B in 2.4 bits from Mirai Labs is on a completely different level compared to the mess from PrismML. Yeah, it's slightly larger in size, but these guys don't claim 98% accuracy either, unlike some others. Pagoda: 1 prompt, medium reasoning, pi agent, 10 turns, 32.5K output tokens, 12 minutes. A single HTML file on three.js. Speed on my rig, GPU at 290 W, vLLM: 1x 3090: 60 tok/s with MTP, 44 without, prefill ~1190 tok/s on a 7K prompt, 159K context 2x 3090 (pipeline parallel): 66 tok/s, prefill ~1890 tok/s, full 262K context Huge thanks to Artur Chakhvadze and ryan mathieu for the opportunity to test this model before release and help the team with testing.

27B model in 8.45 GB. 60 tok/s on a single 3090. And it built this voxel pagoda from a single prompt. Qwen3.8-27B in 2.4 bits from Mirai Labs is on a completely different level compared to the mess from PrismML. Yeah, it's slightly larger in size, but these guys don't claim 98% accuracy either, unlike some others. Pagoda: 1 prompt, medium reasoning, pi agent, 10 turns, 32.5K output tokens, 12 minutes. A single HTML file on three.js. Speed on my rig, GPU at 290 W, vLLM: 1x 3090: 60 tok/s with MTP, 44 without, prefill ~1190 tok/s on a 7K prompt, 159K context 2x 3090 (pipeline parallel): 66 tok/s, prefill ~1890 tok/s, full 262K context Huge thanks to Artur Chakhvadze and ryan mathieu for the opportunity to test this model before release and help the team with testing.

Alexey Fateev

29,548 Aufrufe • vor 18 Tagen

Nothing beats Augment Code answering questions about large codebases. It's a free extension (works in VSCode, JetBrains, and NeoVim), and has a community edition you can run for free forever. Try this experiment: 1. Download any large open-source project from GitHub 2. Augment will index the code 3. Start asking questions about the codebase 4. Do the same using any other coding agent 5. Compare the speed and the depth of the answers you get If you have a small codebase, the differences with other coding agents won't be obvious, but if you are working on a large project, things will click immediately. Suggestion: download the Keras repository (~180,000 lines of code). I know the library well, and the answers I got from Augment were pretty impressive. By the way, Augment is a fully-fledged AI coding assistant. You can use its agent in every imaginable way: • To implement brand new functionality • To make quick edits to your code • To refactor existing functionality • To generate unit tests • To generate documentation • To handle GitHub—pull requests, commits, branching, etc My favorite feature is their "Next Edit" functionality: 1. You edit the code 2. Augment analyzes the change (across the entire codebase!) 3. Augment determines the ripple effects of that change 4. Augment suggests everything you need to update Here is a link to the tool: Thanks to the team at Augment Code for partnering with me on this post.

Santiago

34,977 Aufrufe • vor 1 Jahr

Write 100x better code using AI (for free!) Install the Codacy extension (VSCode, Cursor, WindSurf). This will install an MCP Server locally on your computer. Your coding copilot will use this server with the Codacy CLI to check every line of code it generates. The way this works is pretty awesome (assuming you are using Cursor): 1. You ask Cursor to generate some code 2. Cursor writes the code 3. Cursor then calls Codacy's MCP Server to check it 4. The MCP Server will use the Codacy CLI to find any issues 5. Cursor will fix the issues and call the MCP server again 6. When the code passes all checks, you are done Level of effort on your side: literally zero! Code quality and security because of this: 100x better! Codacy has been doing this for a long, long time, and their Guardrails MCP Server checks for a lot of potential issues: • Security issues • Quality standards • Pattern violations • Linting issues You can see more information about the extension here: I recorded a quick video to show you how it works. Thanks to the Codacy team for helping me set up everything, answering my 1,000 questions, and collaborating with me on this post.

Write 100x better code using AI (for free!) Install the Codacy extension (VSCode, Cursor, WindSurf). This will install an MCP Server locally on your computer. Your coding copilot will use this server with the Codacy CLI to check every line of code it generates. The way this works is pretty awesome (assuming you are using Cursor): 1. You ask Cursor to generate some code 2. Cursor writes the code 3. Cursor then calls Codacy's MCP Server to check it 4. The MCP Server will use the Codacy CLI to find any issues 5. Cursor will fix the issues and call the MCP server again 6. When the code passes all checks, you are done Level of effort on your side: literally zero! Code quality and security because of this: 100x better! Codacy has been doing this for a long, long time, and their Guardrails MCP Server checks for a lot of potential issues: • Security issues • Quality standards • Pattern violations • Linting issues You can see more information about the extension here: I recorded a quick video to show you how it works. Thanks to the Codacy team for helping me set up everything, answering my 1,000 questions, and collaborating with me on this post.

Santiago

63,316 Aufrufe • vor 1 Jahr

Last night, I asked my Builder Grok Bot to send a job to Cursor. I asked it to have Cursor create an “explainer” video pipeline that follows a set of specific instructions, and is easy to run whenever I want to create an educational video vs starting from scratch every time. My Grok Bot started up a Cursor project in the cloud. One Cursor coordinator agent ran the project and handed work to two others. The first working agent built the pipeline in the cloud. After it finished, it sent the work from the cloud down to a Cursor worker living in my Mac Mini. The Mac Mini worker installed all of the programs necessary for building these videos, and will follow these instructions for all future builds when I run this pipeline: 1. Claude Code on the Mini, signed in to my Claude Pro subscription, writes the research, the script and the animation plan. 2. Python on the Mini draws the animations. 3. Kokoro on the Mini records the narration. 4. ffmpeg on the Mini adds the music and captions and exports the MP4. 5. The finished video saves to my iCloud folder. Now every time I ask my Grok Bot to make an educational video, it passes it through this pipeline on my Mac mini. All of the tools are already installed, and the steps are set in stone. All I need to do is give it a topic and it knows to run this. This all happens on my Mac Mini now, and doesn’t involve cloud agents unless I want to bring one on to edit the pipeline. No need to start from scratch every time. Here’s the first video explaining how the power grid works:

Last night, I asked my Builder Grok Bot to send a job to Cursor. I asked it to have Cursor create an “explainer” video pipeline that follows a set of specific instructions, and is easy to run whenever I want to create an educational video vs starting from scratch every time. My Grok Bot started up a Cursor project in the cloud. One Cursor coordinator agent ran the project and handed work to two others. The first working agent built the pipeline in the cloud. After it finished, it sent the work from the cloud down to a Cursor worker living in my Mac Mini. The Mac Mini worker installed all of the programs necessary for building these videos, and will follow these instructions for all future builds when I run this pipeline: 1. Claude Code on the Mini, signed in to my Claude Pro subscription, writes the research, the script and the animation plan. 2. Python on the Mini draws the animations. 3. Kokoro on the Mini records the narration. 4. ffmpeg on the Mini adds the music and captions and exports the MP4. 5. The finished video saves to my iCloud folder. Now every time I ask my Grok Bot to make an educational video, it passes it through this pipeline on my Mac mini. All of the tools are already installed, and the steps are set in stone. All I need to do is give it a topic and it knows to run this. This all happens on my Mac Mini now, and doesn’t involve cloud agents unless I want to bring one on to edit the pipeline. No need to start from scratch every time. Here’s the first video explaining how the power grid works:

Mike P

10,933 Aufrufe • vor 13 Tagen

I installed a new Visual Studio Code extension to help me write code. But this extension doesn't get in your way. It doesn't try to write code for you. It only gives you hints and recommendations about your code. I like that. Like a second pair of eyes, looking over my shoulder and telling me how I can do things better. The extension is Traycer. You can install it directly from inside Visual Studio Code. It's free forever for anyone working on open-source projects (and if you aren't, you can use it for free until the end of June.) I recorded a super-duper-quick video to show you how this extension works. I know code generation is a huge thing, but I really appreciate tools that help me do what I do better. Thanks to the Traycer team for helping me with the extension and collaborating with me on this post!

I installed a new Visual Studio Code extension to help me write code. But this extension doesn't get in your way. It doesn't try to write code for you. It only gives you hints and recommendations about your code. I like that. Like a second pair of eyes, looking over my shoulder and telling me how I can do things better. The extension is Traycer. You can install it directly from inside Visual Studio Code. It's free forever for anyone working on open-source projects (and if you aren't, you can use it for free until the end of June.) I recorded a super-duper-quick video to show you how this extension works. I know code generation is a huge thing, but I really appreciate tools that help me do what I do better. Thanks to the Traycer team for helping me with the extension and collaborating with me on this post!

Santiago

101,068 Aufrufe • vor 2 Jahren

Hyperagent is great if you want agents without having to deal with configuration, servers, or any complexity whatsoever. You don't need a Mac Mini or a server. Every agent runs in the cloud, in its own isolated and secure environment. You get: 1. The ability to learn any new API as a skill 2. Browser support 3. Code execution 4. Image and video generation 5. Integration with hundreds of tools Despite the simplicity, I think their best differentiator is the ability for these agents to learn new skills. Basically, you can teach an agent how your company does marketing, and it will never forget your rules and processes. Obviously, it also lets you deploy your agents to Slack with a single click, so everyone can use them. You don't even need to mention them since they will pay attention to chats and act when necessary. I built a simple agent that allows me to translate any message into French. I installed it on Slack, and every time I ping it, it gives me translations. Took me 10 minutes tops.

Hyperagent is great if you want agents without having to deal with configuration, servers, or any complexity whatsoever. You don't need a Mac Mini or a server. Every agent runs in the cloud, in its own isolated and secure environment. You get: 1. The ability to learn any new API as a skill 2. Browser support 3. Code execution 4. Image and video generation 5. Integration with hundreds of tools Despite the simplicity, I think their best differentiator is the ability for these agents to learn new skills. Basically, you can teach an agent how your company does marketing, and it will never forget your rules and processes. Obviously, it also lets you deploy your agents to Slack with a single click, so everyone can use them. You don't even need to mention them since they will pay attention to chats and act when necessary. I built a simple agent that allows me to translate any message into French. I installed it on Slack, and every time I ping it, it gives me translations. Took me 10 minutes tops.

Santiago

213,793 Aufrufe • vor 2 Monaten

Mirai Labs (Mirai Labs) compressed Qwen3.8-27B to 2.4 bits. I ported it to llama.cpp and now it runs on a 12 GB card with 128K context. With q4 KV cache the 128K window takes 11.3 GB of VRAM, with q8 KV you get 74K in 11.1 GB. Decode is 40 tok/s, still 34-35 tok/s at 62K deep, and prefill around 1000 tok/s. I measured all of it on 1 RTX 3090 at 300 W, capped to what fits in 12 GB. No requant anywhere. The GGUF carries Mirai's compressed codes bit for bit, and greedy output matches their own vLLM plugin. Everything in the video was built by this same model as an agent in pi, 11 builds from 1 prompt each, a few needed a follow up and 1 is a blooper. That run was vLLM on my 3090s. Today the same weights run in llama.cpp. Big thread, all links and recipes 👇

Mirai Labs (Mirai Labs) compressed Qwen3.8-27B to 2.4 bits. I ported it to llama.cpp and now it runs on a 12 GB card with 128K context. With q4 KV cache the 128K window takes 11.3 GB of VRAM, with q8 KV you get 74K in 11.1 GB. Decode is 40 tok/s, still 34-35 tok/s at 62K deep, and prefill around 1000 tok/s. I measured all of it on 1 RTX 3090 at 300 W, capped to what fits in 12 GB. No requant anywhere. The GGUF carries Mirai's compressed codes bit for bit, and greedy output matches their own vLLM plugin. Everything in the video was built by this same model as an agent in pi, 11 builds from 1 prompt each, a few needed a follow up and 1 is a blooper. That run was vLLM on my 3090s. Today the same weights run in llama.cpp. Big thread, all links and recipes 👇

Alexey Fateev

31,893 Aufrufe • vor 17 Tagen

OpenClaw - the agentic software spreading like wildfire - was built on top of Pi, a minimalist, self-modifying agent. I sat down with Pi's creator, Mario Zechner and longtime Pi user (+ the creator of Flask) Armin Ronacher ⇌ to talk Pi, and their (very grounded!) takes on building with AI. Timestamps: 00:00 Intro 07:30 How Mario, Armin, and Peter Steinberger met 15:15 How 30 dev teams use AI agents: learnings 21:50 The importance of judgment 24:26 Challenges when non-engineers write code 28:30 Downsides of over-automation 32:18 Pi 48:09 OpenClaw + Pi 50:54 “Clankers” 57:32 Open source and AI 1:00:22 Complexity as the enemy 1:02:50 Building an AI-native startup 1:11:52 “Slow the F down” 1:16:40 MCPs vs. CLI 1:25:03 Predictions and staying up to date • YouTube: • Spotify: • Apple: Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar — The makers of SonarQube, the industry standard for code verification and automated code review. Try it out for yourself. • WorkOS – WorkOS gives you APIs to ship enterprise features – SSO, directory sync, RBAC, audit logs – in days, not months. Visit learn more. --- Three parts I found especially interesting in this discussion: 1. New trend: AI makes it harder for senior engineers to reject pointless complexity. Historically, senior engineers kept software complexity at bay simply by saying “no” a lot. But Armin observes that these days, more junior engineers and product managers deploy agent-scripted counterarguments when a senior colleague kicks an idea to the curb. This makes decision-making exhausting, and more bad ideas make it into production as a result. 2. It should be MUCH easier to build specialized tools for specific tasks. Different projects need different harness types because, as Mario points out, the same hammer is not ideal for every single construction job. As such, Pi is built with the goal of allowing the creation of specialized harnesses. It can modify itself so that a user can create the bespoke harness needed for any task. Mario believes it’s a preview of how self-modifiable software might look in the future. 3. Automation bias is one of the biggest risks of working with AI agents. Once devs confirm that an AI agent can produce acceptable code, they start to review its output less often, even though agents can – and do! – produce slop. Mario advises being far more sceptical with agents, and cautions that the quality of their output isn’t guaranteed, however well they performed previously.

OpenClaw - the agentic software spreading like wildfire - was built on top of Pi, a minimalist, self-modifying agent. I sat down with Pi's creator, Mario Zechner and longtime Pi user (+ the creator of Flask) Armin Ronacher ⇌ to talk Pi, and their (very grounded!) takes on building with AI. Timestamps: 00:00 Intro 07:30 How Mario, Armin, and Peter Steinberger met 15:15 How 30 dev teams use AI agents: learnings 21:50 The importance of judgment 24:26 Challenges when non-engineers write code 28:30 Downsides of over-automation 32:18 Pi 48:09 OpenClaw + Pi 50:54 “Clankers” 57:32 Open source and AI 1:00:22 Complexity as the enemy 1:02:50 Building an AI-native startup 1:11:52 “Slow the F down” 1:16:40 MCPs vs. CLI 1:25:03 Predictions and staying up to date • YouTube: • Spotify: • Apple: Brought to you by: • Statsig – ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar — The makers of SonarQube, the industry standard for code verification and automated code review. Try it out for yourself. • WorkOS – WorkOS gives you APIs to ship enterprise features – SSO, directory sync, RBAC, audit logs – in days, not months. Visit learn more. --- Three parts I found especially interesting in this discussion: 1. New trend: AI makes it harder for senior engineers to reject pointless complexity. Historically, senior engineers kept software complexity at bay simply by saying “no” a lot. But Armin observes that these days, more junior engineers and product managers deploy agent-scripted counterarguments when a senior colleague kicks an idea to the curb. This makes decision-making exhausting, and more bad ideas make it into production as a result. 2. It should be MUCH easier to build specialized tools for specific tasks. Different projects need different harness types because, as Mario points out, the same hammer is not ideal for every single construction job. As such, Pi is built with the goal of allowing the creation of specialized harnesses. It can modify itself so that a user can create the bespoke harness needed for any task. Mario believes it’s a preview of how self-modifiable software might look in the future. 3. Automation bias is one of the biggest risks of working with AI agents. Once devs confirm that an AI agent can produce acceptable code, they start to review its output less often, even though agents can – and do! – produce slop. Mario advises being far more sceptical with agents, and cautions that the quality of their output isn’t guaranteed, however well they performed previously.

Gergely Orosz

173,334 Aufrufe • vor 5 Monaten

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own

NO1ennn

40,079 Aufrufe • vor 3 Monaten

HOTT NEW PRODUCT!!!! 🚨🚨🚨 Turn your Tesla EV into a power source. Charge 90% of your home appliances w/ your Model 3/Y/S/X car battery using a V2L and rescue other EVs with an NACS port. This is new for North America including a function that bypasses preheating the battery, saving time and energy. 2kW of power output in North America!! That’s 16.7A at 120VAC. 4kW in Europe. My friends at Tlyard extended a special 28% one-time discount using code: TECHGEEK until the end of January. It’s the best and steepest discount on their site!! This is excellent for power outages to power your home, and it serves as an excellent accessory for camping! Here’s the link: Full video review in comment below. 👇

HOTT NEW PRODUCT!!!! 🚨🚨🚨 Turn your Tesla EV into a power source. Charge 90% of your home appliances w/ your Model 3/Y/S/X car battery using a V2L and rescue other EVs with an NACS port. This is new for North America including a function that bypasses preheating the battery, saving time and energy. 2kW of power output in North America!! That’s 16.7A at 120VAC. 4kW in Europe. My friends at Tlyard extended a special 28% one-time discount using code: TECHGEEK until the end of January. It’s the best and steepest discount on their site!! This is excellent for power outages to power your home, and it serves as an excellent accessory for camping! Here’s the link: Full video review in comment below. 👇

TechGeek Tesla 🔋⚡️

44,439 Aufrufe • vor 9 Monaten

8 rules to improve your AI coding agent. All of these rules work with Claude Code, Cursor, VS Code, and with most programming languages. Automating these rules will 10x the code quality and security produced by your AI coding agents. 1. Dependency checks - Prevent your agent from suggesting insecure libraries based on outdated training data. 2. Secret exposure - Auto-fix the use of hardcoded credentials introduced by your coding agent. 3. File and function size - Automatically refactor any files or functions that exceed a reasonable length. 4. Complexity and parameter limits - Simplify overly complex code written by the agent. 5. SQL Injection - Auto-fix all database interactions with unsanitized user input. 6. Unused variables and imports - Detect and remove dead code. 7. Detect invisible unicode characters in AI rules files - Remove zero-width spaces, direction overrides, and other invisible characters that can hide malicious behavior. 8. Insecure OpenAI API usage - Enforce use of secure OpenAI endpoints, proper authentication, and context isolation Here is how you can automate this: Install the Codacy extension. This will give you access to a CLI for local scanning and an MCP server for agent communication. From here on out, every time you need to generate some code: 1. Your agent will write the code 2. It will then call Codacy's CLI to check it 3. It will find any issues in real time 4. Your coding agent will fix the issues 5. When the code passes all checks, you are done Level of effort on your side: literally zero! Code quality and security because of this: 100x better! Here is the link to download the extension for your IDE: Thanks to the Codacy team for collaborating with me on this post.

8 rules to improve your AI coding agent. All of these rules work with Claude Code, Cursor, VS Code, and with most programming languages. Automating these rules will 10x the code quality and security produced by your AI coding agents. 1. Dependency checks - Prevent your agent from suggesting insecure libraries based on outdated training data. 2. Secret exposure - Auto-fix the use of hardcoded credentials introduced by your coding agent. 3. File and function size - Automatically refactor any files or functions that exceed a reasonable length. 4. Complexity and parameter limits - Simplify overly complex code written by the agent. 5. SQL Injection - Auto-fix all database interactions with unsanitized user input. 6. Unused variables and imports - Detect and remove dead code. 7. Detect invisible unicode characters in AI rules files - Remove zero-width spaces, direction overrides, and other invisible characters that can hide malicious behavior. 8. Insecure OpenAI API usage - Enforce use of secure OpenAI endpoints, proper authentication, and context isolation Here is how you can automate this: Install the Codacy extension. This will give you access to a CLI for local scanning and an MCP server for agent communication. From here on out, every time you need to generate some code: 1. Your agent will write the code 2. It will then call Codacy's CLI to check it 3. It will find any issues in real time 4. Your coding agent will fix the issues 5. When the code passes all checks, you are done Level of effort on your side: literally zero! Code quality and security because of this: 100x better! Here is the link to download the extension for your IDE: Thanks to the Codacy team for collaborating with me on this post.

Santiago

49,331 Aufrufe • vor 11 Monaten

Try out Gemini Code: a coding assistant in your terminal powered by Gemini 2.5 Pro. 💜 💜 Release Notes: → It's easy to get started, just run `pip install gemini-code` in a virtual environment, and then run `gemini` to initialize it. It requires your own API key, so don't forget to get one from → I built this because I love Claude Code and wanted a similar experience -- I like that I can just use my own API key and Gemini is generally way cheaper as well. → Found a bug? Feel free to fork the code and fix it (lol)! If you want to try to improve it, please do so - repo is public! → Reminder and a disclaimer that I don't work at Google and this is not an official Google project - just a weekend side hack → It is highly likely you'll get throttled by rate limits - I put as much info as I can in the guide so you know what behavior to expect when that happens. You can also set which default model to use if you want to switch to 1.5 Pro.

Try out Gemini Code: a coding assistant in your terminal powered by Gemini 2.5 Pro. 💜 💜 Release Notes: → It's easy to get started, just run `pip install gemini-code` in a virtual environment, and then run `gemini` to initialize it. It requires your own API key, so don't forget to get one from → I built this because I love Claude Code and wanted a similar experience -- I like that I can just use my own API key and Gemini is generally way cheaper as well. → Found a bug? Feel free to fork the code and fix it (lol)! If you want to try to improve it, please do so - repo is public! → Reminder and a disclaimer that I don't work at Google and this is not an official Google project - just a weekend side hack → It is highly likely you'll get throttled by rate limits - I put as much info as I can in the guide so you know what behavior to expect when that happens. You can also set which default model to use if you want to switch to 1.5 Pro.

Raiza Martin

159,723 Aufrufe • vor 1 Jahr

MLX + TurboQuant = Local Super Power Take a local (private) document(s) or codebase, pre-fill the 256k KV cache (the context) with the document(s) and system prompt, quantise and run on Apple's MLX and you have almost instantaneous, lossless document queries with total privacy. For a 75-page PDF (some 30k tokens), I was seeing sub-150ms TTFT (Time To First Token), that's "instant" for a human. We run this on an office server (one of our Mac Minis) for staff to work with, instant answers on critical, confidential company/client documents on phone, iPads or laptops. Imagine the "What were the 3Q25 profits?", "When did Joe start working for us?", "Please list the project milestones and team count" answered within seconds of hitting the return key. It works with voice too. This example is 17k lines of code from 177 source files plus an index of the entire 1,902 files, the entire Claude Code codebase (oops Anthropic!). Zero loss (unlike Anthropic's security). What this means is the ability to work better on local (private) code bases, off-the-shelf private document analysis with better response than the best public models - all locally.

MLX + TurboQuant = Local Super Power Take a local (private) document(s) or codebase, pre-fill the 256k KV cache (the context) with the document(s) and system prompt, quantise and run on Apple's MLX and you have almost instantaneous, lossless document queries with total privacy. For a 75-page PDF (some 30k tokens), I was seeing sub-150ms TTFT (Time To First Token), that's "instant" for a human. We run this on an office server (one of our Mac Minis) for staff to work with, instant answers on critical, confidential company/client documents on phone, iPads or laptops. Imagine the "What were the 3Q25 profits?", "When did Joe start working for us?", "Please list the project milestones and team count" answered within seconds of hitting the return key. It works with voice too. This example is 17k lines of code from 177 source files plus an index of the entire 1,902 files, the entire Claude Code codebase (oops Anthropic!). Zero loss (unlike Anthropic's security). What this means is the ability to work better on local (private) code bases, off-the-shelf private document analysis with better response than the best public models - all locally.

John T Davies 🇪🇺

85,778 Aufrufe • vor 6 Monaten