Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing HermesAgent-20, a new Bench Pack for BenchLocal. 20 scenarios extracted straight from the Hermes Agent source code, run against a REAL Hermes instance. The actual workload you'd put your model through. Why I built BenchLocal in the first place: most benchmarks are too abstract. We use local LLMs...

stevibe's profile picture

stevibe

22,922 subscribers

38,631 Aufrufe • vor 2 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

I built a macOS app for benchmarking local LLMs. 6 test suites. Multiple providers. One workspace. Open source. There are hundreds of local models now. New ones every week. How do you actually pick one? Leaderboards test for general ability. But if you're building an agent that chains tool calls, or a pipeline that extracts structured data, or a code assistant that needs to debug Rust, you need to know if the model handles that specific thing. Not in theory. On your hardware. With your prompts. The benchmarks that exist are either locked behind papers, too abstract to map to real failures, or impossible to extend. You can't add your own test cases. You can't test what matters to your use case. That's what BenchLocal is for. It's a benchmark platform where every test is practical, deterministic, and built around real-world tasks. And you can build your own tests. It ships with 6 Bench Packs TODAY: → ToolCall-15 — tool-use accuracy → BugFind-15 — debugging capabilities → DataExtract-15 — structured data extraction → InstructFollow-15 — constraint-heavy instruction following → ReasonMath-15 — practical reasoning and math → StructOutput-15 — validator-backed structured output Every pack has 15 fixed scenarios. Every score is deterministic and verifiable. Some of you saw ToolCall-15 and BugFind-15 — the individual test packs I open-sourced over the past few weeks. People ran them, filed issues, sent PRs. But managing separate repos, separate scripts, separate results doesn't scale. BenchLocal puts everything in one place. What the app does: > Workspace with tabs — run BugFind-15 in one tab, ToolCall-15 in another. > Any provider — Ollama, llama.cpp, OpenRouter, any OpenAI-compatible endpoint. Local and cloud, same interface. > Run modes — serial, batch per model, batch per test case, or fully parallel. > Test histories — every run saved. Compare any previous session. But the part I'm most excited about isn't the app. It's the ecosystem. BenchLocal is a platform. Each Bench Pack is a plugin. I'm shipping an SDK so anyone can build their own — test what matters to you, package it, share it. Install and uninstall packs right inside the app, same way you'd manage extensions in VS Code. The registry is GitHub-based, fully public. I built 6 packs. I want the community to build the next 60. Theme system built in too — because if I'm staring at benchmark results for hours, it should at least look good. v0.1.0 is macOS only. Windows and Linux are coming. MIT licensed. Everything — the app, the bench packs, the SDK — is open. PRs welcome. Bench Packs even more welcome.
0:55

I built a macOS app for benchmarking local LLMs. 6 test suites. Multiple providers. One workspace. Open source. There are hundreds of local models now. New ones every week. How do you actually pick one? Leaderboards test for general ability. But if you're building an agent that chains tool calls, or a pipeline that extracts structured data, or a code assistant that needs to debug Rust, you need to know if the model handles that specific thing. Not in theory. On your hardware. With your prompts. The benchmarks that exist are either locked behind papers, too abstract to map to real failures, or impossible to extend. You can't add your own test cases. You can't test what matters to your use case. That's what BenchLocal is for. It's a benchmark platform where every test is practical, deterministic, and built around real-world tasks. And you can build your own tests. It ships with 6 Bench Packs TODAY: → ToolCall-15 — tool-use accuracy → BugFind-15 — debugging capabilities → DataExtract-15 — structured data extraction → InstructFollow-15 — constraint-heavy instruction following → ReasonMath-15 — practical reasoning and math → StructOutput-15 — validator-backed structured output Every pack has 15 fixed scenarios. Every score is deterministic and verifiable. Some of you saw ToolCall-15 and BugFind-15 — the individual test packs I open-sourced over the past few weeks. People ran them, filed issues, sent PRs. But managing separate repos, separate scripts, separate results doesn't scale. BenchLocal puts everything in one place. What the app does: > Workspace with tabs — run BugFind-15 in one tab, ToolCall-15 in another. > Any provider — Ollama, llama.cpp, OpenRouter, any OpenAI-compatible endpoint. Local and cloud, same interface. > Run modes — serial, batch per model, batch per test case, or fully parallel. > Test histories — every run saved. Compare any previous session. But the part I'm most excited about isn't the app. It's the ecosystem. BenchLocal is a platform. Each Bench Pack is a plugin. I'm shipping an SDK so anyone can build their own — test what matters to you, package it, share it. Install and uninstall packs right inside the app, same way you'd manage extensions in VS Code. The registry is GitHub-based, fully public. I built 6 packs. I want the community to build the next 60. Theme system built in too — because if I'm staring at benchmark results for hours, it should at least look good. v0.1.0 is macOS only. Windows and Linux are coming. MIT licensed. Everything — the app, the bench packs, the SDK — is open. PRs welcome. Bench Packs even more welcome.

stevibe

50,584 Aufrufe • vor 3 Monaten

We just launched Comfy MCP in public beta, the first MCP built for production pipelines. Connect Claude, Codex, Cursor, or Hermes to the entire ComfyUI ecosystem. → Run any workflow in natural language → Search models, nodes, and templates → Share a workflow URL - your agent picks it up → Re-run saved workflows on new inputs without touching a single node → Hundreds of pre-built workflows. Always auto-updated 20 product-placement images for your brand ads. Character design for your game art. Script-to-shot ideation at scale and so on! Your agent is now a creative technologist.
0:33

We just launched Comfy MCP in public beta, the first MCP built for production pipelines. Connect Claude, Codex, Cursor, or Hermes to the entire ComfyUI ecosystem. → Run any workflow in natural language → Search models, nodes, and templates → Share a workflow URL - your agent picks it up → Re-run saved workflows on new inputs without touching a single node → Hundreds of pre-built workflows. Always auto-updated 20 product-placement images for your brand ads. Character design for your game art. Script-to-shot ideation at scale and so on! Your agent is now a creative technologist.

ComfyUI

220,946 Aufrufe • vor 14 Tagen

I tested MiniMax M3 so you do not have to. It is free in OpenCode right now and it claims to beat GPT 5.5 on SWE-bench. So I put it on real production code. It broke my push-to-talk. It glitched a game through the floor. It took two tries to render a video and still looked rough. The benchmark said one thing. The real work said another. The one thing it has going for it is price. I ran all of BridgeBench for $4.09. If budget is your only constraint, it is the cheapest Chinese model out there. But cheap is not free when it breaks your codebase. No BridgeMind stamp of approval. Full breakdown in the video.
19:53

I tested MiniMax M3 so you do not have to. It is free in OpenCode right now and it claims to beat GPT 5.5 on SWE-bench. So I put it on real production code. It broke my push-to-talk. It glitched a game through the floor. It took two tries to render a video and still looked rough. The benchmark said one thing. The real work said another. The one thing it has going for it is price. I ran all of BridgeBench for $4.09. If budget is your only constraint, it is the cheapest Chinese model out there. But cheap is not free when it breaks your codebase. No BridgeMind stamp of approval. Full breakdown in the video.

BridgeMind

46,294 Aufrufe • vor 1 Monat

A lot of people are calling Hermes Agent the end of OpenClaw. BRUH! It's not... Nous Research trains actual models and they built an agent around that expertise. The local model routing is solid, but the part that matters for your business is that your conversations become fine-tuning data. You can train a model on how you actually work. 00:00 The Problem with Local AI Models 00:25 Introduction to Nous Research 01:04 Cross-Platform Agent Capabilities 01:44 Deep Local Model Integration 02:30 Routing Tasks to Different Models 03:06 Conversations as Training Data 03:50 Hermes Agent vs. OpenClaw 04:15 Future Plans and Series Overview
4:45

A lot of people are calling Hermes Agent the end of OpenClaw. BRUH! It's not... Nous Research trains actual models and they built an agent around that expertise. The local model routing is solid, but the part that matters for your business is that your conversations become fine-tuning data. You can train a model on how you actually work. 00:00 The Problem with Local AI Models 00:25 Introduction to Nous Research 01:04 Cross-Platform Agent Capabilities 01:44 Deep Local Model Integration 02:30 Routing Tasks to Different Models 03:06 Conversations as Training Data 03:50 Hermes Agent vs. OpenClaw 04:15 Future Plans and Series Overview

Ray Fernando

42,398 Aufrufe • vor 3 Monaten

HERMES AGENT CROSSED 140,000 GITHUB STARS IN 3 MONTHS AND JUST BECAME THE MOST USED AGENT IN THE WORLD. Most AI agents forget everything between sessions. Hermes writes its own skills from experience. Next time it runs the skill, improves it, and gets faster. Independent benchmarks show agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. Qwen 3.6 where the 35B version outperforms last year's 120B models at one third the memory footprint. DGX Spark with 128GB unified memory running everything locally at $0 per month after hardware. The setup takes 30 minutes. LM Studio plus Qwen 3.6 27B for the model server. One install script for Hermes. One config connecting them. Set context window to 65,536 tokens or nothing works. After one month of daily use your skills directory has 20 to 50 learned workflows. Your Hermes is genuinely different from anyone else's.
0:25

HERMES AGENT CROSSED 140,000 GITHUB STARS IN 3 MONTHS AND JUST BECAME THE MOST USED AGENT IN THE WORLD. Most AI agents forget everything between sessions. Hermes writes its own skills from experience. Next time it runs the skill, improves it, and gets faster. Independent benchmarks show agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. Qwen 3.6 where the 35B version outperforms last year's 120B models at one third the memory footprint. DGX Spark with 128GB unified memory running everything locally at $0 per month after hardware. The setup takes 30 minutes. LM Studio plus Qwen 3.6 27B for the model server. One install script for Hermes. One config connecting them. Set context window to 65,536 tokens or nothing works. After one month of daily use your skills directory has 20 to 50 learned workflows. Your Hermes is genuinely different from anyone else's.

Lummox

69,484 Aufrufe • vor 1 Monat

A CHINESE LAB VISUALIZED 2,000 NEURONS AND 1,191,000 SYNAPSES IN 3D. HERMES AGENT HIT 140,000 STARS RUNNING THE SAME NETWORK LOCALLY FOR $0 hermes is the first agent that writes its own skills from experience. complete a task once and it saves the procedure as a markdown file for next time agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. less time and less tokens to get the same result qwen 3.6 35b outperforms last year's 120b models and runs on 20gb of memory. the intelligence that needed a data center now fits on your desk setup takes 30 minutes. install lm studio, pull qwen 3.6, install hermes, point it at localhost. zero api fees, zero data leaving your machine most people pay for cloud agents that forget everything between sessions. the ones running hermes locally in 2026 will look very far ahead in 2028 bookmark this and read the article below
1:37

A CHINESE LAB VISUALIZED 2,000 NEURONS AND 1,191,000 SYNAPSES IN 3D. HERMES AGENT HIT 140,000 STARS RUNNING THE SAME NETWORK LOCALLY FOR $0 hermes is the first agent that writes its own skills from experience. complete a task once and it saves the procedure as a markdown file for next time agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. less time and less tokens to get the same result qwen 3.6 35b outperforms last year's 120b models and runs on 20gb of memory. the intelligence that needed a data center now fits on your desk setup takes 30 minutes. install lm studio, pull qwen 3.6, install hermes, point it at localhost. zero api fees, zero data leaving your machine most people pay for cloud agents that forget everything between sessions. the ones running hermes locally in 2026 will look very far ahead in 2028 bookmark this and read the article below

starmex

13,420 Aufrufe • vor 1 Monat

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2
11:32

i tested a fully local ai coding setup: gemma4 from Google DeepMind running through LM Studio, paired with pi which is a minimal open source coding harness this is part 1 of a two part series. here we cover the why and the complete hands-on setup 0:00 - Running a coding agent fully local 0:47 - Why Pi as the coding agent harness 1:39 - Why Gemma 4 and which variant to pick 2:44 - Architecture: Pi, LM Studio and Gemma 4 3:45 - LM Studio setup and model download 5:01 - Starting the local server 5:19 - API sanity check with curl 6:08 - Installing the Pi coding agent 6:20 - Configuring models.json for local models 8:20 - First run: selecting the model in Pi 8:58 - Fixing the context length error 10:16 - Testing the working setup 10:50 - What is coming in Part 2

Nidhi Singh

16,289 Aufrufe • vor 1 Monat

look anon, those of you who kept saying local AI is not there yet, who said open source can't compete, who said you need cloud APIs to get anything serious done, look at this gameplay for one minute. every pixel on this screen was written by one model, in one shot, on a single rtx 3090 with 24gb of vram. the model is qwen 3.6 27b dense q4. the harness is hermes agent. the hardware is a single consumer card you can buy used for 900 dollars. the prompt is open source on github. every claim verifiable, on your own desk. if your local AI take is from 2024, update it. the consumer tier is shipping work that was supposed to need 8 gpus and an api key. open source moved the floor while the rest of the field was busy explaining why it cannot. 24gb tier owners are eating ramen with half boiled egg and double chocolate.
1:39

look anon, those of you who kept saying local AI is not there yet, who said open source can't compete, who said you need cloud APIs to get anything serious done, look at this gameplay for one minute. every pixel on this screen was written by one model, in one shot, on a single rtx 3090 with 24gb of vram. the model is qwen 3.6 27b dense q4. the harness is hermes agent. the hardware is a single consumer card you can buy used for 900 dollars. the prompt is open source on github. every claim verifiable, on your own desk. if your local AI take is from 2024, update it. the consumer tier is shipping work that was supposed to need 8 gpus and an api key. open source moved the floor while the rest of the field was busy explaining why it cannot. 24gb tier owners are eating ramen with half boiled egg and double chocolate.

Sudo su

29,584 Aufrufe • vor 2 Monaten

There's a new AI-native IDE out there named Aide, and there are four things I like about it: 1. It's open-source 2. It's proactive and suggests changes before you ask 3. It uses an intermediate layer between the editor and LLM 4. It's SOTA on the SWE-Bench benchmark I'm excited about #3 because this layer (called "Sidecar") will help people connect any model to the IDE. While this isn't possible right now, it's clearly the goal. Sidecar is also open-source. As I write this, the code agent behind Aide, "CodeStory Midwit Agent," is second in the Verified SWE-Bench leaderboard (it finished in 2024 in the first place.) This is a huge deal! If I'm gonna have an agent help me code, it might as well be the best of the best. The proactive aspect is also cool: the IDE provides suggestions to improve your code without you having to ask. I recorded a quick video showing you how Aide works. The team behind the IDE partnered with me on this video. Right now, it feels like *everyone* is building AI-powered coding tools. As a career developer, nothing excites me more than seeing how quickly we transform the development experience. And 2025 is just getting started!
5:32

There's a new AI-native IDE out there named Aide, and there are four things I like about it: 1. It's open-source 2. It's proactive and suggests changes before you ask 3. It uses an intermediate layer between the editor and LLM 4. It's SOTA on the SWE-Bench benchmark I'm excited about #3 because this layer (called "Sidecar") will help people connect any model to the IDE. While this isn't possible right now, it's clearly the goal. Sidecar is also open-source. As I write this, the code agent behind Aide, "CodeStory Midwit Agent," is second in the Verified SWE-Bench leaderboard (it finished in 2024 in the first place.) This is a huge deal! If I'm gonna have an agent help me code, it might as well be the best of the best. The proactive aspect is also cool: the IDE provides suggestions to improve your code without you having to ask. I recorded a quick video showing you how Aide works. The team behind the IDE partnered with me on this video. Right now, it feels like *everyone* is building AI-powered coding tools. As a career developer, nothing excites me more than seeing how quickly we transform the development experience. And 2025 is just getting started!

Santiago

69,422 Aufrufe • vor 1 Jahr

I am stocked to announce that I won the OpenAI Developers Codex x Mollie Hacka Worldwide Hackathon in Paris. 60+ builders, every one of us working solo, one day to ship. I built mine around a single question: who gets to own intelligence? The default answer is scary. You hand your data to a handful of labs, they train the model, they own it, and you rent back a thin slice of what your own data made possible. That is the bargain on the table today. I do not accept it. So I built Lensemble: a Tapestry like distributed training platform for JEPA based World Models. What does it enable: World Models that a community improves together, keeps sovereign, and co-owns. Two bets sit underneath it. First, the paradigm. Language models predict the next token. Powerful for text, a dead end for the physical world. A robot does not need to autocomplete sentences, it needs to predict what happens next in the world. That is what JEPA does: it learns by predicting representations instead of pixels or tokens. I am convinced world models are the most underrated paradigm in AI right now, and the closest thing we have to a ChatGPT moment for robotics. Second, the politics. Your raw trajectories never leave your machine. Each participant trains locally against a shared protocol and ships only an update, never the data. A federated round folds those updates into one shared world model, a LeWorldModel based model, and the gain is measured, not claimed: a 12k-parameter adapter on a frozen backbone, held-out prediction error down about 12 percent, the model measurably less surprised by the world. Then the upside is split by contribution weight, so the people who improved the model own a share of what it earns. This is the thesis behind Project Tapestry, the AI Alliance and Yann LeCun's push for federated, sovereign frontier AI, carried into world models and robotics. Call it Tapestry for the physical world. All of it built solo, in a single day, with Codex as my pair the whole way. Thank you to OpenAI Codex and Mollie for backing builders who ship real things, and to Boris and the organizing crew for the room and the standard you set. Intelligence the world improves, and the world owns. That is the future I want for my kids, and the one I will keep building.
1:13

I am stocked to announce that I won the OpenAI Developers Codex x Mollie Hacka Worldwide Hackathon in Paris. 60+ builders, every one of us working solo, one day to ship. I built mine around a single question: who gets to own intelligence? The default answer is scary. You hand your data to a handful of labs, they train the model, they own it, and you rent back a thin slice of what your own data made possible. That is the bargain on the table today. I do not accept it. So I built Lensemble: a Tapestry like distributed training platform for JEPA based World Models. What does it enable: World Models that a community improves together, keeps sovereign, and co-owns. Two bets sit underneath it. First, the paradigm. Language models predict the next token. Powerful for text, a dead end for the physical world. A robot does not need to autocomplete sentences, it needs to predict what happens next in the world. That is what JEPA does: it learns by predicting representations instead of pixels or tokens. I am convinced world models are the most underrated paradigm in AI right now, and the closest thing we have to a ChatGPT moment for robotics. Second, the politics. Your raw trajectories never leave your machine. Each participant trains locally against a shared protocol and ships only an update, never the data. A federated round folds those updates into one shared world model, a LeWorldModel based model, and the gain is measured, not claimed: a 12k-parameter adapter on a frozen backbone, held-out prediction error down about 12 percent, the model measurably less surprised by the world. Then the upside is split by contribution weight, so the people who improved the model own a share of what it earns. This is the thesis behind Project Tapestry, the AI Alliance and Yann LeCun's push for federated, sovereign frontier AI, carried into world models and robotics. Call it Tapestry for the physical world. All of it built solo, in a single day, with Codex as my pair the whole way. Thank you to OpenAI Codex and Mollie for backing builders who ship real things, and to Boris and the organizing crew for the room and the standard you set. Intelligence the world improves, and the world owns. That is the future I want for my kids, and the one I will keep building.

abdel

17,134 Aufrufe • vor 22 Tagen

Hermes agent just left the terminal. 𝗛𝗲𝗿𝗺𝗲𝘀 𝗗𝗲𝘀𝗸𝘁𝗼𝗽 dropped yesterday. native app for macOS, Windows, and Linux. for months Hermes was the agent that learned your projects, wrote its own skills, and built a model of who you are. all of it buried in terminal logs. now it has a window. the important part is that it's not a wrapper. it runs the same agent core, the same sessions, memory, and skills as the CLI. you can start a task in the terminal and finish it in the app without anything resetting. the state is shared across every interface, not copied between them. what the GUI actually adds: → streaming chat that shows live tool calls and inline reasoning instead of a spinner → a preview rail that renders pages, code, and images right beside the conversation → an artifacts panel that collects every file the agent has ever produced → remote gateway mode, so you can point the app at a VPS and run the heavy work elsewhere → skills, cron, profiles, and gateways managed point-and-click instead of through YAML → voice mode, drag-drop files, and inline image generation remote gateway mode is the one worth slowing down on. the agent runs 24/7 on a $5 server while you control it from your laptop like a local app. other agent UIs are chatboxes with a logo. this one shows the autonomy instead of hiding it, so you watch the skills load, the tools fire, and the artifacts pile up as it works. it was teased in Jensen's GTC keynote. MIT licensed, local-first, no telemetry. if you already run Hermes, download it and everything is already there. your chats, memory, and skills carry straight over. i wrote a full masterclass on Hermes Agent that walks through the SOUL. md identity layer, the three-tier memory system, the self-evolving skills loop, and how to run three specialized agents 24/7. desktop is the interface that finally does all of it justice. the article is quoted below.
1:18

Hermes agent just left the terminal. 𝗛𝗲𝗿𝗺𝗲𝘀 𝗗𝗲𝘀𝗸𝘁𝗼𝗽 dropped yesterday. native app for macOS, Windows, and Linux. for months Hermes was the agent that learned your projects, wrote its own skills, and built a model of who you are. all of it buried in terminal logs. now it has a window. the important part is that it's not a wrapper. it runs the same agent core, the same sessions, memory, and skills as the CLI. you can start a task in the terminal and finish it in the app without anything resetting. the state is shared across every interface, not copied between them. what the GUI actually adds: → streaming chat that shows live tool calls and inline reasoning instead of a spinner → a preview rail that renders pages, code, and images right beside the conversation → an artifacts panel that collects every file the agent has ever produced → remote gateway mode, so you can point the app at a VPS and run the heavy work elsewhere → skills, cron, profiles, and gateways managed point-and-click instead of through YAML → voice mode, drag-drop files, and inline image generation remote gateway mode is the one worth slowing down on. the agent runs 24/7 on a $5 server while you control it from your laptop like a local app. other agent UIs are chatboxes with a logo. this one shows the autonomy instead of hiding it, so you watch the skills load, the tools fire, and the artifacts pile up as it works. it was teased in Jensen's GTC keynote. MIT licensed, local-first, no telemetry. if you already run Hermes, download it and everything is already there. your chats, memory, and skills carry straight over. i wrote a full masterclass on Hermes Agent that walks through the SOUL. md identity layer, the three-tier memory system, the self-evolving skills loop, and how to run three specialized agents 24/7. desktop is the interface that finally does all of it justice. the article is quoted below.

Akshay 🚀

51,091 Aufrufe • vor 1 Monat

Your coding agent can run all night. It still can't tell if what it built actually works. Today we're open-sourcing the TestSprite CLl (Apache-2.0) A tool your agent calls on its own to test your app end-to-end like a real user, fix what broke, and re-check everything it ever got right. It's the same engine 100,000+ teams already use. We proved it in public, on a public leaderboard: Most correct app on the board:89% Built by the cheapest model in the field At half the cost of the priciest one You no longer need the biggest, most expensive model to ship software you can trust. Setup is 2 commands: npm install -g @testsprite/testsprite-cli testsprite init That's the last command you'll ever type - from there, your agent runs the tests itself.
3:44

Your coding agent can run all night. It still can't tell if what it built actually works. Today we're open-sourcing the TestSprite CLl (Apache-2.0) A tool your agent calls on its own to test your app end-to-end like a real user, fix what broke, and re-check everything it ever got right. It's the same engine 100,000+ teams already use. We proved it in public, on a public leaderboard: Most correct app on the board:89% Built by the cheapest model in the field At half the cost of the priciest one You no longer need the biggest, most expensive model to ship software you can trust. Setup is 2 commands: npm install -g @testsprite/testsprite-cli testsprite init That's the last command you'll ever type - from there, your agent runs the tests itself.

TestSprite

1,038,682 Aufrufe • vor 1 Monat

For the first time, the latest LLMs run on the Apple Neural Engine — and NexaSDK is the only framework that makes it possible, powered by the NexaML engine. Last year, our two co-founders were invited by Apple DMLI team (Data & Machine Learning Innovation) to share their research about on-device multimodal model for local AI agents. One of the big questions in the room was: “Can the newest LLMs actually run on ANE?” At the time, nobody had a clear path. Today, that path exists. NexaSDK now runs Granite-4.0 (IBM), Qwen3 (Qwen), Gemma3 (Google), and Parakeet-v3 (NVIDIA) fully on Apple’s NPU — unlocking low-power, always-on, fast inference across Mac and iPhone. A new wave of NPU-first local AI apps is coming to Apple devices. Start with one line of code on Mac. iOS SDK coming soon.
1:42

For the first time, the latest LLMs run on the Apple Neural Engine — and NexaSDK is the only framework that makes it possible, powered by the NexaML engine. Last year, our two co-founders were invited by Apple DMLI team (Data & Machine Learning Innovation) to share their research about on-device multimodal model for local AI agents. One of the big questions in the room was: “Can the newest LLMs actually run on ANE?” At the time, nobody had a clear path. Today, that path exists. NexaSDK now runs Granite-4.0 (IBM), Qwen3 (Qwen), Gemma3 (Google), and Parakeet-v3 (NVIDIA) fully on Apple’s NPU — unlocking low-power, always-on, fast inference across Mac and iPhone. A new wave of NPU-first local AI apps is coming to Apple devices. Start with one line of code on Mac. iOS SDK coming soon.

NEXA AI

30,213 Aufrufe • vor 7 Monaten

Introducing SuperDesign.Dev The first open-source Design Agent that lives inside Cursor, Windsurf, or any IDE: 👉 Instantly spin up 10 different design in parallel 👉 Explore, fork, and iterate 10x faster 👉 Create new or iterate on your existing UI — all fully local JackJack and I shipped this as a quick weekend experiment to validate a hunch: design is the killer use case for parallel agents. Currently at V0.005 — a simple customization of the Claude Code SDK + static HTML generation. (We might be the first product powered by the Anthropic Claude Code SDK!) More coming soon: web app, mobile app, SVG generation, and more. GitHub: Try it out: ----- Upvote on Hacker News ❤️:
0:55

Introducing SuperDesign.Dev The first open-source Design Agent that lives inside Cursor, Windsurf, or any IDE: 👉 Instantly spin up 10 different design in parallel 👉 Explore, fork, and iterate 10x faster 👉 Create new or iterate on your existing UI — all fully local JackJack and I shipped this as a quick weekend experiment to validate a hunch: design is the killer use case for parallel agents. Currently at V0.005 — a simple customization of the Claude Code SDK + static HTML generation. (We might be the first product powered by the Anthropic Claude Code SDK!) More coming soon: web app, mobile app, SVG generation, and more. GitHub: Try it out: ----- Upvote on Hacker News ❤️:

Jason Zhou

49,194 Aufrufe • vor 1 Jahr

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.
1:46

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

Sudo su

27,339 Aufrufe • vor 1 Monat

Terminal-Bench 2.0 went from ~25% → 80% in four months and became the standard eval for frontier CLI agents. Now, TB3 is in the works. I talked to Alex Shaw about what happens when model capabilities climb faster than we can measure them. His answer: the benchmark factory (Harbor Framework)— infrastructure to develop hard, representative evals at the pace that the frontier moves. As Alex put it: "we need a thousand times more benchmarks than we have right now." 00:23 - How quickly models hill-climbed TB2 01:46 - What rapid progress reveals about benchmarks vs. real-world capability 03:28 - What made Terminal-Bench stick 04:58 - Why the terminal is the right abstraction for agentic AI 07:14 - How TB2 maintains task quality at scale 09:23 - Managing benchmark integrity in a benchmaxxing world 10:47 - Harbor: from experiment to benchmark factory 12:19 - What Harbor does that nothing else did 14:37 - The invariants: what won't change as agent evals evolve 16:55 - The benchmark Alex most wants to see built 18:18 - The ideal human-in-the-loop task creation flywheel 20:32 - How to contribute to Terminal-Bench 3.0
22:17

Terminal-Bench 2.0 went from ~25% → 80% in four months and became the standard eval for frontier CLI agents. Now, TB3 is in the works. I talked to Alex Shaw about what happens when model capabilities climb faster than we can measure them. His answer: the benchmark factory (Harbor Framework)— infrastructure to develop hard, representative evals at the pace that the frontier moves. As Alex put it: "we need a thousand times more benchmarks than we have right now." 00:23 - How quickly models hill-climbed TB2 01:46 - What rapid progress reveals about benchmarks vs. real-world capability 03:28 - What made Terminal-Bench stick 04:58 - Why the terminal is the right abstraction for agentic AI 07:14 - How TB2 maintains task quality at scale 09:23 - Managing benchmark integrity in a benchmaxxing world 10:47 - Harbor: from experiment to benchmark factory 12:19 - What Harbor does that nothing else did 14:37 - The invariants: what won't change as agent evals evolve 16:55 - The benchmark Alex most wants to see built 18:18 - The ideal human-in-the-loop task creation flywheel 20:32 - How to contribute to Terminal-Bench 3.0

vincent sunn chen

11,547 Aufrufe • vor 3 Monaten

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗
0:26

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗

Visual Studio

11,873 Aufrufe • vor 10 Monaten

I built an agent that answers machine-learning questions. It's autonomous, and the best part is that I built the whole thing without writing a single line of Python code. Here is what I did and how I did it: Over a year ago, a friend and I built a site that publishes multi-choice questions. You get a new one every day. I decided to have GPT-3.5 answer questions. Here is what I needed to build: 1. Connect to the site's API to retrieve today's question 2. Extract the question and the potential choices 3. Connect to OpenAI's API and ask GPT-3.5 to answer the question 4. Parse the answer from the model 5. Submit the answer back to the API to get the score Not difficult. Likely several hours of work. But I didn't have to write any code. I built the whole thing by dragging and dropping components using Vellum is a YC-backed platform for developers to build LLM applications. They are the only ones I've seen offering this functionality. They sponsored this post, and their team helped me with all my questions while I built this. I created a workflow. The platform supports several node types to build whatever you have in mind. I show how I put the whole thing together in the attached video. The only code I had to write was a few lines of Jinja to parse and transform the API and the LLM results. There are three lessons I want to share from this experience: First, the best possible code is the one you didn't write. I'm a big fan of no-code tools because they help me materialize my ideas fast. They help product people, designers, and no coders collaborate on the solution. Second, Large Language Models are sensitive to how you prompt them. Small changes to prompts can make a big difference in results. This is more pronounced when you are building a multi-step workflow. Third, automated testing and evaluation for prompts is critical. There aren't many companies thinking about this. They'll have a hard time moving from a demo phase. The attached video will show you what I did.
3:46

I built an agent that answers machine-learning questions. It's autonomous, and the best part is that I built the whole thing without writing a single line of Python code. Here is what I did and how I did it: Over a year ago, a friend and I built a site that publishes multi-choice questions. You get a new one every day. I decided to have GPT-3.5 answer questions. Here is what I needed to build: 1. Connect to the site's API to retrieve today's question 2. Extract the question and the potential choices 3. Connect to OpenAI's API and ask GPT-3.5 to answer the question 4. Parse the answer from the model 5. Submit the answer back to the API to get the score Not difficult. Likely several hours of work. But I didn't have to write any code. I built the whole thing by dragging and dropping components using Vellum is a YC-backed platform for developers to build LLM applications. They are the only ones I've seen offering this functionality. They sponsored this post, and their team helped me with all my questions while I built this. I created a workflow. The platform supports several node types to build whatever you have in mind. I show how I put the whole thing together in the attached video. The only code I had to write was a few lines of Jinja to parse and transform the API and the LLM results. There are three lessons I want to share from this experience: First, the best possible code is the one you didn't write. I'm a big fan of no-code tools because they help me materialize my ideas fast. They help product people, designers, and no coders collaborate on the solution. Second, Large Language Models are sensitive to how you prompt them. Small changes to prompts can make a big difference in results. This is more pronounced when you are building a multi-step workflow. Third, automated testing and evaluation for prompts is critical. There aren't many companies thinking about this. They'll have a hard time moving from a demo phase. The attached video will show you what I did.

Santiago

309,825 Aufrufe • vor 2 Jahren

One week ago, we launched Typeahead. The product is already meaningfully better. We’ve shipped new features, Typeahead 2.0 is almost ready, and we’re already planning 3.0. 3.0 is going to be big. That pace is part of the point. Sam Asante and I started a new company together because we wanted to see how far we could push local AI software. Typeahead is the first thing we shipped. It is a local AI writing app for Mac. You type, suggestions appear inline, and it learns how you actually write. It works offline. You pay once. $79 and you own it. The product is intentionally simple. The thesis behind it is bigger. We think local models are going to create a new class of software. Fast, private and offline. By default. Personal without being creepy. Useful without turning everything into a subscription. Most AI products today assume the model lives in the cloud. That will not be the only path. The machines we already own are getting powerful enough. The models are getting small enough. And the experience can start to feel less like chatting with a remote service and more like using software that belongs on your computer. We have already built a few fully local experiments together. Typeahead felt right because the value is immediate. A few people have asked me how this fits with Crazy Egg. Crazy Egg is still my main work. This is a focused company with Sam, built around a thesis we both believe in. I’ve learned that the people I work with shape the work more than almost anything else. Sam was already on my short list of people I wanted to build with. Typeahead is the first public proof of the thesis. Watch the video. Get it here:
1:00

One week ago, we launched Typeahead. The product is already meaningfully better. We’ve shipped new features, Typeahead 2.0 is almost ready, and we’re already planning 3.0. 3.0 is going to be big. That pace is part of the point. Sam Asante and I started a new company together because we wanted to see how far we could push local AI software. Typeahead is the first thing we shipped. It is a local AI writing app for Mac. You type, suggestions appear inline, and it learns how you actually write. It works offline. You pay once. $79 and you own it. The product is intentionally simple. The thesis behind it is bigger. We think local models are going to create a new class of software. Fast, private and offline. By default. Personal without being creepy. Useful without turning everything into a subscription. Most AI products today assume the model lives in the cloud. That will not be the only path. The machines we already own are getting powerful enough. The models are getting small enough. And the experience can start to feel less like chatting with a remote service and more like using software that belongs on your computer. We have already built a few fully local experiments together. Typeahead felt right because the value is immediate. A few people have asked me how this fits with Crazy Egg. Crazy Egg is still my main work. This is a focused company with Sam, built around a thesis we both believe in. I’ve learned that the people I work with shape the work more than almost anything else. Sam was already on my short list of people I wanted to build with. Typeahead is the first public proof of the thesis. Watch the video. Get it here:

Hiten Shah

18,566 Aufrufe • vor 1 Monat