Loading video...

Video Failed to Load

Go Home

Introducing Batchwork ⚡️ Unified batch API for AI providers. Process LLM requests in bulk with a single call for lower costs. Processing, uploading, polling, and result parsing handled for you. → 7 providers → JSONL handled → ▲ AI SDK support → Server + Next.js helpers

78,899 views • 2 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Anthropic won't like this open-source repo. It is going to cost LLM providers a lot of money. Every CI run of an AI app today sends real requests to providers like OpenAI or Anthropic. Like any other LLM call, this too gets billed at actual API rates. So for teams with high commit volumes, this accumulates into a meaningful chunk of API spend. One common hack devs use is that instead of invoking the LLM API, the test calls a fake local server that speaks the same API and returns a dummy response. The catch is that the dummy response is a copy of what the provider returned on the day it was saved, and providers keep adding fields and changing types. So the tests keep passing against a schema that's no longer valid, while the real integration breaks in production. A smart approach is now actually implemented in CopilotKit🪁's recently open-sourced aimock project. Every day, the repo's own CI sends a handful of requests to the real API and the same requests to the fake server, then compares both against the official client library's type definitions. Those are the only real API calls in the whole setup, and they run on the repo's own keys, not in anyone else's CI. A single team can push hundreds of commits a day, and thousands of teams are already doing that with coding agents. All of those runs stay offline, because one repo checks against the real API on everyone's behalf. When a check fails, a coding agent updates aimock's built-in response schema, the full test suite has to pass, and a patch version ships to npm. By simply upgrading the package, the corrected schema gets reflected in every project using it. The capability is not just limited to a single provider. The same server works for Claude, OpenAI, Gemini, Bedrock, Azure, Ollama, plus MCP tools, A2A agents, AG-UI event streams, vector DBs like Pinecone and Qdrant, and search, speech, image, and video endpoints. Here's the repo: (don't forget to star it ⭐) That said, mocking your API calls is one thing. AI engineers should also know how to test agents properly in the first place, which several teams still skip. I wrote a full walkthrough on that, covering build, testing, evals, tracing, and deployment. Read it below.

Akshay 🚀

57,828 views • 1 day ago