正在加载视频...

视频加载失败

In practice, this advantage is even more apparent than in benchmarks. For example: a task to move 151 Linear issues and verify each took 312 tool calls in the past. With direct calls, that's about 36 requests and 3.1M input tokens. Now, as a Script, it took 2 requests...

47,085 次观看 • 6 天前 •via X (Twitter)

7 条评论

Factory 的头像
Factory6 天前

The biggest change in Droid 1.0: programmatic tool calling is now the default. Instead of calling tools one at a time, Droid writes a program that is capable of running up to hundreds of tools per single model request.

Factory 的头像
Factory6 天前

On Terminal-Bench 4, Droid outperforms popular models’ native harnesses, with up to 20% better efficiency.

Factory 的头像
Factory6 天前

Introducing Droid 1.0: the new engine behind Factory’s agent. Built for complex projects and everyday development: >more accurate >more efficient >faster than ever before

Factory 的头像
Factory6 天前

We built Droid 1.0 on our core design principles and an internal Agent Factory that tests every change on every model family, which we outline in our blog. Read more about our methodology here:

gary IH fung 的头像
gary IH fung5 天前

how does Droid compare with Pi though?

SimonLab 的头像
SimonLab5 天前

LLMs already get the idea of writing a helper script for repetitive tasks. Great idea to embrace such a flow on a harness level; side effect: agents probably do more targeted mid-job reasoning bursts. Testing now.

climate alarmist 的头像
climate alarmist5 天前

Phenomenal

相关视频

Redis built a cache that cuts LLM costs by 90%! Production LLM apps do not receive completely new questions every time. A customer-support assistant might receive all three of these: - "Can I get a refund after buying the monthly plan?" - "Is the monthly subscription refundable?" - "Can I cancel the plan and get my money back?" The wording is different, but the underlying question and its answer remain the same. Yet LLM apps process every version as a new request. They assemble the prompt, send it to the model, and generate an answer that may have already been generated. Prefix caching reduces part of these repeated calls. When requests begin with the same system prompt or context, the model can reuse the KV states already computed for that shared prefix. But the request still hits the LLM. The new tokens must be processed, and the complete answer must still be decoded. So even with a prefix-cache hit, there's another generation call involved. To solve this, instead of only caching computation inside the model, the application can cache the generated response outside it. When another question arrives, the system embeds it and compares it with previously answered questions. If it finds a sufficiently close match, it returns the stored response without invoking the LLM again. A cache hit removes the input tokens, output tokens, and decoding time associated with another LLM call. In practice, it is important to decide which questions can safely share an answer since a production setup needs well-tuned similarity thresholds, expiration policies, data isolation, and monitoring for incorrect matches. If you want to use this in practice, Redis already implements it as a managed service called Redis LangCache. Under the hood, it generates embeddings, searches previous responses, and returns a matching answer before another model call occurs. Redis also handles access scopes, custom filtering, TTL and eviction controls, and cache monitoring through Redis Cloud. I built an interface to compare it against direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put this together. For the paraphrased question in my run, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. Redis returned the earlier response in 0.37 seconds with zero LLM input or output tokens. That was roughly 6x faster in this run. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster. The actual result depends on how much safe repetition exists in the workload. You can try Redis LangCache here: If you want to dive deeper, I have already written a detailed breakdown of KV, prefix, prompt, and semantic caching in the article quoted below. This demo builds on the final technique and shows it running in practice. Read it below.

Avi Chawla

166,230 次观看 • 1 个月前