Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

In practice, this advantage is even more apparent than in benchmarks. For example: a task to move 151 Linear issues and verify each took 312 tool calls in the past. With direct calls, that's about 36 requests and 3.1M input tokens. Now, as a Script, it took 2 requests...

47,085 Aufrufe • vor 6 Tagen •via X (Twitter)

7 Kommentare

Profilbild von Factory
Factoryvor 6 Tagen

The biggest change in Droid 1.0: programmatic tool calling is now the default. Instead of calling tools one at a time, Droid writes a program that is capable of running up to hundreds of tools per single model request.

Profilbild von Factory
Factoryvor 6 Tagen

On Terminal-Bench 4, Droid outperforms popular models’ native harnesses, with up to 20% better efficiency.

Profilbild von Factory
Factoryvor 6 Tagen

Introducing Droid 1.0: the new engine behind Factory’s agent. Built for complex projects and everyday development: >more accurate >more efficient >faster than ever before

Profilbild von Factory
Factoryvor 6 Tagen

We built Droid 1.0 on our core design principles and an internal Agent Factory that tests every change on every model family, which we outline in our blog. Read more about our methodology here:

Profilbild von gary IH fung
gary IH fungvor 6 Tagen

how does Droid compare with Pi though?

Profilbild von SimonLab
SimonLabvor 6 Tagen

LLMs already get the idea of writing a helper script for repetitive tasks. Great idea to embrace such a flow on a harness level; side effect: agents probably do more targeted mid-job reasoning bursts. Testing now.

Profilbild von climate alarmist
climate alarmistvor 6 Tagen

Phenomenal

Ähnliche Videos

Redis built a cache that cuts LLM costs by 90%! Production LLM apps do not receive completely new questions every time. A customer-support assistant might receive all three of these: - "Can I get a refund after buying the monthly plan?" - "Is the monthly subscription refundable?" - "Can I cancel the plan and get my money back?" The wording is different, but the underlying question and its answer remain the same. Yet LLM apps process every version as a new request. They assemble the prompt, send it to the model, and generate an answer that may have already been generated. Prefix caching reduces part of these repeated calls. When requests begin with the same system prompt or context, the model can reuse the KV states already computed for that shared prefix. But the request still hits the LLM. The new tokens must be processed, and the complete answer must still be decoded. So even with a prefix-cache hit, there's another generation call involved. To solve this, instead of only caching computation inside the model, the application can cache the generated response outside it. When another question arrives, the system embeds it and compares it with previously answered questions. If it finds a sufficiently close match, it returns the stored response without invoking the LLM again. A cache hit removes the input tokens, output tokens, and decoding time associated with another LLM call. In practice, it is important to decide which questions can safely share an answer since a production setup needs well-tuned similarity thresholds, expiration policies, data isolation, and monitoring for incorrect matches. If you want to use this in practice, Redis already implements it as a managed service called Redis LangCache. Under the hood, it generates embeddings, searches previous responses, and returns a matching answer before another model call occurs. Redis also handles access scopes, custom filtering, TTL and eviction controls, and cache monitoring through Redis Cloud. I built an interface to compare it against direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put this together. For the paraphrased question in my run, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. Redis returned the earlier response in 0.37 seconds with zero LLM input or output tokens. That was roughly 6x faster in this run. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster. The actual result depends on how much safe repetition exists in the workload. You can try Redis LangCache here: If you want to dive deeper, I have already written a detailed breakdown of KV, prefix, prompt, and semantic caching in the article quoted below. This demo builds on the final technique and shows it running in practice. Read it below.

Avi Chawla

166,230 Aufrufe • vor 1 Monat