正在加载视频...

视频加载失败

Jev v1.13 dodges rockets with probability calculation 🚀 TypeSafe AI's new non-LLM model returns decisions instead of text so we had it calculate a safe tile every 330 ms while rockets fell and it survived 25 of 26 for under a cent Run Jev via API ->

54,019 次观看 • 3 天前 •via X (Twitter)

4 条评论

Julian Pscheid 的头像
Julian Pscheid3 天前

@typesafeai I wish it was actually running local. Seems like other labs should be able to replicate this pretty quickly.

BasedAntiLeft 的头像
BasedAntiLeft3 天前

@typesafeai Jev just solved the nuclear threat

Daniel Yurkin 的头像
Daniel Yurkin3 天前

@typesafeai 🚀

Berelz 的头像
Berelz3 天前

@typesafeai All i see is new drone warfare capabilities

相关视频

Jev is HERE and this is the CLEAREST explanation of what it is and what NEW businesses it unlocks. (and at the end I'll tell you how to get Jev even if you're on the waitlist) WHAT IT IS You know how you open your inbox and have to decide what's junk, what needs a reply, and what can wait? Jev does that part. It looks at each thing and says "this is junk, I'm 94% sure." It doesn't write anything back to you. It just sorts. 1,700 emails for 18 cents, instantly. That sounds kinda trivial but the important part WHAT IT UNLOCKS My explanation of Jev sounds small until you realize HOW MANY jobs are exactly this. Someone reading a stack of applications. Someone deciding which support ticket goes to which team. Someone looking at inbound and deciding who's worth calling back. A few ideas on what it unlocks: 1/ Instant quotes that are actually instant. Every quote form on the internet says "we'll email you by end of day." Build the version that answers in under a second, for roofers, movers, insurance, legal intake. 2/ Lead scoring as a product. Every agency and service business has a contact form full of junk. Score every submission and send the real ones straight to the owner's phone. 3/ Support triage for companies with no support team. The ticket gets classified and routed before anyone opens it. 4/ Clipping tools. Pass in a transcript, get the best moments scored in three seconds. Every clipping product just got a cheaper engine. 5/ Application piles. Grants, permits, insurance claims, job apps, loan docs. Someone reads that stack one item at a time today. 6/ Marketplace matching. Someone types what they need and gets matched to the right local business instantly instead of waiting for callbacks. 7/ Browser agents that actually move FAST. That makes bulk browser work practical: pulling quotes from five carriers, filing the same form for 200 clients, checking supplier inventory in real time etc. TLDR; find an expensive queue and put Jev at the front of it. HOW TO GET IT I didn't realize you can skip the waitlist because Jev is live on the Vercel AI Gateway right now, so you can start calling it today. In this episode, we share how. Episode now live on The Startup Ideas Podcast (SIP) 🧃 (thanks to vogel for coming on and spilling the sauce today) Watch: Jev is a big deal because this is a whole new way to do AI Really cool Happy Jev day.

GREG ISENBERG

148,535 次观看 • 2 天前

Redis built a cache that cuts LLM costs by 90%! Production LLM apps do not receive completely new questions every time. A customer-support assistant might receive all three of these: - "Can I get a refund after buying the monthly plan?" - "Is the monthly subscription refundable?" - "Can I cancel the plan and get my money back?" The wording is different, but the underlying question and its answer remain the same. Yet LLM apps process every version as a new request. They assemble the prompt, send it to the model, and generate an answer that may have already been generated. Prefix caching reduces part of these repeated calls. When requests begin with the same system prompt or context, the model can reuse the KV states already computed for that shared prefix. But the request still hits the LLM. The new tokens must be processed, and the complete answer must still be decoded. So even with a prefix-cache hit, there's another generation call involved. To solve this, instead of only caching computation inside the model, the application can cache the generated response outside it. When another question arrives, the system embeds it and compares it with previously answered questions. If it finds a sufficiently close match, it returns the stored response without invoking the LLM again. A cache hit removes the input tokens, output tokens, and decoding time associated with another LLM call. In practice, it is important to decide which questions can safely share an answer since a production setup needs well-tuned similarity thresholds, expiration policies, data isolation, and monitoring for incorrect matches. If you want to use this in practice, Redis already implements it as a managed service called Redis LangCache. Under the hood, it generates embeddings, searches previous responses, and returns a matching answer before another model call occurs. Redis also handles access scopes, custom filtering, TTL and eviction controls, and cache monitoring through Redis Cloud. I built an interface to compare it against direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put this together. For the paraphrased question in my run, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. Redis returned the earlier response in 0.37 seconds with zero LLM input or output tokens. That was roughly 6x faster in this run. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster. The actual result depends on how much safe repetition exists in the workload. You can try Redis LangCache here: If you want to dive deeper, I have already written a detailed breakdown of KV, prefix, prompt, and semantic caching in the article quoted below. This demo builds on the final technique and shows it running in practice. Read it below.

Avi Chawla

165,288 次观看 • 10 天前