Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

With Intelligent Model Routing, Replit picks the model for every task you send, so you can operate with quality, speed, and efficiency, at no extra cost. Here’s how it works, plus a breakdown on the difference between Free, Power and Max ↓

Replit ⠕

238,854 subscribers

11,941 görüntüleme • 1 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

OpenRouter

1,330,859 görüntüleme • 19 gün önce

Introducing Smart Routing in Unity AI Gateway. Stop overpaying for AI. Smart Routing matches each coding task to the right model and harness based on what the task needs, so higher-cost models can differentiate on intelligence while lower-cost models differentiate on cost and performance. Match frontier quality and cut task costs by 30%+.

Introducing Smart Routing in Unity AI Gateway. Stop overpaying for AI. Smart Routing matches each coding task to the right model and harness based on what the task needs, so higher-cost models can differentiate on intelligence while lower-cost models differentiate on cost and performance. Match frontier quality and cut task costs by 30%+.

Databricks

126,674 görüntüleme • 2 ay önce

not every task needs a frontier model! we built a model router into our coding agent's harness and cut costs by 64% w/o reducing quality. this vid walks through how it works, and how you can build one into your own harness!

not every task needs a frontier model! we built a model router into our coding agent's harness and cut costs by 64% w/o reducing quality. this vid walks through how it works, and how you can build one into your own harness!

Sydney Runkle

12,620 görüntüleme • 8 gün önce

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Santiago

15,543 görüntüleme • 1 ay önce

I was skeptical of GitHub Copilot auto mode...Then Nhu explained how the routing actually works and I can't go back. Turns out a system that picks the right model every time doesn't just save cost, it outperforms using one model alone. give auto another shot 👇🏽

I was skeptical of GitHub Copilot auto mode...Then Nhu explained how the routing actually works and I can't go back. Turns out a system that picks the right model every time doesn't just save cost, it outperforms using one model alone. give auto another shot 👇🏽

Andrea

13,229 görüntüleme • 4 ay önce

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.

Sub-agent Model Selection — Different Tasks, Different Models Your main agent runs Qwen3.6-Plus for quality. But not every subtask needs a flagship model. Now sub-agents can use a different model. Create a skill file with model: openai:qwen3.5-plus and the sub-agent runs on that model. Powerful model for the hard parts, fast model for the easy parts. Save tokens without sacrificing quality on what matters.

Qwen

21,333 görüntüleme • 6 ay önce

Factory benchmarks let you build your own model bench using your past coding agent runs. - Mirrors your environment, secrets, and MCPs for each run - Scores output on judging criteria you define - Generates a report with model recs using cost vs. quality Here's how it works:

Factory benchmarks let you build your own model bench using your past coding agent runs. - Mirrors your environment, secrets, and MCPs for each run - Scores output on judging criteria you define - Generates a report with model recs using cost vs. quality Here's how it works:

Warp

72,768 görüntüleme • 1 ay önce

Now you can use Gemma directly in the Gemini CLI! 🚀 v0.40.0 introduces experimental support for local Gemma models, starting with intelligent model routing (with full local execution on the roadmap!).

Now you can use Gemma directly in the Gemini CLI! 🚀 v0.40.0 introduces experimental support for local Gemma models, starting with intelligent model routing (with full local execution on the roadmap!).

Google Gemma

89,093 görüntüleme • 5 ay önce

The right model depends on the task. NVIDIA NeMo Switchyard helps developers route each agent workflow step across a chosen model pool based on their own quality, latency and cost criteria. Kari Briski joins MTS to explain why agent workflows need model routing.

The right model depends on the task. NVIDIA NeMo Switchyard helps developers route each agent workflow step across a chosen model pool based on their own quality, latency and cost criteria. Kari Briski joins MTS to explain why agent workflows need model routing.

NVIDIA

97,218 görüntüleme • 1 ay önce

Meta Tribe2 is a model that predicts your brain's response on videos so now you can edit videos with more clarity of how to compose them, what works and what doesn't and we built a free tool for you based on this model yes, totally free see details below

Meta Tribe2 is a model that predicts your brain's response on videos so now you can edit videos with more clarity of how to compose them, what works and what doesn't and we built a free tool for you based on this model yes, totally free see details below

AmirMušić

853,555 görüntüleme • 5 ay önce

MAI-Voice-1 exports flawless audio in record speed. Can you tell the difference between the real recording and our model? Drop your best guess. Try it today.

MAI-Voice-1 exports flawless audio in record speed. Can you tell the difference between the real recording and our model? Drop your best guess. Try it today.

Microsoft AI

29,481 görüntüleme • 5 ay önce

I tested the new Replit ⠕ Agent 4, and it completely changed how building apps feels. You can design, plan, and build at the same time. No waiting for the AI to finish one task before starting the next. Here’s a simple workflow I tried 👇

I tested the new Replit ⠕ Agent 4, and it completely changed how building apps feels. You can design, plan, and build at the same time. No waiting for the AI to finish one task before starting the next. Here’s a simple workflow I tried 👇

Your Tech Girl

12,661 görüntüleme • 7 ay önce

NEWS: Sam Altman says he thinks people are tired of the model picker, and that ideally GPT becomes a single model with a speed dial when you need it to go fast, and a cost dial when you don't:

NEWS: Sam Altman says he thinks people are tired of the model picker, and that ideally GPT becomes a single model with a speed dial when you need it to go fast, and a cost dial when you don't:

TBPN

95,631 görüntüleme • 15 gün önce

FREE access to Claude Opus 4.8 just opened on Chorus. Max plan costs $200/month. You can get it for free. Chorus gives you every latest Claude model including Opus 4.8 on the Max plan. Here's the exact flow: → chorus(.)com - register an account → select a plan - hit "continue free trial" → once inside, go to Settings bottom left → Billing - two plans available, Pro and Max → select Max → add your card and confirm. You can delete the card later (I always set up a card with no money on it) Trial works with your own card too - tested it myself, went through without issues. $200/month plan. Free trial. Opus 4.8 available immediately. Window is open right now. This method works right away, I've tried it myself.

FREE access to Claude Opus 4.8 just opened on Chorus. Max plan costs $200/month. You can get it for free. Chorus gives you every latest Claude model including Opus 4.8 on the Max plan. Here's the exact flow: → chorus(.)com - register an account → select a plan - hit "continue free trial" → once inside, go to Settings bottom left → Billing - two plans available, Pro and Max → select Max → add your card and confirm. You can delete the card later (I always set up a card with no money on it) Trial works with your own card too - tested it myself, went through without issues. $200/month plan. Free trial. Opus 4.8 available immediately. Window is open right now. This method works right away, I've tried it myself.

Atenov int.

136,987 görüntüleme • 4 ay önce

Having a stay-at-home wife can wipe out your entire tax bill 👇 Watch to see how it works... Comment the word "BONUS" and I'll send you a free breakdown of how anyone can do this 📊

Having a stay-at-home wife can wipe out your entire tax bill 👇 Watch to see how it works... Comment the word "BONUS" and I'll send you a free breakdown of how anyone can do this 📊

Michael Elefante

60,462 görüntüleme • 6 ay önce

With this release, users can opt for the ideal combination of intelligence, speed, and cost to suit their use case. Opus, our most intelligent model, achieves near-human comprehension capabilities. It can deftly handle open-ended prompts and tackle complex tasks.

With this release, users can opt for the ideal combination of intelligence, speed, and cost to suit their use case. Opus, our most intelligent model, achieves near-human comprehension capabilities. It can deftly handle open-ended prompts and tackle complex tasks.

Anthropic

179,549 görüntüleme • 2 yıl önce

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

Akshay 🚀

16,168 görüntüleme • 8 gün önce

36 GROK AGENTS. ASTRA ON FREE CREDITS. DUAL-MODEL ROUTING IS THE EDGE. not one chat window burning a paid invoice all day a swarm that routes cheap work to grok and only wakes gpt-6 astra when the task is actually hard ▹ the stack 36 agents on grok bot with flexible settings per role monitor, plan, write, code, QA, ship, each with its own lane part of the fleet is wired to gpt-6 astra through a china free-credit service layer free credits are not a toy promo here they are the fuel for frontier spikes without a monthly bleed ▹ dual-model routing easy jobs stay on grok: speed, volume, always-on loops hard jobs jump to astra: reasoning, long builds, sharp code the router decides by task type, not by ego if astra is not needed, astra does not spend free-credit bursts buy the expensive brain grok agents keep the factory running between bursts that is how the system feels unlimited not by breaking quotas, by refusing to waste them ▹ why it hits different most people pay frontier prices for every mid task operators split the brain and protect the credits 36 agents = parallel throughput dual routing = cost control with quality when it matters free credits = astra access without living on the invoice the constraint moved from "can i afford the model" to "did i route the job to the right model" ▹ the take single-model stacks die on bills and on boredom multi-agent + dual routing is the new default factory grok for the grind astra for the cut free credits for the spikes that used to empty the wallet bookmark this before everyone copies the route map comments: what % of your tasks actually deserve astra

36 GROK AGENTS. ASTRA ON FREE CREDITS. DUAL-MODEL ROUTING IS THE EDGE. not one chat window burning a paid invoice all day a swarm that routes cheap work to grok and only wakes gpt-6 astra when the task is actually hard ▹ the stack 36 agents on grok bot with flexible settings per role monitor, plan, write, code, QA, ship, each with its own lane part of the fleet is wired to gpt-6 astra through a china free-credit service layer free credits are not a toy promo here they are the fuel for frontier spikes without a monthly bleed ▹ dual-model routing easy jobs stay on grok: speed, volume, always-on loops hard jobs jump to astra: reasoning, long builds, sharp code the router decides by task type, not by ego if astra is not needed, astra does not spend free-credit bursts buy the expensive brain grok agents keep the factory running between bursts that is how the system feels unlimited not by breaking quotas, by refusing to waste them ▹ why it hits different most people pay frontier prices for every mid task operators split the brain and protect the credits 36 agents = parallel throughput dual routing = cost control with quality when it matters free credits = astra access without living on the invoice the constraint moved from "can i afford the model" to "did i route the job to the right model" ▹ the take single-model stacks die on bills and on boredom multi-agent + dual routing is the new default factory grok for the grind astra for the cut free credits for the spikes that used to empty the wallet bookmark this before everyone copies the route map comments: what % of your tasks actually deserve astra

cryptopsihoz

44,187 görüntüleme • 1 ay önce

Every model we serve is quality-scored and benchmarked before it enters our orchestration layer. From there, Corent selects the best execution path based on quality, price and speed. Same output quality, but with an average cost reduction of around 50%. That’s what orchestration should do.

Every model we serve is quality-scored and benchmarked before it enters our orchestration layer. From there, Corent selects the best execution path based on quality, price and speed. Same output quality, but with an average cost reduction of around 50%. That’s what orchestration should do.

corent

14,228 görüntüleme • 29 gün önce