Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or... routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.show more

Alok Bishoyi

6,009 subscribers

16,327 görüntüleme • 3 ay önce •via X (Twitter)

Bilim & Teknoloji Finans

Anya Rossi• Live Now

Private livecam show

44 Yorum

SKYLINE🥷 profil fotoğrafı

SKYLINE🥷3 ay önce

single model workflows are aging fast.

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

we are rolling out early access to EVO Router. Get it here

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

router achieves similar performance of frontier closed source models while costing costing a fraction of it

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

more about our launch here

Mukesh profil fotoğrafı

Mukesh3 ay önce

Congrats!

Boardy profil fotoğrafı

Boardy3 ay önce

@andrewdsouza Alok, this could be a strong fit for the infra and platform teams I know wrestling with model cost and latency. Happy to see who’d be useful to meet.

Lakshmi Narayana profil fotoğrafı

Lakshmi Narayana3 ay önce

Let's go!! I'm done with the generic routers and optimizers that are rerouting based on public benchmarks that have nothing to do with my workloads.

Adarsh profil fotoğrafı

Adarsh3 ay önce

WOW

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

🙏🫡

Kshitij Arora profil fotoğrafı

Kshitij Arora3 ay önce

Woah! Can’t wait to try this out, 30-60% reduction is bonkers

Ella | AI Agents Developer profil fotoğrafı

Ella | AI Agents Developer3 ay önce

I'd love to see the workloads behind those numbers

Arun Setty Kodavali profil fotoğrafı

Arun Setty Kodavali3 ay önce

Much required!!! When can I get access!?

Saaket Sharma profil fotoğrafı

Saaket Sharma3 ay önce

Lfg!! 🥂🥂

Rahul Parmar profil fotoğrafı

Rahul Parmar3 ay önce

Inference always bites my head, will try!

pH profil fotoğrafı

pH3 ay önce

NotYourAverageRouter™️ 🚀

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

you know it !

Aditya Sharma profil fotoğrafı

Aditya Sharma3 ay önce

with open source catching up to frontier intelligence this is much needed

Ankit profil fotoğrafı

Ankit3 ay önce

congratulations 🥂

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

thanks boss 🙏

Kanak profil fotoğrafı

Kanak3 ay önce

Crazy! Is it based on vllm or something custom inference framework?

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Hey Kanak, at this moment we mostly do routing orchestration rather than at the inference serving layer. We partner with various inference providers who have the necessary SLAs that meet our customers standards

Sathvik Kollu profil fotoğrafı

Sathvik Kollu3 ay önce

congrats!

Shivam Jaglan profil fotoğrafı

Shivam Jaglan3 ay önce

Much needed!

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

yessir !

Amarnath profil fotoğrafı

Amarnath3 ay önce

Looks promising. Congratulations on the launch Alok.

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Thank you !!

Yash Khemchandani profil fotoğrafı

Yash Khemchandani3 ay önce

Congratulations!!

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Thank you !

jay.agent 🤖 profil fotoğrafı

jay.agent 🤖3 ay önce

🚀👀

Abhishek Kankani profil fotoğrafı

Abhishek Kankani3 ay önce

This looks awesome @alokbishoyi97! Congrats 🚀

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Thanks Abhishek !

sandipan profil fotoğrafı

sandipan3 ay önce

Excellent

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Thank you Sandipan ji 🙏

Vinay Dadi profil fotoğrafı

Vinay Dadi3 ay önce

This is super cool. Congratulations

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

thanks vinay !

Gaurav profil fotoğrafı

Gaurav3 ay önce

Congratulations on the launch, Alok!

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

Thanks Gaurav!

Abu-Bakr profil fotoğrafı

Abu-Bakr3 ay önce

This is lit

Fajar M Reza profil fotoğrafı

Fajar M Reza3 ay önce

Inference routing matters when production feedback optimizes quality, latency, and cost together.

BLAZE profil fotoğrafı

BLAZE3 ay önce

Wow this is epic!! How long did it take you to make this video?

Alok Bishoyi profil fotoğrafı

Alok Bishoyi3 ay önce

hey! A couple of days tbh

jani profil fotoğrafı

jani3 ay önce

super cool stuff! congrats 🥂

Biti8 profil fotoğrafı

Biti83 ay önce

$evo

Bharat Jajoo profil fotoğrafı

Bharat Jajoo3 ay önce

Congratulations on the launch!

Benzer Videolar

Weave is launching the number 1 prompt router in the world. It enables you to get 70% more efficient use of your tokens. We analyzed millions of prompts and found that the vast majority don't need a frontier model. Weave Router fixes that. It analyzes your prompt and routes it to the highest quality model with the lowest cost (across open and closed source models). This all happens in your current workflow on Claude, Cursor or Codex so you don't have to change a thing. Early customers have seen an ~70% reduction in costs without any slowdowns. Source code available.

Weave is launching the number 1 prompt router in the world. It enables you to get 70% more efficient use of your tokens. We analyzed millions of prompts and found that the vast majority don't need a frontier model. Weave Router fixes that. It analyzes your prompt and routes it to the highest quality model with the lowest cost (across open and closed source models). This all happens in your current workflow on Claude, Cursor or Codex so you don't have to change a thing. Early customers have seen an ~70% reduction in costs without any slowdowns. Source code available.

Adam Cohen

28,389 görüntüleme • 5 ay önce

Introducing Cursor Router, our intelligent model router that selects the right model for the task at hand. Router delivers frontier-quality results at 60% lower cost.

Introducing Cursor Router, our intelligent model router that selects the right model for the task at hand. Router delivers frontier-quality results at 60% lower cost.

Cursor

2,678,471 görüntüleme • 3 ay önce

Introducing Smart Routing in Unity AI Gateway. Stop overpaying for AI. Smart Routing matches each coding task to the right model and harness based on what the task needs, so higher-cost models can differentiate on intelligence while lower-cost models differentiate on cost and performance. Match frontier quality and cut task costs by 30%+.

Introducing Smart Routing in Unity AI Gateway. Stop overpaying for AI. Smart Routing matches each coding task to the right model and harness based on what the task needs, so higher-cost models can differentiate on intelligence while lower-cost models differentiate on cost and performance. Match frontier quality and cut task costs by 30%+.

Databricks

126,791 görüntüleme • 2 ay önce

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

Akshay 🚀

16,223 görüntüleme • 1 ay önce

Introducing Entelligence AI Model Router, dynamic turn based routing trained on engineering tasks that cuts costs by over 50% Automatically route every coding turn to the model with the best balance of quality, latency, and cost. Escalate to frontier models when they’re actually needed Balanced mode reduces spend by 46% per session Eco mode helps teams save > 76%

Introducing Entelligence AI Model Router, dynamic turn based routing trained on engineering tasks that cuts costs by over 50% Automatically route every coding turn to the model with the best balance of quality, latency, and cost. Escalate to frontier models when they’re actually needed Balanced mode reduces spend by 46% per session Eco mode helps teams save > 76%

Aiswarya Sankar

87,590 görüntüleme • 3 ay önce

The most effective AI systems don't rely on a single model. Frontier models provide state-of-the-art performance for complex tasks, while routers automatically select lightweight, open-source models for simpler jobs to optimize accuracy, latency, and cost. Learn more:

The most effective AI systems don't rely on a single model. Frontier models provide state-of-the-art performance for complex tasks, while routers automatically select lightweight, open-source models for simpler jobs to optimize accuracy, latency, and cost. Learn more:

NVIDIA AI

12,152 görüntüleme • 8 ay önce

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

DigitalOcean

318,497 görüntüleme • 2 ay önce

🚨 Announcing Smart Model Router - Create Your Own Custom Router To Route To Your Favorite LLM - Route to GPT Sol, Muse Spark, Grok 4.5 and Fable - Optimize for cost or performance - Use Fable only for hard coding Once you create a router, you can use it on our Chat, Agent or in an API! Coming on Monday - Support for all CLI clients including Code, Claude Code and others

🚨 Announcing Smart Model Router - Create Your Own Custom Router To Route To Your Favorite LLM - Route to GPT Sol, Muse Spark, Grok 4.5 and Fable - Optimize for cost or performance - Use Fable only for hard coding Once you create a router, you can use it on our Chat, Agent or in an API! Coming on Monday - Support for all CLI clients including Code, Claude Code and others

Bindu Reddy

2,572,380 görüntüleme • 4 ay önce

Introducing model routing to Factory. Factory Router picks the right model for every task, automatically. Maintain frontier performance while cutting costs by 25%.

Introducing model routing to Factory. Factory Router picks the right model for every task, automatically. Maintain frontier performance while cutting costs by 25%.

Factory

2,273,088 görüntüleme • 5 ay önce

AI has two sides: the applications that need intelligence, and the computers that provide it. OptimAI Router brings them closer together. For developers: access a growing range of open AI models through one connection. For node operators: turn your computer into a potential source of AI inference, contributing computing resources to real workloads across the network. One side brings the demand. The other brings the capacity. OptimAI connects them. We're building toward an AI network where developers can access diverse models, more computing resources can participate, and intelligence isn't tied to a single provider. Build with open models. Contribute computing power. Help grow the network. One network. From AI requests to distributed compute. OptimAI Router. Coming soon. ⚡

AI has two sides: the applications that need intelligence, and the computers that provide it. OptimAI Router brings them closer together. For developers: access a growing range of open AI models through one connection. For node operators: turn your computer into a potential source of AI inference, contributing computing resources to real workloads across the network. One side brings the demand. The other brings the capacity. OptimAI connects them. We're building toward an AI network where developers can access diverse models, more computing resources can participate, and intelligence isn't tied to a single provider. Build with open models. Contribute computing power. Help grow the network. One network. From AI requests to distributed compute. OptimAI Router. Coming soon. ⚡

OptimAI Network

15,091 görüntüleme • 1 ay önce

Monitor and control your AI spend on every provider on Our early users save 40% on average. Every week, the price-intelligence-latency frontier shifts, and we expect this trend to continue. Tradeoffs between latency, reasoning, cost, service tier, open source and closed source models are shifting constantly. Router sends every request to the model that's actually best for the task and helps you control what tokens you buy. We benchmark it against real work: ~40% lower cost for the same outputs. Today we're opening it to everyone. Two lines of code or just change your base URL. No Ramp account needed. Free through 2026, first $26 on us. Get an API key today at

Monitor and control your AI spend on every provider on Our early users save 40% on average. Every week, the price-intelligence-latency frontier shifts, and we expect this trend to continue. Tradeoffs between latency, reasoning, cost, service tier, open source and closed source models are shifting constantly. Router sends every request to the model that's actually best for the task and helps you control what tokens you buy. We benchmark it against real work: ~40% lower cost for the same outputs. Today we're opening it to everyone. Two lines of code or just change your base URL. No Ramp account needed. Free through 2026, first $26 on us. Get an API key today at

Veeral Patel

1,432,108 görüntüleme • 2 ay önce

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Santiago

15,543 görüntüleme • 2 ay önce

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

OpenRouter

1,334,435 görüntüleme • 1 ay önce

Inference, training, routing, governance, and sandboxes are the building blocks of every AI product today. We’ve learned a lot from our customers about what it takes to build that system well: OpenEvidence trains models that outperform the closed frontier, then serves them on Dedicated Inference. Datadog, Inc. post-trains on Loops. Opendoor routes every AI request across the company through Grid. This week we have three launches expanding our inference platform: - Grid: gateway, intelligent router, and guardrails for every model your company uses (launching tomorrow) - Sandboxes: stateful, VM-isolated environments for agents that scale to thousands of instances in seconds (launching Thursday) - Loops: serverless SFT and RL on open-weight models, deployable straight to Dedicated Inference (GA Friday) Amir Haghighat, sai, Paul Sinaï, and Charlie O'Neill break down what we're launching. You might notice the visuals have a new look as well.

Inference, training, routing, governance, and sandboxes are the building blocks of every AI product today. We’ve learned a lot from our customers about what it takes to build that system well: OpenEvidence trains models that outperform the closed frontier, then serves them on Dedicated Inference. Datadog, Inc. post-trains on Loops. Opendoor routes every AI request across the company through Grid. This week we have three launches expanding our inference platform: - Grid: gateway, intelligent router, and guardrails for every model your company uses (launching tomorrow) - Sandboxes: stateful, VM-isolated environments for agents that scale to thousands of instances in seconds (launching Thursday) - Loops: serverless SFT and RL on open-weight models, deployable straight to Dedicated Inference (GA Friday) Amir Haghighat, sai, Paul Sinaï, and Charlie O'Neill break down what we're launching. You might notice the visuals have a new look as well.

Baseten

37,826 görüntüleme • 7 gün önce

Introducing the Morph Model Router. It chooses the best model for each task in under 50ms. Keep frontier performance while lowering latency and cost. Available today in our API

Introducing the Morph Model Router. It chooses the best model for each task in under 50ms. Keep frontier performance while lowering latency and cost. Available today in our API

Morph

20,281 görüntüleme • 5 ay önce

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

Introducing NVIDIA Nemotron 3 Ultra. A frontier smart open model built for long-running agents that need to plan, reason, use tools and keep working across complex coding, research and enterprise workflows. Up to 5x faster inference and up to 30% lower cost for agentic tasks. Learn more:

NVIDIA

230,981 görüntüleme • 5 ay önce

Inference providers hate this man... Your LLM deployment can deliver 30 to 50% better performance after initial deployment. If your current inference provider isn't learning your workload and tuning the stack continuously after deployment, you need another provider. Wafer's continual inference agents do that work. Wafer's continual inference agents profile your production workload to identify where scheduling, decoding, or GPU memory pressure limits performance. The agents benchmark changes to batching, serving engines, and GPU kernels against your workload. Each candidate must preserve correctness and meet your reliability targets before it ships. The agents keep profiling as traffic changes, identifying new bottlenecks and further opportunities to improve performance per dollar. Watch emilio andere explain how that work continues after deployment in his conversation with theCUBE.

wafer

42,840 görüntüleme • 1 ay önce

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

19,009 görüntüleme • 4 ay önce

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.

Runway

62,291 görüntüleme • 3 ay önce

🚨 Frontier Model Router - Mix And Match All the Frontier LLMs Our most popular custom router is our frontier-model router that combines the best LLMs for all your tasks GPT 5.6 sol - data analysis Fable / Opus 4.8 - coding Flash 3.5 - summarization Grok 4.5 - real-time Pick your favorite LLMs for all your tasks. As of today the custom routers can be used in ANY harness - Abacus AI, Claude, Codex, Open-Code....

🚨 Frontier Model Router - Mix And Match All the Frontier LLMs Our most popular custom router is our frontier-model router that combines the best LLMs for all your tasks GPT 5.6 sol - data analysis Fable / Opus 4.8 - coding Flash 3.5 - summarization Grok 4.5 - real-time Pick your favorite LLMs for all your tasks. As of today the custom routers can be used in ANY harness - Abacus AI, Claude, Codex, Open-Code....

Bindu Reddy

4,592,609 görüntüleme • 3 ay önce