正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

DigitalOcean

225,927 subscribers

318,497 次观看 • 2 个月前 •via X (Twitter)

新闻政治教育科学技术

Anya Rossi• Live Now

Private livecam show

3 条评论

Debarshi 的头像

Debarshi2 个月前

This is what we were waiting for

DigitalOcean 的头像

DigitalOcean2 个月前

No more waiting, @DebarshiRaha! 🙌

AI Civilization Consortium 的头像

AI Civilization Consortium2 个月前

Model stickiness preferences (“affinity” is a good term!) need not only be restricted to sessions. In application where agents create other agents, stickiness could apply to an agent or in some cases a family of agents (like subs or inherited).

相关视频

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.

DigitalOcean

672,368 次观看 • 3 个月前

Treating every inference request the same is lighting money on fire. DigitalOcean Inference Router sends each request to the right model automatically–lower cost, faster response, and no quality tradeoff. Open-source engine. Drop-in API. Live since April. ✨

Treating every inference request the same is lighting money on fire. DigitalOcean Inference Router sends each request to the right model automatically–lower cost, faster response, and no quality tradeoff. Open-source engine. Drop-in API. Live since April. ✨

DigitalOcean

595,422 次观看 • 5 个月前

DigitalOcean Inference Router is now available through OpenCode & setup is 3 steps. ☁️🆕🤝 Run/connect & it automatically routes to the right model for the job. No more paying frontier prices for a commit message.

DigitalOcean Inference Router is now available through OpenCode & setup is 3 steps. ☁️🆕🤝 Run/connect & it automatically routes to the right model for the job. No more paying frontier prices for a commit message.

DigitalOcean

20,992 次观看 • 5 个月前

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

LLM Routing, clearly explained! (routing can cost more than not routing at all) The basic idea sounds simple: Send easy requests to cheaper models and reserve frontier models for difficult tasks. But production routing is rarely that straightforward. Every classification adds latency. General-purpose models can misread intent. Routing rules become outdated. And switching models during an agent session can destroy prefix-cache savings, increase costs, and create inconsistent behavior. In this video, I cover: → Why DIY routing often breaks in production → How prefix caching changes the economics → Why agents need session pinning and model affinity → How Plano and Arch-Router handle intent resolution → How to build, test, and evaluate a router with DigitalOcean Routing works, but only when the infrastructure accounts for the full session, not just the latest request. The real question is not whether routing can reduce costs. It is how much of your AI traffic has been overpaying this whole time. Get started with Digital Ocean inference router: I recently wrote a detailed article on the same topic. The article is quoted below.

Akshay 🚀

16,223 次观看 • 1 个月前

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

Introducing typesafe/jev-router: a cache-aware model router powered by Jev and TypeSafe AI The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost. Here's how it works 👇🏻

OpenRouter

1,334,435 次观看 • 1 个月前

Stop overpaying for every LLM call. 💸 The DigitalOcean Inference Router resolves intent in ~200ms and beats GPT-5.1 and Claude Sonnet 4.5 on routing accuracy. Powered by Rust, WASM, and purpose-built SLMs, it adds milliseconds, not seconds, to every request. 🌊

Stop overpaying for every LLM call. 💸 The DigitalOcean Inference Router resolves intent in ~200ms and beats GPT-5.1 and Claude Sonnet 4.5 on routing accuracy. Powered by Rust, WASM, and purpose-built SLMs, it adds milliseconds, not seconds, to every request. 🌊

DigitalOcean

19,853 次观看 • 5 个月前

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.

DigitalOcean

2,192,364 次观看 • 3 个月前

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊

DigitalOcean

10,297 次观看 • 9 个月前

🚨 GMI estimated ~$0.11 in savings on one 3-step Python coding workflow. I gave GMI Router the same e-commerce codebase and split the work into 3 requests: 👉 Code analysis → Cost Mode → DeepSeek-V4-Flash → ~$0.046 saved 👉 Debugging → Balanced Mode → DeepSeek-V4-Flash → ~$0.042 saved 👉 Implementation + testing → Quality Mode → DeepSeek-V4-Flash → ~$0.019 saved Total: ~$0.11 estimated savings vs Claude Opus 4.8, based on GMI's estimated token usage. But here's what caught my attention: Router selected the same model for all 3 tasks — even though I changed the mode each time. And that's actually the point. GMI Cloud 's Router is cache-aware. When the same model can handle the next request, sticking with it can help preserve cache reuse and reduce cost. There's another reason: These tasks weren't especially difficult. So even in Quality Mode, Router didn't need to move to a heavier model. The result: 1 codebase. 3 tasks. 3 routing modes. 1 lightweight model. ~$0.11 estimated savings. This is why request-level routing is more interesting than simply assigning a different model to every task.

🚨 GMI estimated ~$0.11 in savings on one 3-step Python coding workflow. I gave GMI Router the same e-commerce codebase and split the work into 3 requests: 👉 Code analysis → Cost Mode → DeepSeek-V4-Flash → ~$0.046 saved 👉 Debugging → Balanced Mode → DeepSeek-V4-Flash → ~$0.042 saved 👉 Implementation + testing → Quality Mode → DeepSeek-V4-Flash → ~$0.019 saved Total: ~$0.11 estimated savings vs Claude Opus 4.8, based on GMI's estimated token usage. But here's what caught my attention: Router selected the same model for all 3 tasks — even though I changed the mode each time. And that's actually the point. GMI Cloud 's Router is cache-aware. When the same model can handle the next request, sticking with it can help preserve cache reuse and reduce cost. There's another reason: These tasks weren't especially difficult. So even in Quality Mode, Router didn't need to move to a heavier model. The result: 1 codebase. 3 tasks. 3 routing modes. 1 lightweight model. ~$0.11 estimated savings. This is why request-level routing is more interesting than simply assigning a different model to every task.

Anthara Fairooz

28,520 次观看 • 2 个月前

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.

Runway

62,291 次观看 • 3 个月前

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

19,009 次观看 • 4 个月前

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.

Alok Bishoyi

16,327 次观看 • 3 个月前

🚨 POC for CVE-2025-55182 that works on Next.js 16.0.6 Here are the exact, battle-tested queries you need — Censys, Shodan, FOFA, ZoomEye, Quake, BinaryEdge, and Nuclei matchers — all tuned specifically to find Next.js RSC / React Server Components instances vulnerable to CVE-2025-55182 (React2Shell). ⸻ ✅ 1. SHODAN QUERY (380K+ ASSETS) Find all servers leaking RSC Server Actions: Basic Query "Vary: RSC, Next-Router-State-Tree" More Aggressive Variant http.headers.vary:"RSC" AND http.headers.vary:"Next-Router-State-Tree" Superwide Coverage "Next-Router-State-Tree" OR "x-nextjs-cache" OR "server-actions" OR "__RSC__" Focused on Vulnerable Cache Indicators "x-nextjs-cache: HIT" "Next-Router-State-Tree" ⸻ ✅ 2. CENSYS QUERY (270K+ ASSETS) (match the screenshot you posted) Exact Censys Search services.http.response.headers.vary: "RSC, Next-Router-State-Tree" Safer Multi-Matcher services.http.response.headers.vary: "RSC" AND services.http.response.headers.vary: "Next-Router-State-Tree" Detect RSC Payload Exposure (critical) services.http.response.body: "__RSC__" Detect Flight Data Leaks services.http.response.body: "server-reference-manifest" ⸻ ✅ 3. FOFA QUERY (CHINA’S OSINT GIANT) (VERY POWERFUL for RSC/Next.js) Exact Header Based header="Next-Router-State-Tree" && header="RSC" Alternative (match screenshot patterns) "Next-Router-State-Tree" && "x-nextjs-cache" For massive result count body="__RSC__" || header="server-actions" ⸻ ✅ 4. ZOOMEYE QUERY ZoomEye scans often catch Node.js apps Shodan misses. Exact Unicode-Ready Query "Next-Router-State-Tree" && "RSC" Advanced app:"Next.js" && header:"RSC" ⸻ ✅ 5. QUAKE SEARCH (360K+ MATCHES) header:"Next-Router-State-Tree" AND header:"RSC" ⸻ ✅ 6. BINARYEDGE QUERY http.response.headers.vary:"Next-Router-State-Tree" ⸻ ✅ 7. QUERY headers:"Next-Router-State-Tree" && headers:"RSC" ⸻ 🎯 8. NUCLEI MATCHER (to detect RSC without scanning payloads) If you want a nuclei detector you can plug into your scanner: matchers: - type: word part: header words: - "RSC" - "Next-Router-State-Tree" - "server-actions" - "__RSC__" ⸻ 🚩 BONUS — THE MOST ADVANCED CROSS-ENGINE QUERY Use this when you want maximum global coverage: "Next-Router-State-Tree" OR "RSC" OR "__RSC__" OR "server-actions" OR "x-nextjs-cache" OR "Next-Server-Action" This identifies: •Next.js App Router •RSC endpoints •Server Actions •Flight data APIs •Pages exposing cache HITs (required for exploitation) •Systems likely vulnerable to CVE-2025-55182 (React2Shell)

🚨 POC for CVE-2025-55182 that works on Next.js 16.0.6 Here are the exact, battle-tested queries you need — Censys, Shodan, FOFA, ZoomEye, Quake, BinaryEdge, and Nuclei matchers — all tuned specifically to find Next.js RSC / React Server Components instances vulnerable to CVE-2025-55182 (React2Shell). ⸻ ✅ 1. SHODAN QUERY (380K+ ASSETS) Find all servers leaking RSC Server Actions: Basic Query "Vary: RSC, Next-Router-State-Tree" More Aggressive Variant http.headers.vary:"RSC" AND http.headers.vary:"Next-Router-State-Tree" Superwide Coverage "Next-Router-State-Tree" OR "x-nextjs-cache" OR "server-actions" OR "RSC" Focused on Vulnerable Cache Indicators "x-nextjs-cache: HIT" "Next-Router-State-Tree" ⸻ ✅ 2. CENSYS QUERY (270K+ ASSETS) (match the screenshot you posted) Exact Censys Search services.http.response.headers.vary: "RSC, Next-Router-State-Tree" Safer Multi-Matcher services.http.response.headers.vary: "RSC" AND services.http.response.headers.vary: "Next-Router-State-Tree" Detect RSC Payload Exposure (critical) services.http.response.body: "RSC" Detect Flight Data Leaks services.http.response.body: "server-reference-manifest" ⸻ ✅ 3. FOFA QUERY (CHINA’S OSINT GIANT) (VERY POWERFUL for RSC/Next.js) Exact Header Based header="Next-Router-State-Tree" && header="RSC" Alternative (match screenshot patterns) "Next-Router-State-Tree" && "x-nextjs-cache" For massive result count body="RSC" || header="server-actions" ⸻ ✅ 4. ZOOMEYE QUERY ZoomEye scans often catch Node.js apps Shodan misses. Exact Unicode-Ready Query "Next-Router-State-Tree" && "RSC" Advanced app:"Next.js" && header:"RSC" ⸻ ✅ 5. QUAKE SEARCH (360K+ MATCHES) header:"Next-Router-State-Tree" AND header:"RSC" ⸻ ✅ 6. BINARYEDGE QUERY http.response.headers.vary:"Next-Router-State-Tree" ⸻ ✅ 7. QUERY headers:"Next-Router-State-Tree" && headers:"RSC" ⸻ 🎯 8. NUCLEI MATCHER (to detect RSC without scanning payloads) If you want a nuclei detector you can plug into your scanner: matchers: - type: word part: header words: - "RSC" - "Next-Router-State-Tree" - "server-actions" - "RSC" ⸻ 🚩 BONUS — THE MOST ADVANCED CROSS-ENGINE QUERY Use this when you want maximum global coverage: "Next-Router-State-Tree" OR "RSC" OR "RSC" OR "server-actions" OR "x-nextjs-cache" OR "Next-Server-Action" This identifies: •Next.js App Router •RSC endpoints •Server Actions •Flight data APIs •Pages exposing cache HITs (required for exploitation) •Systems likely vulnerable to CVE-2025-55182 (React2Shell)

X

10,544 次观看 • 11 个月前

Ran a real 3-turn LinkedIn research workflow through GMI Cloud Router in Balanced mode. Here’s what it actually picked, turn by turn : Turn 1 — “Research 2026 AI/marketing trends and summarize 5 insights” → DeepSeek-V4-Flash-0731 → ~$0.047 cheaper than Claude Opus 4.8 Turn 2 — “Analyze those insights for content angles + pain points” → GPT-5.6-Luna → ~$0.026 saved vs Opus 4.8 Turn 3 — “Turn the top angle into 5 LinkedIn post concepts” → GPT-5.6-Luna → ~$0.040 saved vs Opus 4.8 Total estimated savings: ~$0.11 across 3 turns. The interesting part was Turns 2 and 3. The task changed from analysis to writing, but Router stayed on GPT-5.6-Luna. That makes sense because GMI Router is cache-aware: when the same model can still handle the next task well, staying on that model can preserve cache reuse and avoid the cost of re-serving the context to another model. And all 3 tasks were relatively lightweight. They didn't suddenly require a more expensive frontier model just because the task type changed. So the routing wasn't about constantly switching models. It was about finding a model that was good enough for the task while keeping the cost down, and staying there when that still made sense. That’s a much more interesting way to think about model routing. GMI Cloud #AI #LLM #AIAgents #GenerativeAI

Ran a real 3-turn LinkedIn research workflow through GMI Cloud Router in Balanced mode. Here’s what it actually picked, turn by turn : Turn 1 — “Research 2026 AI/marketing trends and summarize 5 insights” → DeepSeek-V4-Flash-0731 → ~$0.047 cheaper than Claude Opus 4.8 Turn 2 — “Analyze those insights for content angles + pain points” → GPT-5.6-Luna → ~$0.026 saved vs Opus 4.8 Turn 3 — “Turn the top angle into 5 LinkedIn post concepts” → GPT-5.6-Luna → ~$0.040 saved vs Opus 4.8 Total estimated savings: ~$0.11 across 3 turns. The interesting part was Turns 2 and 3. The task changed from analysis to writing, but Router stayed on GPT-5.6-Luna. That makes sense because GMI Router is cache-aware: when the same model can still handle the next task well, staying on that model can preserve cache reuse and avoid the cost of re-serving the context to another model. And all 3 tasks were relatively lightweight. They didn't suddenly require a more expensive frontier model just because the task type changed. So the routing wasn't about constantly switching models. It was about finding a model that was good enough for the task while keeping the cost down, and staying there when that still made sense. That’s a much more interesting way to think about model routing. GMI Cloud #AI #LLM #AIAgents #GenerativeAI

Tom

29,592 次观看 • 2 个月前

A cache miss happens when a model has to re-read (and re-charge you for) text it had already processed before. This can be caused by: - switching models - idling past 5mins - editing earlier threads Pi can notify you when this happens and how much it costs!

A cache miss happens when a model has to re-read (and re-charge you for) text it had already processed before. This can be caused by: - switching models - idling past 5mins - editing earlier threads Pi can notify you when this happens and how much it costs!

Pi

83,375 次观看 • 2 个月前

Switching models is scary when the only real test is in production. Gateway Evals grades a new model on your agent's real tasks, then shadows it on live traffic. You see cost, latency, and every answer before a customer does. Try it now in Merge Gateway:

Switching models is scary when the only real test is in production. Gateway Evals grades a new model on your agent's real tasks, then shadows it on live traffic. You see cost, latency, and every answer before a customer does. Try it now in Merge Gateway:

Merge

12,358 次观看 • 1 个月前

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇

DigitalOcean

140,795 次观看 • 2 个月前

.RadixArk brought DeepSeek V4 to production with DigitalOcean and AMD. AMD Instinct™ MI350X GPU Droplets, plus joint engineering that drove a ~10x throughput gain. When the team that builds the inference engine picks your stack, that's the signal.

.RadixArk brought DeepSeek V4 to production with DigitalOcean and AMD. AMD Instinct™ MI350X GPU Droplets, plus joint engineering that drove a ~10x throughput gain. When the team that builds the inference engine picks your stack, that's the signal.

DigitalOcean

13,868 次观看 • 3 个月前

Among the fastest DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B inference in the market, per Artificial Analysis benchmarks (April 2026). ⚡️🤖 Sub-1-second TTFT. 230 tokens per second. Co-designed every layer of the stack with Inferact, performance optimized vLLM, all on NVIDIA HGX B300. Live on DigitalOcean Serverless Inference now. Full breakdown in the comments. ⬇️

Among the fastest DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B inference in the market, per Artificial Analysis benchmarks (April 2026). ⚡️🤖 Sub-1-second TTFT. 230 tokens per second. Co-designed every layer of the stack with Inferact, performance optimized vLLM, all on NVIDIA HGX B300. Live on DigitalOcean Serverless Inference now. Full breakdown in the comments. ⬇️

DigitalOcean

39,063 次观看 • 6 个月前

Introducing Liquid Inference by Architect, a new LLM model router where inference providers compete dynamically to offer lowest-cost tokens. Providers onboard in minutes, not weeks. Buyers have full price visibility and receive 20% of referred fees in free inference. More info: Liquid Inference can be used in place of any OpenAI or Anthropic API-compatible backend. Behind it, our team has leveraged our experience trading and building financial exchanges to create real-time two-sided price discovery for inference. Benefits for buyers and providers: INFERENCE BUYERS • Fully compatible with agentic coding tools: Claude Code, Codex, OpenCode, Cursor, Pi, Cline, and many more. Full multi-modal support. • Hundreds of open- and closed-weight models. Create routing rule presets or use our Auto routing algorithms. • Providers compete dynamically on price for every prompt, offering the lowest marginal cost. • Sign up for free with email, first 500 users get $20 of free inference. • Receive 20% of referred fees as free inference, 10% for second-level referrals. INFERENCE PROVIDERS • Onboard through the Liquid Inference app. We verify new providers in minutes, not weeks. • All prompts are OpenAI API standard. Simple REST/Websocket API to register models and quotes. • Update quotes dynamically based on your own costs. Monetize GPU capacity only when you want. • Standardize payouts via Stripe, full itemized records of all jobs performed.

Introducing Liquid Inference by Architect, a new LLM model router where inference providers compete dynamically to offer lowest-cost tokens. Providers onboard in minutes, not weeks. Buyers have full price visibility and receive 20% of referred fees in free inference. More info: Liquid Inference can be used in place of any OpenAI or Anthropic API-compatible backend. Behind it, our team has leveraged our experience trading and building financial exchanges to create real-time two-sided price discovery for inference. Benefits for buyers and providers: INFERENCE BUYERS • Fully compatible with agentic coding tools: Claude Code, Codex, OpenCode, Cursor, Pi, Cline, and many more. Full multi-modal support. • Hundreds of open- and closed-weight models. Create routing rule presets or use our Auto routing algorithms. • Providers compete dynamically on price for every prompt, offering the lowest marginal cost. • Sign up for free with email, first 500 users get $20 of free inference. • Receive 20% of referred fees as free inference, 10% for second-level referrals. INFERENCE PROVIDERS • Onboard through the Liquid Inference app. We verify new providers in minutes, not weeks. • All prompts are OpenAI API standard. Simple REST/Websocket API to register models and quotes. • Update quotes dynamically based on your own costs. Monetize GPU capacity only when you want. • Standardize payouts via Stripe, full itemized records of all jobs performed.

Brett Harrison

39,450 次观看 • 1 个月前