Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.

digitalocean's profile picture

DigitalOcean

222,668 subscribers

318,366 Aufrufe • vor 23 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Debarshi
Debarshivor 23 Tagen

This is what we were waiting for

Profilbild von DigitalOcean
DigitalOceanvor 23 Tagen

No more waiting, @DebarshiRaha! 🙌

Profilbild von AI Civilization Consortium
AI Civilization Consortiumvor 23 Tagen

Model stickiness preferences (“affinity” is a good term!) need not only be restricted to sessions. In application where agents create other agents, stickiness could apply to an agent or in some cases a family of agents (like subs or inherited).

Ähnliche Videos

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.
0:36

.Kimi.ai K3 from Moonshot AI is now live on DigitalOcean Inference Engine. 1M-token context, native vision, built to run agentic tasks for hours. Supported on Inference Router and model synthesis to maximize intelligence per dollar. No setup, no model ops.

DigitalOcean

665,105 Aufrufe • vor 1 Monat

Treating every inference request the same is lighting money on fire. DigitalOcean Inference Router sends each request to the right model automatically–lower cost, faster response, and no quality tradeoff. Open-source engine. Drop-in API. Live since April. ✨
0:28

Treating every inference request the same is lighting money on fire. DigitalOcean Inference Router sends each request to the right model automatically–lower cost, faster response, and no quality tradeoff. Open-source engine. Drop-in API. Live since April. ✨

DigitalOcean

594,950 Aufrufe • vor 3 Monaten

DigitalOcean Inference Router is now available through OpenCode & setup is 3 steps. ☁️🆕🤝 Run/connect & it automatically routes to the right model for the job. No more paying frontier prices for a commit message.
0:33

DigitalOcean Inference Router is now available through OpenCode & setup is 3 steps. ☁️🆕🤝 Run/connect & it automatically routes to the right model for the job. No more paying frontier prices for a commit message.

DigitalOcean

20,992 Aufrufe • vor 3 Monaten

Stop overpaying for every LLM call. 💸 The DigitalOcean Inference Router resolves intent in ~200ms and beats GPT-5.1 and Claude Sonnet 4.5 on routing accuracy. Powered by Rust, WASM, and purpose-built SLMs, it adds milliseconds, not seconds, to every request. 🌊
0:40

Stop overpaying for every LLM call. 💸 The DigitalOcean Inference Router resolves intent in ~200ms and beats GPT-5.1 and Claude Sonnet 4.5 on routing accuracy. Powered by Rust, WASM, and purpose-built SLMs, it adds milliseconds, not seconds, to every request. 🌊

DigitalOcean

19,853 Aufrufe • vor 3 Monaten

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.
3:22

DigitalOcean’s model synthesis tool is now available in public preview. 🆕 A new server-side tool: run multiple models on one inference request and get a single synthesized response, no custom orchestration. Just add it to your existing inference request.

DigitalOcean

2,192,210 Aufrufe • vor 1 Monat

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊
0:23

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊

DigitalOcean

10,297 Aufrufe • vor 7 Monaten

🚨 GMI estimated ~$0.11 in savings on one 3-step Python coding workflow. I gave GMI Router the same e-commerce codebase and split the work into 3 requests: 👉 Code analysis → Cost Mode → DeepSeek-V4-Flash → ~$0.046 saved 👉 Debugging → Balanced Mode → DeepSeek-V4-Flash → ~$0.042 saved 👉 Implementation + testing → Quality Mode → DeepSeek-V4-Flash → ~$0.019 saved Total: ~$0.11 estimated savings vs Claude Opus 4.8, based on GMI's estimated token usage. But here's what caught my attention: Router selected the same model for all 3 tasks — even though I changed the mode each time. And that's actually the point. GMI Cloud 's Router is cache-aware. When the same model can handle the next request, sticking with it can help preserve cache reuse and reduce cost. There's another reason: These tasks weren't especially difficult. So even in Quality Mode, Router didn't need to move to a heavier model. The result: 1 codebase. 3 tasks. 3 routing modes. 1 lightweight model. ~$0.11 estimated savings. This is why request-level routing is more interesting than simply assigning a different model to every task.
1:11

🚨 GMI estimated ~$0.11 in savings on one 3-step Python coding workflow. I gave GMI Router the same e-commerce codebase and split the work into 3 requests: 👉 Code analysis → Cost Mode → DeepSeek-V4-Flash → ~$0.046 saved 👉 Debugging → Balanced Mode → DeepSeek-V4-Flash → ~$0.042 saved 👉 Implementation + testing → Quality Mode → DeepSeek-V4-Flash → ~$0.019 saved Total: ~$0.11 estimated savings vs Claude Opus 4.8, based on GMI's estimated token usage. But here's what caught my attention: Router selected the same model for all 3 tasks — even though I changed the mode each time. And that's actually the point. GMI Cloud 's Router is cache-aware. When the same model can handle the next request, sticking with it can help preserve cache reuse and reduce cost. There's another reason: These tasks weren't especially difficult. So even in Quality Mode, Router didn't need to move to a heavier model. The result: 1 codebase. 3 tasks. 3 routing modes. 1 lightweight model. ~$0.11 estimated savings. This is why request-level routing is more interesting than simply assigning a different model to every task.

Anthara Fairooz

28,520 Aufrufe • vor 16 Tagen

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.
0:31

Introducing Runway Media Router. The first preference-optimized router for generative media. Instead of hand-picking a model for every request, you define what "best" means once, for cost, quality, or latency, and the router selects the right video, image, or audio model automatically. Live now in Runway Dev.

Runway

62,291 Aufrufe • vor 1 Monat

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:
0:34

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

18,979 Aufrufe • vor 2 Monaten

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or routing through a classifier. EVO (YC F26) can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.
1:13

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or routing through a classifier. EVO (YC F26) can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.

Alok Bishoyi

15,996 Aufrufe • vor 1 Monat

🚨 POC for CVE-2025-55182 that works on Next.js 16.0.6 Here are the exact, battle-tested queries you need — Censys, Shodan, FOFA, ZoomEye, Quake, BinaryEdge, and Nuclei matchers — all tuned specifically to find Next.js RSC / React Server Components instances vulnerable to CVE-2025-55182 (React2Shell). ⸻ ✅ 1. SHODAN QUERY (380K+ ASSETS) Find all servers leaking RSC Server Actions: Basic Query "Vary: RSC, Next-Router-State-Tree" More Aggressive Variant http.headers.vary:"RSC" AND http.headers.vary:"Next-Router-State-Tree" Superwide Coverage "Next-Router-State-Tree" OR "x-nextjs-cache" OR "server-actions" OR "__RSC__" Focused on Vulnerable Cache Indicators "x-nextjs-cache: HIT" "Next-Router-State-Tree" ⸻ ✅ 2. CENSYS QUERY (270K+ ASSETS) (match the screenshot you posted) Exact Censys Search services.http.response.headers.vary: "RSC, Next-Router-State-Tree" Safer Multi-Matcher services.http.response.headers.vary: "RSC" AND services.http.response.headers.vary: "Next-Router-State-Tree" Detect RSC Payload Exposure (critical) services.http.response.body: "__RSC__" Detect Flight Data Leaks services.http.response.body: "server-reference-manifest" ⸻ ✅ 3. FOFA QUERY (CHINA’S OSINT GIANT) (VERY POWERFUL for RSC/Next.js) Exact Header Based header="Next-Router-State-Tree" && header="RSC" Alternative (match screenshot patterns) "Next-Router-State-Tree" && "x-nextjs-cache" For massive result count body="__RSC__" || header="server-actions" ⸻ ✅ 4. ZOOMEYE QUERY ZoomEye scans often catch Node.js apps Shodan misses. Exact Unicode-Ready Query "Next-Router-State-Tree" && "RSC" Advanced app:"Next.js" && header:"RSC" ⸻ ✅ 5. QUAKE SEARCH (360K+ MATCHES) header:"Next-Router-State-Tree" AND header:"RSC" ⸻ ✅ 6. BINARYEDGE QUERY http.response.headers.vary:"Next-Router-State-Tree" ⸻ ✅ 7. QUERY headers:"Next-Router-State-Tree" && headers:"RSC" ⸻ 🎯 8. NUCLEI MATCHER (to detect RSC without scanning payloads) If you want a nuclei detector you can plug into your scanner: matchers: - type: word part: header words: - "RSC" - "Next-Router-State-Tree" - "server-actions" - "__RSC__" ⸻ 🚩 BONUS — THE MOST ADVANCED CROSS-ENGINE QUERY Use this when you want maximum global coverage: "Next-Router-State-Tree" OR "RSC" OR "__RSC__" OR "server-actions" OR "x-nextjs-cache" OR "Next-Server-Action" This identifies: •Next.js App Router •RSC endpoints •Server Actions •Flight data APIs •Pages exposing cache HITs (required for exploitation) •Systems likely vulnerable to CVE-2025-55182 (React2Shell)
0:32

🚨 POC for CVE-2025-55182 that works on Next.js 16.0.6 Here are the exact, battle-tested queries you need — Censys, Shodan, FOFA, ZoomEye, Quake, BinaryEdge, and Nuclei matchers — all tuned specifically to find Next.js RSC / React Server Components instances vulnerable to CVE-2025-55182 (React2Shell). ⸻ ✅ 1. SHODAN QUERY (380K+ ASSETS) Find all servers leaking RSC Server Actions: Basic Query "Vary: RSC, Next-Router-State-Tree" More Aggressive Variant http.headers.vary:"RSC" AND http.headers.vary:"Next-Router-State-Tree" Superwide Coverage "Next-Router-State-Tree" OR "x-nextjs-cache" OR "server-actions" OR "__RSC__" Focused on Vulnerable Cache Indicators "x-nextjs-cache: HIT" "Next-Router-State-Tree" ⸻ ✅ 2. CENSYS QUERY (270K+ ASSETS) (match the screenshot you posted) Exact Censys Search services.http.response.headers.vary: "RSC, Next-Router-State-Tree" Safer Multi-Matcher services.http.response.headers.vary: "RSC" AND services.http.response.headers.vary: "Next-Router-State-Tree" Detect RSC Payload Exposure (critical) services.http.response.body: "__RSC__" Detect Flight Data Leaks services.http.response.body: "server-reference-manifest" ⸻ ✅ 3. FOFA QUERY (CHINA’S OSINT GIANT) (VERY POWERFUL for RSC/Next.js) Exact Header Based header="Next-Router-State-Tree" && header="RSC" Alternative (match screenshot patterns) "Next-Router-State-Tree" && "x-nextjs-cache" For massive result count body="__RSC__" || header="server-actions" ⸻ ✅ 4. ZOOMEYE QUERY ZoomEye scans often catch Node.js apps Shodan misses. Exact Unicode-Ready Query "Next-Router-State-Tree" && "RSC" Advanced app:"Next.js" && header:"RSC" ⸻ ✅ 5. QUAKE SEARCH (360K+ MATCHES) header:"Next-Router-State-Tree" AND header:"RSC" ⸻ ✅ 6. BINARYEDGE QUERY http.response.headers.vary:"Next-Router-State-Tree" ⸻ ✅ 7. QUERY headers:"Next-Router-State-Tree" && headers:"RSC" ⸻ 🎯 8. NUCLEI MATCHER (to detect RSC without scanning payloads) If you want a nuclei detector you can plug into your scanner: matchers: - type: word part: header words: - "RSC" - "Next-Router-State-Tree" - "server-actions" - "__RSC__" ⸻ 🚩 BONUS — THE MOST ADVANCED CROSS-ENGINE QUERY Use this when you want maximum global coverage: "Next-Router-State-Tree" OR "RSC" OR "__RSC__" OR "server-actions" OR "x-nextjs-cache" OR "Next-Server-Action" This identifies: •Next.js App Router •RSC endpoints •Server Actions •Flight data APIs •Pages exposing cache HITs (required for exploitation) •Systems likely vulnerable to CVE-2025-55182 (React2Shell)

X

10,544 Aufrufe • vor 9 Monaten

.RadixArk brought DeepSeek V4 to production with DigitalOcean and AMD. AMD Instinct™ MI350X GPU Droplets, plus joint engineering that drove a ~10x throughput gain. When the team that builds the inference engine picks your stack, that's the signal.
0:42

.RadixArk brought DeepSeek V4 to production with DigitalOcean and AMD. AMD Instinct™ MI350X GPU Droplets, plus joint engineering that drove a ~10x throughput gain. When the team that builds the inference engine picks your stack, that's the signal.

DigitalOcean

13,381 Aufrufe • vor 2 Monaten

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇
1:10

New on DigitalOcean's Inference Engine: GLM-5.3-Flash from Z.ai. Agentic coding that rivals top-tier models, now natively multimodal for browser and computer use. 1M token context, MoE architecture, one endpoint swap to try it. Senior Developer Advocate Ryan O'Connor breaks it down, and Z.ai for Startups is there to help AI startups build faster with GLM models. 👇

DigitalOcean

140,583 Aufrufe • vor 12 Tagen

A cache miss happens when a model has to re-read (and re-charge you for) text it had already processed before. This can be caused by: - switching models - idling past 5mins - editing earlier threads Pi can notify you when this happens and how much it costs!
0:24

A cache miss happens when a model has to re-read (and re-charge you for) text it had already processed before. This can be caused by: - switching models - idling past 5mins - editing earlier threads Pi can notify you when this happens and how much it costs!

Pi

79,012 Aufrufe • vor 3 Tagen

Among the fastest DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B inference in the market, per Artificial Analysis benchmarks (April 2026). ⚡️🤖 Sub-1-second TTFT. 230 tokens per second. Co-designed every layer of the stack with Inferact, performance optimized vLLM, all on NVIDIA HGX B300. Live on DigitalOcean Serverless Inference now. Full breakdown in the comments. ⬇️
0:25

Among the fastest DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B inference in the market, per Artificial Analysis benchmarks (April 2026). ⚡️🤖 Sub-1-second TTFT. 230 tokens per second. Co-designed every layer of the stack with Inferact, performance optimized vLLM, all on NVIDIA HGX B300. Live on DigitalOcean Serverless Inference now. Full breakdown in the comments. ⬇️

DigitalOcean

39,063 Aufrufe • vor 4 Monaten

.Grok Build is on DigitalOcean Marketplace. 🆕☁️🤖 SSH in. Run grok. Build. @xAI's coding agent, pre-configured on DigitalOcean Serverless Inference.
0:38

.Grok Build is on DigitalOcean Marketplace. 🆕☁️🤖 SSH in. Run grok. Build. @xAI's coding agent, pre-configured on DigitalOcean Serverless Inference.

DigitalOcean

807,645 Aufrufe • vor 2 Monaten

From POC to production inference. 🤖🌁 Join us at #DOdeploy in San Francisco where we'll discuss inference-first architectures, predictable inference economics, operational simplicity at scale, & more.
0:26

From POC to production inference. 🤖🌁 Join us at #DOdeploy in San Francisco where we'll discuss inference-first architectures, predictable inference economics, operational simplicity at scale, & more.

DigitalOcean

12,195 Aufrufe • vor 6 Monaten

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.
34:46

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.

h100envy

61,470 Aufrufe • vor 2 Monaten

Kimi K3 is out. 2.8T parameters. 1.4 TB of weights, frontier-level performance. Serving it on your own infra is the hard part - you need multi-node inference spread across every cluster you can grab capacity from. And then autoscaling, cache-aware routing, PD disaggregation - an entire serving stack around the model. SkyPilot Endpoints ships a complete stack on day 0 for Kimi.ai's Kimi K3: • Runs on GPUs you own - hyperscalers, neoclouds or on-prem. • Production-ready out of the box - autoscaling, KV-aware routing, PD disaggregation, observability • Fault-tolerant by design - endpoint stays up even when GPUs fail Frontier open models, on GPUs you own. 🔗
0:20

Kimi K3 is out. 2.8T parameters. 1.4 TB of weights, frontier-level performance. Serving it on your own infra is the hard part - you need multi-node inference spread across every cluster you can grab capacity from. And then autoscaling, cache-aware routing, PD disaggregation - an entire serving stack around the model. SkyPilot Endpoints ships a complete stack on day 0 for Kimi.ai's Kimi K3: • Runs on GPUs you own - hyperscalers, neoclouds or on-prem. • Production-ready out of the box - autoscaling, KV-aware routing, PD disaggregation, observability • Fault-tolerant by design - endpoint stays up even when GPUs fail Frontier open models, on GPUs you own. 🔗

SkyPilot

14,794 Aufrufe • vor 1 Monat

.Character.AI, AMD, & #DigitalOcean are scaling the Qwen3-235B open-weight Mixture-of-Experts AI model to serve approximately 20M users. 🤖📈☁️ This required tuning a 5600/140 (ISL/OSL) workload with FP8 inference on AMD Instinct™ MI325X GPUs to maximize request throughput (QPS).
0:23

.Character.AI, AMD, & #DigitalOcean are scaling the Qwen3-235B open-weight Mixture-of-Experts AI model to serve approximately 20M users. 🤖📈☁️ This required tuning a 5600/140 (ISL/OSL) workload with FP8 inference on AMD Instinct™ MI325X GPUs to maximize request throughput (QPS).

DigitalOcean

27,726 Aufrufe • vor 8 Monaten