Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Today we’re launching Model Vault — a dedicated, fully managed platform to run Cohere models securely and at scale. Model Vault delivers the control and isolation of self-hosting, without the operational burden: 🔒 Dedicated, isolated VPC ⚡ No noisy neighbors or rate limits 🔁 Elastic inference with predictable performance... show more

Cohere

118,719 subscribers

29,444 views • 6 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

We’re excited to introduce our newest state-of-the-art model: Command A! Command A provides enterprises maximum performance across agentic tasks with minimal compute requirements.

We’re excited to introduce our newest state-of-the-art model: Command A! Command A provides enterprises maximum performance across agentic tasks with minimal compute requirements.

Cohere

349,674 views • 1 year ago

We’re redefining what’s possible with AI. With the release of our latest model, Command A, optimized for real-world agentic and multilingual tasks, we’re demonstrating our commitment to bringing enterprises AI that goes beyond the ordinary, and offers security & efficiency. Our team has developed highly capable and efficient models that can be run on just 2 GPUs. Check out our tech report to learn more:

We’re redefining what’s possible with AI. With the release of our latest model, Command A, optimized for real-world agentic and multilingual tasks, we’re demonstrating our commitment to bringing enterprises AI that goes beyond the ordinary, and offers security & efficiency. Our team has developed highly capable and efficient models that can be run on just 2 GPUs. Check out our tech report to learn more:

Cohere

16,426 views • 1 year ago

Not everyone will succeed in the model business. "You've seen a lot of companies spend a huge amount on compute and then not end up with a good model." - Nick Frosst

Not everyone will succeed in the model business. "You've seen a lot of companies spend a huge amount on compute and then not end up with a good model." - Nick Frosst

Cohere

13,172 views • 1 month ago

Introducing Cohere's first open-source coding model: North Mini Code Small & efficient, designed for agentic performance and built for community input.

Introducing Cohere's first open-source coding model: North Mini Code Small & efficient, designed for agentic performance and built for community input.

Cohere

596,239 views • 2 months ago

Choose a model (any model) and build your application with it. Do not spend time swapping models early on. Do not try to optimize before you have a working system. This is one of the first recommendations I make to every new team I consult with. Eventually, it will be time to optimize the model. • You may need a cheaper model • You may need a faster model • You might need a smarter model Good luck if you stitched together 12 different APIs and SDKs from 7 different vendors. Over half of the companies I consult for run on Microsoft software and have access to Microsoft Foundry. Microsoft Foundry is a complete agentic ecosystem. If you're in that world and building AI applications, Microsoft Foundry is where everything lives: • Models (largest selection in the market) • Agentic SDK (Python, C#, JavaScript/TypeScript) • Tools • Evaluations • Monitoring They are fully integrated with GitHub and Visual Studio Code. The best part: Their agentic platform is fully agnostic of the models you use. You can integrate with any model using the same OpenAI-style API. Swapping one model for another takes 1 second.

Choose a model (any model) and build your application with it. Do not spend time swapping models early on. Do not try to optimize before you have a working system. This is one of the first recommendations I make to every new team I consult with. Eventually, it will be time to optimize the model. • You may need a cheaper model • You may need a faster model • You might need a smarter model Good luck if you stitched together 12 different APIs and SDKs from 7 different vendors. Over half of the companies I consult for run on Microsoft software and have access to Microsoft Foundry. Microsoft Foundry is a complete agentic ecosystem. If you're in that world and building AI applications, Microsoft Foundry is where everything lives: • Models (largest selection in the market) • Agentic SDK (Python, C#, JavaScript/TypeScript) • Tools • Evaluations • Monitoring They are fully integrated with GitHub and Visual Studio Code. The best part: Their agentic platform is fully agnostic of the models you use. You can integrate with any model using the same OpenAI-style API. Swapping one model for another takes 1 second.

Santiago

12,014 views • 6 months ago

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:

Daniel San

65,228 views • 1 year ago

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊

#Heroku is transitioning to a “sustaining engineering model”, or dev-speak for no new roadmap. 👀 Let us help you migrate your workloads to an Inference Cloud delivering trusted performance, radically simpler operations, and predictable economics—DigitalOcean. We’re ready to help you evaluate your options in a way that is realistic, structured, and built for the long term. 🌊

DigitalOcean

10,297 views • 6 months ago

How precise can a robot get… without ever seeing the object move? Turns out, just a geometric model is enough. A robot hit 73 precise 3D goals in a row, using only a geometric model. No demonstrations. No imitation learning. No fancy vision models. The secret? Sample where the robot could make contact. Let local contact-implicit control take over from there. That’s it. ✅ Real-time performance ✅ Natural contact making and breaking ✅ Works on tricky tasks like pushing and rotating This approach mixes global planning with local MPC. Fast. Accurate. Sample-efficient. Thanks for sharing, Michael Posa ! 📍Watch it in action: Paper:

How precise can a robot get… without ever seeing the object move? Turns out, just a geometric model is enough. A robot hit 73 precise 3D goals in a row, using only a geometric model. No demonstrations. No imitation learning. No fancy vision models. The secret? Sample where the robot could make contact. Let local contact-implicit control take over from there. That’s it. ✅ Real-time performance ✅ Natural contact making and breaking ✅ Works on tricky tasks like pushing and rotating This approach mixes global planning with local MPC. Fast. Accurate. Sample-efficient. Thanks for sharing, Michael Posa ! 📍Watch it in action: Paper:

Ilir Aliu

19,012 views • 1 year ago

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

45,459 views • 18 days ago

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 views • 2 years ago

We made a thing! Very happy to announce sqlcoder-pro and the Defog Alignment Platform. Available to use immediately without a wait-list, weights will be open-sourced very soon. The video does a quick show and tell comparison against ChatGPT (with gpt-4o). Read on for more details! TLDR 💪 equal (or better) performance on text-to-SQL as the most capable Claude-3.5 or GPT-4 models 🤝 You can use it today on a free plan/free trial, without a waitlist 🪽 self-hostable on a single RTX4090, with 2 second median generation times for SQL queries 🔁 exactly the same output every time, give the same prompt 👨🏻‍🏫 teachable and steerable: show the model what you want it to do 🛞 debuggable – you can understand WTF is going on inside the model, instead of treating it like a black box Let's dig into each of these one-by-one! Performance SQLCoder-8b-pro significantly exceeds the performance of our previous sqlcoder-8b model on Postgres text-to-SQL (from 88.2% to 90.2% accuracy - gpt-4o is at 87.6%, for reference). It is also better at following instructions. This was done via self-merges, hand crafted fine-tuning data, and adapting the training data to fit our tokenizer. Cost You can host this on the model on a single $3,500 RTX4090, and support ~5 requests/second via VLLM. If you're looking to host on the cloud instead, you can run it on a single L4 GPU that costs $300/mo on GCP Repeatability We have a dense 8b model with no MoE shenanigans. For the same prompt with temperature=0, you'll always get the same answer – which is critical in BI. Teachable In our alignment and feedback modes, you can give the model feedback on how it answered certain questions, and it will automatically adapt to the feedback. Debuggable You can use logprobs and attention scores to determine where, exactly is the model paying attention to inside a prompt + what it's getting confused by when generating outputs. Available today You can use Defog on the cloud today by going to docs[dot]defog[dot]ai, and getting an API key. Excited to hear what you think!

We made a thing! Very happy to announce sqlcoder-pro and the Defog Alignment Platform. Available to use immediately without a wait-list, weights will be open-sourced very soon. The video does a quick show and tell comparison against ChatGPT (with gpt-4o). Read on for more details! TLDR 💪 equal (or better) performance on text-to-SQL as the most capable Claude-3.5 or GPT-4 models 🤝 You can use it today on a free plan/free trial, without a waitlist 🪽 self-hostable on a single RTX4090, with 2 second median generation times for SQL queries 🔁 exactly the same output every time, give the same prompt 👨🏻‍🏫 teachable and steerable: show the model what you want it to do 🛞 debuggable – you can understand WTF is going on inside the model, instead of treating it like a black box Let's dig into each of these one-by-one! Performance SQLCoder-8b-pro significantly exceeds the performance of our previous sqlcoder-8b model on Postgres text-to-SQL (from 88.2% to 90.2% accuracy - gpt-4o is at 87.6%, for reference). It is also better at following instructions. This was done via self-merges, hand crafted fine-tuning data, and adapting the training data to fit our tokenizer. Cost You can host this on the model on a single $3,500 RTX4090, and support ~5 requests/second via VLLM. If you're looking to host on the cloud instead, you can run it on a single L4 GPU that costs $300/mo on GCP Repeatability We have a dense 8b model with no MoE shenanigans. For the same prompt with temperature=0, you'll always get the same answer – which is critical in BI. Teachable In our alignment and feedback modes, you can give the model feedback on how it answered certain questions, and it will automatically adapt to the feedback. Debuggable You can use logprobs and attention scores to determine where, exactly is the model paying attention to inside a prompt + what it's getting confused by when generating outputs. Available today You can use Defog on the cloud today by going to docs[dot]defog[dot]ai, and getting an API key. Excited to hear what you think!

Rishabh Srivastava

13,465 views • 2 years ago

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:

Tencent Hy

20,581 views • 7 months ago

zkAster Vault Private Sharing Link is LIVE! You can now share files instantly through a link or even privately to a specific address, all while keeping your identity, metadata, and content fully protected with zero-knowledge privacy. zkAster No exposure. No central storage. No one sees your data except the person you choose. zkAster keeps building. Welcome to the new standard of private, decentralized file sharing. #BNB $ZKASTER $ASTER

zkAster Vault Private Sharing Link is LIVE! You can now share files instantly through a link or even privately to a specific address, all while keeping your identity, metadata, and content fully protected with zero-knowledge privacy. zkAster No exposure. No central storage. No one sees your data except the person you choose. zkAster keeps building. Welcome to the new standard of private, decentralized file sharing. #BNB $ZKASTER $ASTER

Super Degen 💎

15,671 views • 9 months ago

We've added a new "reasoning" focus (beta) on Perplexity for Pro users. It will use the new OpenAI o1-mini. There is no search integration yet. The model is slow, and usage is limited because of rate limits. It is good for puzzles, math, and coding. Pro searches are your best way to use multi-step reasoning + web search.

We've added a new "reasoning" focus (beta) on Perplexity for Pro users. It will use the new OpenAI o1-mini. There is no search integration yet. The model is slow, and usage is limited because of rate limits. It is good for puzzles, math, and coding. Pro searches are your best way to use multi-step reasoning + web search.

Aravind Srinivas

288,808 views • 1 year ago

How to use 50+ API keys (models) for FREE on OpenClaw API??? - go to - login or register your account - click on "more models" - click on "use case" and select what you need it for - choose the model and open it - click on "view code" → "Generate API key" many models don't allow direct deploy, so use the "view code" button to generate API access basically Nvidia NIM gives you the ability to test almost any model from their list for FREE some of them are not worse than GPT 5.2 or Claude Opus 4.6, some might even perform better depending on the task how to understand if a model is efficient and compare it with others??? - go to - type the model name in search - click on "benchmarks" - you’ll see performance tests and rankings this way you can easily compare free models with paid ones of course there are RPM limits, on many models it’s around ~40 requests per minute each model is different, after generating the API key, RPM limits are shown in the top-right corner nothing stops you from using them, many work perfectly fine, super solid option for first tests and for learning OpenClaw or any other system where you need an AI API model

How to use 50+ API keys (models) for FREE on OpenClaw API??? - go to - login or register your account - click on "more models" - click on "use case" and select what you need it for - choose the model and open it - click on "view code" → "Generate API key" many models don't allow direct deploy, so use the "view code" button to generate API access basically Nvidia NIM gives you the ability to test almost any model from their list for FREE some of them are not worse than GPT 5.2 or Claude Opus 4.6, some might even perform better depending on the task how to understand if a model is efficient and compare it with others??? - go to - type the model name in search - click on "benchmarks" - you’ll see performance tests and rankings this way you can easily compare free models with paid ones of course there are RPM limits, on many models it’s around ~40 requests per minute each model is different, after generating the API key, RPM limits are shown in the top-right corner nothing stops you from using them, many work perfectly fine, super solid option for first tests and for learning OpenClaw or any other system where you need an AI API model

Ronin

58,846 views • 6 months ago

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

QVAC

11,382,936 views • 2 months ago

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

Roboflow

39,416 views • 2 months ago

🚨 GLM 5.3 just dropped, and here’s how to access the GLM family completely FREE. no subscription. no API credit cost & no rate limit. step 1️⃣: download the OpenCode app on your PC. it’s available on both Mac and Windows. 🔗: step 2️⃣: go to NVIDIA’s website, sign up with your email, verify your account, and generate a unified API key for free. NVIDIA offers free model endpoints for developers, and GLM 5.2 is one of the top models currently available there. 🔗: step 3️⃣: launch OpenCode, open the model selection menu, search for NVIDIA, and paste the API key you generated. also, there’s no rate limit, but traffic to the model through NVIDIA can get busy at times, so responses may occasionally be slower or temporarily unavailable. the only catch: NVIDIA hasn’t listed GLM 5.3 yet, so for now you’ll be using GLM 5.2. once GLM 5.3 gets added to NVIDIA’s model catalog, you should be able to switch to it the same way.

🚨 GLM 5.3 just dropped, and here’s how to access the GLM family completely FREE. no subscription. no API credit cost & no rate limit. step 1️⃣: download the OpenCode app on your PC. it’s available on both Mac and Windows. 🔗: step 2️⃣: go to NVIDIA’s website, sign up with your email, verify your account, and generate a unified API key for free. NVIDIA offers free model endpoints for developers, and GLM 5.2 is one of the top models currently available there. 🔗: step 3️⃣: launch OpenCode, open the model selection menu, search for NVIDIA, and paste the API key you generated. also, there’s no rate limit, but traffic to the model through NVIDIA can get busy at times, so responses may occasionally be slower or temporarily unavailable. the only catch: NVIDIA hasn’t listed GLM 5.3 yet, so for now you’ll be using GLM 5.2. once GLM 5.3 gets added to NVIDIA’s model catalog, you should be able to switch to it the same way.

m0h

78,449 views • 8 days ago

50% of my consulting work right now is helping companies use open-source models at scale. Everyone knows how to use an open-source LLM on their computers, but it's really hard to do this at scale for thousands of users. Here is how this plays out: 1. A team builds a prototype using DeepSeek. 2. Everything looks good. It works! 3. They follow an online guide to deploy the model online. 4. They ask 10 users to try the app. 5. Latency spikes everywhere. 6. The entire system halts. 7. They blame DeepSeek and try again using a new model. The problem is always with scaling inference, not the model. Here is one recommendation I give companies: Check out Nebius Token Factory if you don't want to ever think about deploying an open-source model again. This is a managed inference platform for deploying open-source LLMs at scale. This is not for prototypes or research experiments. This is for when you have a real application with real users. Three important notes about Token Factory: • You have complete control over how inference runs. • You have predictable tail latency (P99, not averages). • No surprise costs when you scale up. You can preplan your budget. Check it out here: Here are two codes you can use to get 100 hours of GPU usage on me: ymJLFa2ARYSKEdqb AdckcZaYjm7KqYY7 Thanks to the Nebius team for their continuous partnership.

50% of my consulting work right now is helping companies use open-source models at scale. Everyone knows how to use an open-source LLM on their computers, but it's really hard to do this at scale for thousands of users. Here is how this plays out: 1. A team builds a prototype using DeepSeek. 2. Everything looks good. It works! 3. They follow an online guide to deploy the model online. 4. They ask 10 users to try the app. 5. Latency spikes everywhere. 6. The entire system halts. 7. They blame DeepSeek and try again using a new model. The problem is always with scaling inference, not the model. Here is one recommendation I give companies: Check out Nebius Token Factory if you don't want to ever think about deploying an open-source model again. This is a managed inference platform for deploying open-source LLMs at scale. This is not for prototypes or research experiments. This is for when you have a real application with real users. Three important notes about Token Factory: • You have complete control over how inference runs. • You have predictable tail latency (P99, not averages). • No surprise costs when you scale up. You can preplan your budget. Check it out here: Here are two codes you can use to get 100 hours of GPU usage on me: ymJLFa2ARYSKEdqb AdckcZaYjm7KqYY7 Thanks to the Nebius team for their continuous partnership.

Santiago

48,443 views • 6 months ago

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗

📢 New in Visual Studio: Bring Your Own Model to Chat We’re expanding flexibility in Copilot for Visual Studio. You can now connect API keys from providers like OpenAI, Anthropic, and Google to run Chat on the model of your choice. This update gives you: ✅ Choice – Access the latest models beyond the built-in defaults. ✅ Customization – Select models that fit your infrastructure and requirements. ✅ Control – Manage API usage, limits, and billing directly with your provider. ✅ Flexibility – Switch seamlessly between Copilot’s built-in models and your own. This release is the first step toward a more open and customizable chat experience in Visual Studio, with more providers and capabilities to come. 🔗

Visual Studio

11,873 views • 1 year ago