Загрузка видео...

Не удалось загрузить видео

На главную

Somebody quit his data center job and leased an empty warehouse. filled it with rows of server machines. charges clients between $4-6 K p/month for private isolated deployment. Each client receives dedicated machines to host model. He runs all on vLLM.

129,027 просмотров • 2 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

New course: MCP: Build Rich-Context AI Apps with Anthropic. Learn to build AI apps that access tools, data, and prompts using the Model Context Protocol in this short course, created in partnership with Anthropic Anthropic and taught by Elie Schoppik Elie Schoppik, its Head of Technical Education. Connecting AI applications to external systems that bring rich context to LLM-based applications has often meant writing custom integrations for each use case. MCP is an open protocol that standardizes how LLMs access tools, data, and prompts from external sources, and simplifies how you provide context to your LLM-based applications. For example, you can provide context via third-party tools that let your LLM make API calls to search the web, access data from local docs, retrieve code from a GitHub repo, and so on. MCP, developed by Anthropic, is based on a client-server architecture that defines the communication details between an MCP client, hosted inside the AI application, and an MCP server that exposes tools, resources, and prompt templates. The server can be a subprocess launched by the client that runs locally or an independent process running remotely. In this hands-on course, you'll learn the core architecture behind MCP. You’ll create an MCP-compatible chatbot, build and deploy an MCP server, and connect the chatbot to your MCP server and other open-source servers. Here’s what you’ll do: - Understand why MCP makes AI development less fragmented and standardizes connections between AI applications and external data sources - Learn the core components of the client-server architecture of MCP and the underlying communication mechanism - Build a chatbot with custom tools for searching academic papers, and transform it into an MCP-compatible application - Build a local MCP server that exposes tools, resources, and prompt templates using FastMCP, and test it using MCP Inspector - Create an MCP client inside your chatbot to dynamically connect to your server - Connect your chatbot to reference servers built by Anthropic’s MCP team, such as filesystem, which implements filesystem operations, and fetch, which extracts contents from the web as markdown - Configure Claude Desktop to connect to your server and others, and explore how it abstracts away the low-level logic of MCP clients - Deploy your MCP server remotely and test it with the Inspector or other MCP-compatible applications - Learn about the roadmap for future MCP development, such as multi-agent architecture, MCP registry API, server discovery, authorization, and authentication MCP is an exciting and important technology that lets you build rich-context AI applications that connect to a growing ecosystem of MCP servers, with minimal integration work. Please sign up here!

Andrew Ng

142,234 просмотров • 1 год назад

dude runs a private AI host out of his basement on 8 stacked 3090s an accounting firm signed the moment they realized their client financials would physically never touch a third party the rig pays his mortgage now the setup is eight RTX 3090s lined up on an open frame down in his basement - fans roaring, yellow zip ties holding cables, a little monitor blinking stats beside a cheap keyboard. it was built as a mining rig and the dashboard still looks the part the accounting firm didn’t care that it looked like a crypto leftover they cared about one promise no cloud provider could make them - that their clients’ financial records would never leave a machine the firm could physically point to that promise closed the deal here’s the corner they were backed into: accounting firms hold the most sensitive data imaginable - tax filings, payroll, full financial histories. they wanted AI to speed up the tedious work, but routing that data through a cloud model was a liability their partners would never sign. so they sat frozen while the tech moved on without them he handed them the one version that cleared legal: a private model running on hardware in a basement, not a server farm they’d never see the build behind the rig: eight 3090s stack up to 192GB of combined vram - enough to run a 70B model with real context through vLLM. the firm’s documents get indexed into an isolated vector database, so every answer pulls only from their own files. queries hit a local endpoint, replies come back, nothing ever leaves the basement. used 3090s were the quiet genius move - around $700 a card instead of triple that for new silicon the rig that once mined coins for pennies now serves inference that bills like a service the contract that covers his house: → managing partner came down for an in-person demo → asked the only question that mattered: where does the data live → he pointed at the rig and said “right here, nowhere else” → they signed a monthly deal before leaving the basement the economics underneath: → 8 used 3090s: ~$5,600, paid off in the first two months → electricity: a few hundred a month under load → what the firm pays: enough to clear his mortgage every month → margin: almost all of it, because the data center is under his own house real firms burn a fortune on compliant cloud infrastructure he does the same job in a basement, and the data is safer for the dumbest reason possible - it has nowhere else it can go the rig that looks one loose cable from death is quietly the most trusted machine his client has ever touched

regent0x

13,334 просмотров • 2 месяцев назад