正在加载视频...

视频加载失败

返回首页

Agentic Website Scraping using Firecrawl: How to Setup Locally? 🔥 Integrate with @LangChainAI LlamaIndex 🦙 🤖 AI Agent Integration PraisonAI 🌐 Set up locally Docker 📊 Data Processing Automation Llama 3 LLM Groq Inc 🔄 Search @yousearchengine Mendable @firecrawl_dev Subscribe: YT:

MervinPraison's profile picture

Mervin Praison

10,733 subscribers

20,909 次观看 • 2 年前 •via X (Twitter)

教育科学技术新闻政治
LIVE

Anya Rossi• Live Now

Private livecam show

10 条评论

Nicolas Camara 的头像
Nicolas Camara2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev 🔥🔥🔥

Eric Ciarla 的头像
Eric Ciarla2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev Great work @MervinPraison!

Adam Silverman (Hiring!) 🖇️ 的头像
Adam Silverman (Hiring!) 🖇️2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev Awesome

Happy 的头像
Happy2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev like

Essa Mamdani 的头像
Essa Mamdani2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev Use free opensource for crawling.

Дмитрий Суриков 的头像
Дмитрий Суриков2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev AI + Web3 = Innovation! 🚀🌐 @magnetaixyz @din_lol_ GODIN

Igor Andreichenk 的头像
Igor Andreichenk2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev Your AI, your profit! 💰🤖 @magnetaixyz @din_lol_ GODIN

NONAME❤️ WELL3 的头像
NONAME❤️ WELL32 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev ModelFi: The AI game-changer! 🎮💫 @magnetaixyz @din_lol_ GODIN

Ninew💙NOYA.ai 的头像
Ninew💙NOYA.ai2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev 🔥 Web3Go who? It's all about DIN now! Go @din_lol_ $ARCA

Mercury.fi 的头像
Mercury.fi2 年前

@LangChainAI @llama_index @PraisonAI @Docker @GroqInc @yousearchengine @mendableai @firecrawl_dev AI + Web3 = Innovation! 🚀🌐 @magnetaixyz @din_lol_ GODIN

相关视频

How to Fine Tune Llama 3 for Better Instruction Following? 🚀 Setup Environment 📊 Train Model 💾 Save Model Locally 🌐 Upload Model to Hugging Face 📈 Evaluate Performance 🛠 Using 🦥 Unsloth AI Subscribe: YT: Meta AI at Meta Daniel Han
8:54

How to Fine Tune Llama 3 for Better Instruction Following? 🚀 Setup Environment 📊 Train Model 💾 Save Model Locally 🌐 Upload Model to Hugging Face 📈 Evaluate Performance 🛠 Using 🦥 Unsloth AI Subscribe: YT: Meta AI at Meta Daniel Han

Mervin Praison

14,107 次观看 • 2 年前

Building “RAG 2.0” is just making Claude Code running over your filesystem 🤖🗂️ To make this work well, you need to solve three things 1️⃣ Virtualize your filesystem to prevent the agent from messing stuff up. AgentFS by Turso is a nice example of how you can give the agent access to a copy of all your files without messing up your raw data. 2️⃣ Parse unstructured documents like PDFs, pptx, Word into an LLM-ready format. Agentic OCR solutions like LlamaParse can help here 3️⃣ Creating an agentic loop with human-in-the-loop. If you want to control the agent implementation instead of using Claude Code out of the box, you can use LlamaIndex 🦙 workflows to help orchestrate these long-running agent tasks. Shoutout Clelia Bertelli (🦙/acc), check it out! Blog: Repo:
1:35

Building “RAG 2.0” is just making Claude Code running over your filesystem 🤖🗂️ To make this work well, you need to solve three things 1️⃣ Virtualize your filesystem to prevent the agent from messing stuff up. AgentFS by Turso is a nice example of how you can give the agent access to a copy of all your files without messing up your raw data. 2️⃣ Parse unstructured documents like PDFs, pptx, Word into an LLM-ready format. Agentic OCR solutions like LlamaParse can help here 3️⃣ Creating an agentic loop with human-in-the-loop. If you want to control the agent implementation instead of using Claude Code out of the box, you can use LlamaIndex 🦙 workflows to help orchestrate these long-running agent tasks. Shoutout Clelia Bertelli (🦙/acc), check it out! Blog: Repo:

Jerry Liu

55,620 次观看 • 7 个月前

This is how I use LLM to scrape 99% of websites Many people didn't realise you can build agentic scraper to: 1. Handle Authentication, Human verification, Captcha 2. Handle pagination & complex UI interactions 3. Adaptive as website structure change 4. Scrape large set of data What used to take hours now can be automated in mins; Here I show case how do you automate a scraping job on Upwork where people are paying $50~$80 per hour; 0:00 Intro 1:54 Methods overview 4:52 Web Scraper agent using Spider Cloud 🕷️ Jina AI 8:37 Handle website auth & captcha using AgentQL 20:27 AI buy tickets MultiOn AI If you have any further question or want to get deep dive into the code example in the video, you can join my community where I post tips weekly:
22:43

This is how I use LLM to scrape 99% of websites Many people didn't realise you can build agentic scraper to: 1. Handle Authentication, Human verification, Captcha 2. Handle pagination & complex UI interactions 3. Adaptive as website structure change 4. Scrape large set of data What used to take hours now can be automated in mins; Here I show case how do you automate a scraping job on Upwork where people are paying $50~$80 per hour; 0:00 Intro 1:54 Methods overview 4:52 Web Scraper agent using Spider Cloud 🕷️ Jina AI 8:37 Handle website auth & captcha using AgentQL 20:27 AI buy tickets MultiOn AI If you have any further question or want to get deep dive into the code example in the video, you can join my community where I post tips weekly:

Jason Zhou

128,111 次观看 • 1 年前

Introducing Sharpe Search: On-Chain Search AI Agent Powered by Hive Intelligence We’re thrilled to announce the launch of Sharpe Search, a crypto search AI agent powered by Hive Intelligence Designed to simplify blockchain data interaction, Sharpe Search represents a significant step toward making crypto more accessible and actionable for users at every level. Sharpe Search leverages Hive Intelligence’s advanced search API to provide real-time, actionable insights across the blockchain ecosystem. Here’s a detailed look at what Sharpe Search is, how it works: What Is Sharpe Search? At its core, Sharpe Search is an AI agent purpose-built for querying and analyzing on-chain data. It takes the complexity out of blockchain exploration by enabling users to ask questions in plain language and receive detailed, accurate responses. Whether you’re looking to monitor wallet activity, track portfolio positions, or analyze transaction history, Sharpe Search ensures that the answers are at your fingertips—accurate, comprehensive, and delivered instantly. How Does Sharpe Search Work? Sharpe Search is powered by Hive Intelligence, a search engine API designed to make blockchain data easily accessible and AI-ready. Here’s a breakdown of how it enables Sharpe Search to function effectively: 1. LLM-Optimized Query Processing Sharpe Search leverages Hive Intelligence's optimized responses for large language models. This ensures that AI agents can process blockchain data in a structured format, delivering precise answers to complex user queries. 2. Natural Language Interaction Forget the need for technical knowledge. Sharpe Search supports natural language queries, making it as simple as typing: - “What tokens are in my wallet? Am I eligible for any airdrop I haven't claimed yet?” - “Check me my last 100 transactions, tell me if I interacted with any protocol with recent hacks” - “Track my wallet activity over the past month, suggest optimised portfolio based on best stable yields available” 3. Real-Time Insights Across Multi-Chains Using Hive Intelligence, Sharpe Search connects to over 20 chains and 5000+ Protocols. This real-time access ensures that the AI agent provides up-to-date and actionable insights, no matter how dynamic the blockchain environment. 4. Unified API Access Sharpe Search consolidates fragmented blockchain data through Hive’s unified API. Instead of dealing with multiple integrations, Sharpe Search uses a single access point to aggregate and query data, reducing complexity for both users and developers. Technical Depth: The AI Agent Advantage Sharpe Search's design philosophy revolves around the principle of creating an intuitive, AI-driven experience. Here’s what makes its technology stand out: Data Indexing and Aggregation: Hive Intelligence employs advanced indexing algorithms to aggregate data from multiple chains. This ensures that Sharpe Search can retrieve information within milliseconds, even when querying vast datasets. Dynamic Updates: Blockchain data is volatile. Sharpe Search processes dynamic updates in real time, enabling users to act on the most recent metrics, transactions, and balances without delays. Contextual Understanding: The AI agent parses natural language queries and contextualizes them to blockchain-specific scenarios. For instance, when querying “Show portfolio details,” Sharpe Search understands the underlying requirements—fetching wallet holdings, token values, and current positions. Hive Intelligence: The Backbone of Sharpe Search While Sharpe Search takes center stage, Hive Intelligence provides the critical infrastructure to make it all possible. Its LLM-ready responses and multi-chain support ensure that Sharpe Search operates at the forefront of blockchain data accessibility. By launching Hive Intelligence through Sharpe Launchpad, Sharpe reinforces its commitment to supporting innovation in the blockchain space. Hive’s infrastructure not only powers Sharpe Search but also lays the groundwork for future AI agents to thrive in the ecosystem. What’s Next for Sharpe Search? Currently in invite-only access, Sharpe Search is preparing for a broader public release. Future updates will include: - Expanded Blockchain Coverage: More chains and protocols will be added. - Enhanced Query Flexibility: Even more advanced natural language capabilities. Stay tuned for the public launch and get ready to explore crypto like never before!
1:33

Introducing Sharpe Search: On-Chain Search AI Agent Powered by Hive Intelligence We’re thrilled to announce the launch of Sharpe Search, a crypto search AI agent powered by Hive Intelligence Designed to simplify blockchain data interaction, Sharpe Search represents a significant step toward making crypto more accessible and actionable for users at every level. Sharpe Search leverages Hive Intelligence’s advanced search API to provide real-time, actionable insights across the blockchain ecosystem. Here’s a detailed look at what Sharpe Search is, how it works: What Is Sharpe Search? At its core, Sharpe Search is an AI agent purpose-built for querying and analyzing on-chain data. It takes the complexity out of blockchain exploration by enabling users to ask questions in plain language and receive detailed, accurate responses. Whether you’re looking to monitor wallet activity, track portfolio positions, or analyze transaction history, Sharpe Search ensures that the answers are at your fingertips—accurate, comprehensive, and delivered instantly. How Does Sharpe Search Work? Sharpe Search is powered by Hive Intelligence, a search engine API designed to make blockchain data easily accessible and AI-ready. Here’s a breakdown of how it enables Sharpe Search to function effectively: 1. LLM-Optimized Query Processing Sharpe Search leverages Hive Intelligence's optimized responses for large language models. This ensures that AI agents can process blockchain data in a structured format, delivering precise answers to complex user queries. 2. Natural Language Interaction Forget the need for technical knowledge. Sharpe Search supports natural language queries, making it as simple as typing: - “What tokens are in my wallet? Am I eligible for any airdrop I haven't claimed yet?” - “Check me my last 100 transactions, tell me if I interacted with any protocol with recent hacks” - “Track my wallet activity over the past month, suggest optimised portfolio based on best stable yields available” 3. Real-Time Insights Across Multi-Chains Using Hive Intelligence, Sharpe Search connects to over 20 chains and 5000+ Protocols. This real-time access ensures that the AI agent provides up-to-date and actionable insights, no matter how dynamic the blockchain environment. 4. Unified API Access Sharpe Search consolidates fragmented blockchain data through Hive’s unified API. Instead of dealing with multiple integrations, Sharpe Search uses a single access point to aggregate and query data, reducing complexity for both users and developers. Technical Depth: The AI Agent Advantage Sharpe Search's design philosophy revolves around the principle of creating an intuitive, AI-driven experience. Here’s what makes its technology stand out: Data Indexing and Aggregation: Hive Intelligence employs advanced indexing algorithms to aggregate data from multiple chains. This ensures that Sharpe Search can retrieve information within milliseconds, even when querying vast datasets. Dynamic Updates: Blockchain data is volatile. Sharpe Search processes dynamic updates in real time, enabling users to act on the most recent metrics, transactions, and balances without delays. Contextual Understanding: The AI agent parses natural language queries and contextualizes them to blockchain-specific scenarios. For instance, when querying “Show portfolio details,” Sharpe Search understands the underlying requirements—fetching wallet holdings, token values, and current positions. Hive Intelligence: The Backbone of Sharpe Search While Sharpe Search takes center stage, Hive Intelligence provides the critical infrastructure to make it all possible. Its LLM-ready responses and multi-chain support ensure that Sharpe Search operates at the forefront of blockchain data accessibility. By launching Hive Intelligence through Sharpe Launchpad, Sharpe reinforces its commitment to supporting innovation in the blockchain space. Hive’s infrastructure not only powers Sharpe Search but also lays the groundwork for future AI agents to thrive in the ecosystem. What’s Next for Sharpe Search? Currently in invite-only access, Sharpe Search is preparing for a broader public release. Future updates will include: - Expanded Blockchain Coverage: More chains and protocols will be added. - Enhanced Query Flexibility: Even more advanced natural language capabilities. Stay tuned for the public launch and get ready to explore crypto like never before!

Sharpe AI

263,278 次观看 • 1 年前

how to use firecrawl to give your AI eyes and actually build startups that outperform 99% of apps: 1. your AI is smart but blind. it can't go to a website, read a page, or grab data on its own. firecrawl fixes that. you put in a URL. you get back clean markdown, structured JSON, screenshots. feed it to any model. 2. three lines of code. that's it. no proxies. no anti-bot detection. no custom scrapers that break when a site changes. one API call. clean data back in seconds. works on 98%+ of sites. 3. firecrawl has six core capabilities: scrape a single page. crawl an entire site. map all URLs on a domain. search google and return full content. an agent endpoint where you describe what you want and it goes and finds it. and a browser sandbox where AI controls a real browser like filling forms, clicking buttons, handles logins. 4. the agent endpoint is wild. you can say "find all of YC's winter 24 dev tool companies and their founders and emails" and get back structured data. or "compare pricing tiers across stripe, square, and paypal" and get a side-by-side table. 5. the browser sandbox lets your AI stay logged in across sessions, navigate pagination, watch live as it browses. this is computer use without building the infrastructure yourself. 6. think of it in layers. every builder needs: an agent harness (claude code, cursor, codex), a search layer (perplexity, exa), a web data layer (firecrawl), an ops brain (obsidian, notion), and an outbound stack. the web data layer is the one most people are sleeping on. 7. this is the AWS moment for web data. in 2006 building a web app meant buying servers and managing racks. AWS said one API call, use our servers. some of the biggest companies of the last decade were built on that. firecrawl is doing the same thing for web data in 2026. 8. the framework i'd use for coming up with startup ideas building with clean data: take a massive horizontal platform. rebuild it for one niche using firecrawl. the vertical version always wins because people want specific, not generic. price for outcome. 9. a year ago firecrawl posted a job listing that said "please only apply if you're an AI agent." content creator agents. customer support agents. junior dev agents. it looked weird. it was a signal for where this is all going. the people who understand how to get clean web data, wrap it around an LLM, and package it as a product are the the ones with a 12-month head start. i use Firecrawl with Idea Browser . once you see what's possible with structured web data, you can't unsee it. episode is live on The Startup Ideas Podcast (SIP) 🧃 (full breakdown there) i tried to explain this as clear as possible for even the non technical. send it to a builder friend. watch
27:27

how to use firecrawl to give your AI eyes and actually build startups that outperform 99% of apps: 1. your AI is smart but blind. it can't go to a website, read a page, or grab data on its own. firecrawl fixes that. you put in a URL. you get back clean markdown, structured JSON, screenshots. feed it to any model. 2. three lines of code. that's it. no proxies. no anti-bot detection. no custom scrapers that break when a site changes. one API call. clean data back in seconds. works on 98%+ of sites. 3. firecrawl has six core capabilities: scrape a single page. crawl an entire site. map all URLs on a domain. search google and return full content. an agent endpoint where you describe what you want and it goes and finds it. and a browser sandbox where AI controls a real browser like filling forms, clicking buttons, handles logins. 4. the agent endpoint is wild. you can say "find all of YC's winter 24 dev tool companies and their founders and emails" and get back structured data. or "compare pricing tiers across stripe, square, and paypal" and get a side-by-side table. 5. the browser sandbox lets your AI stay logged in across sessions, navigate pagination, watch live as it browses. this is computer use without building the infrastructure yourself. 6. think of it in layers. every builder needs: an agent harness (claude code, cursor, codex), a search layer (perplexity, exa), a web data layer (firecrawl), an ops brain (obsidian, notion), and an outbound stack. the web data layer is the one most people are sleeping on. 7. this is the AWS moment for web data. in 2006 building a web app meant buying servers and managing racks. AWS said one API call, use our servers. some of the biggest companies of the last decade were built on that. firecrawl is doing the same thing for web data in 2026. 8. the framework i'd use for coming up with startup ideas building with clean data: take a massive horizontal platform. rebuild it for one niche using firecrawl. the vertical version always wins because people want specific, not generic. price for outcome. 9. a year ago firecrawl posted a job listing that said "please only apply if you're an AI agent." content creator agents. customer support agents. junior dev agents. it looked weird. it was a signal for where this is all going. the people who understand how to get clean web data, wrap it around an LLM, and package it as a product are the the ones with a 12-month head start. i use Firecrawl with Idea Browser . once you see what's possible with structured web data, you can't unsee it. episode is live on The Startup Ideas Podcast (SIP) 🧃 (full breakdown there) i tried to explain this as clear as possible for even the non technical. send it to a builder friend. watch

GREG ISENBERG

135,017 次观看 • 4 个月前

How to setup a multi agent system? Bookmark it 📂 "The Trading Floor" Multi-Agent Market Analysis Council to analyze a stock ticker Z.ai GLM-4.7 🤝 OpenCode Agent framework: CrewAI How it works? 1. User enters a stock ticker to analyze 2. 5 AI agents wake up, each with distinct expertise: - Quant Analyst — technical indicators & price patterns - Sentiment Scout — market mood & crowd psychology - Macro Strategist — sector dynamics & economic context - Risk Manager — volatility, drawdowns & position sizing - Portfolio Chief — synthesizes all perspectives 3. Agents analyze independently using real market data 4. They debate, challenge assumptions, and identify disagreements 5. Portfolio Chief resolves conflicts and delivers a consensus recommendation 6. Final output: buy/hold/sell rating with confidence level, position size, and key risks How to built The Trading Floor? 1. Chose CrewAI as the agent framework — handles multi-agent orchestration out of the box 2. Defined 5 agents with distinct roles, goals, and backstories in Python 3. Built custom tools wrapping yfinance for real market data (prices, indicators, volatility) 4. Configured sequential workflow — specialists analyze first, Portfolio Chief synthesizes last 5. Set up FastAPI backend with SSE to stream agent thoughts in real-time 6. Built Next.js frontend to visualize the "board of directors" deliberating live 7. One environment variable (MODEL=openai/gpt-5.2) powers all agents 8. Generated unique agent icons with AI image tools Total cost: $0 for the framework, pay only for LLM API calls Tech stack: - GLM-4.7 with opencode to build the app - CrewAI (open source) for agent orchestration - GPT-5.2 powering each agent - FastAPI + SSE for real-time streaming - Next.js frontend showing live agent deliberations
0:40

How to setup a multi agent system? Bookmark it 📂 "The Trading Floor" Multi-Agent Market Analysis Council to analyze a stock ticker Z.ai GLM-4.7 🤝 OpenCode Agent framework: CrewAI How it works? 1. User enters a stock ticker to analyze 2. 5 AI agents wake up, each with distinct expertise: - Quant Analyst — technical indicators & price patterns - Sentiment Scout — market mood & crowd psychology - Macro Strategist — sector dynamics & economic context - Risk Manager — volatility, drawdowns & position sizing - Portfolio Chief — synthesizes all perspectives 3. Agents analyze independently using real market data 4. They debate, challenge assumptions, and identify disagreements 5. Portfolio Chief resolves conflicts and delivers a consensus recommendation 6. Final output: buy/hold/sell rating with confidence level, position size, and key risks How to built The Trading Floor? 1. Chose CrewAI as the agent framework — handles multi-agent orchestration out of the box 2. Defined 5 agents with distinct roles, goals, and backstories in Python 3. Built custom tools wrapping yfinance for real market data (prices, indicators, volatility) 4. Configured sequential workflow — specialists analyze first, Portfolio Chief synthesizes last 5. Set up FastAPI backend with SSE to stream agent thoughts in real-time 6. Built Next.js frontend to visualize the "board of directors" deliberating live 7. One environment variable (MODEL=openai/gpt-5.2) powers all agents 8. Generated unique agent icons with AI image tools Total cost: $0 for the framework, pay only for LLM API calls Tech stack: - GLM-4.7 with opencode to build the app - CrewAI (open source) for agent orchestration - GPT-5.2 powering each agent - FastAPI + SSE for real-time streaming - Next.js frontend showing live agent deliberations

CloudAI-X

58,189 次观看 • 7 个月前

over the labor day weekend i built out an app to analyze my twitter mutuals, using my Hyperspace agentic-os prototype. this is likely the most powerful agentic-os setup every assembled. this goes beyond a single model, or agent, or data source or a single smart CLI tool. this goes beyond hardware, or cloud or distributed networks. it is all of it, combined. the LLM OS is here, Andrej Karpathy. and this combined environment enabled me to build an app I craved for all these years, without writing a single line of code, saving me 150 hours of development work effort. i now have a deep understanding of my twitter mutuals here - and to construct the app - my agentic-os utilized multiple models including Grok Code, GPT-OSS-120B running locally, Qwen2.5 Coder and Google's Nano Banana, multiple orchestrators include Claude Code, 24 AI agents, several MCPs, and specific data from my Twitter archive which I plugged in and other data which it researched, analyzed and aggregated into helping me understand my network here. consider again: the absolute best work from all the top AI firms - all of it, in one actually-useful-integrated-environment. RT if you want this as well. any feature requests ? eg show when a mutual is in the same city in the future.. will DM with access.. this is madness, but it is here 🤖🔥
1:27

over the labor day weekend i built out an app to analyze my twitter mutuals, using my Hyperspace agentic-os prototype. this is likely the most powerful agentic-os setup every assembled. this goes beyond a single model, or agent, or data source or a single smart CLI tool. this goes beyond hardware, or cloud or distributed networks. it is all of it, combined. the LLM OS is here, Andrej Karpathy. and this combined environment enabled me to build an app I craved for all these years, without writing a single line of code, saving me 150 hours of development work effort. i now have a deep understanding of my twitter mutuals here - and to construct the app - my agentic-os utilized multiple models including Grok Code, GPT-OSS-120B running locally, Qwen2.5 Coder and Google's Nano Banana, multiple orchestrators include Claude Code, 24 AI agents, several MCPs, and specific data from my Twitter archive which I plugged in and other data which it researched, analyzed and aggregated into helping me understand my network here. consider again: the absolute best work from all the top AI firms - all of it, in one actually-useful-integrated-environment. RT if you want this as well. any feature requests ? eg show when a mutual is in the same city in the future.. will DM with access.. this is madness, but it is here 🤖🔥

Varun

22,204 次观看 • 11 个月前

New course: MCP: Build Rich-Context AI Apps with Anthropic. Learn to build AI apps that access tools, data, and prompts using the Model Context Protocol in this short course, created in partnership with Anthropic Anthropic and taught by Elie Schoppik Elie Schoppik, its Head of Technical Education. Connecting AI applications to external systems that bring rich context to LLM-based applications has often meant writing custom integrations for each use case. MCP is an open protocol that standardizes how LLMs access tools, data, and prompts from external sources, and simplifies how you provide context to your LLM-based applications. For example, you can provide context via third-party tools that let your LLM make API calls to search the web, access data from local docs, retrieve code from a GitHub repo, and so on. MCP, developed by Anthropic, is based on a client-server architecture that defines the communication details between an MCP client, hosted inside the AI application, and an MCP server that exposes tools, resources, and prompt templates. The server can be a subprocess launched by the client that runs locally or an independent process running remotely. In this hands-on course, you'll learn the core architecture behind MCP. You’ll create an MCP-compatible chatbot, build and deploy an MCP server, and connect the chatbot to your MCP server and other open-source servers. Here’s what you’ll do: - Understand why MCP makes AI development less fragmented and standardizes connections between AI applications and external data sources - Learn the core components of the client-server architecture of MCP and the underlying communication mechanism - Build a chatbot with custom tools for searching academic papers, and transform it into an MCP-compatible application - Build a local MCP server that exposes tools, resources, and prompt templates using FastMCP, and test it using MCP Inspector - Create an MCP client inside your chatbot to dynamically connect to your server - Connect your chatbot to reference servers built by Anthropic’s MCP team, such as filesystem, which implements filesystem operations, and fetch, which extracts contents from the web as markdown - Configure Claude Desktop to connect to your server and others, and explore how it abstracts away the low-level logic of MCP clients - Deploy your MCP server remotely and test it with the Inspector or other MCP-compatible applications - Learn about the roadmap for future MCP development, such as multi-agent architecture, MCP registry API, server discovery, authorization, and authentication MCP is an exciting and important technology that lets you build rich-context AI applications that connect to a growing ecosystem of MCP servers, with minimal integration work. Please sign up here!
2:43

New course: MCP: Build Rich-Context AI Apps with Anthropic. Learn to build AI apps that access tools, data, and prompts using the Model Context Protocol in this short course, created in partnership with Anthropic Anthropic and taught by Elie Schoppik Elie Schoppik, its Head of Technical Education. Connecting AI applications to external systems that bring rich context to LLM-based applications has often meant writing custom integrations for each use case. MCP is an open protocol that standardizes how LLMs access tools, data, and prompts from external sources, and simplifies how you provide context to your LLM-based applications. For example, you can provide context via third-party tools that let your LLM make API calls to search the web, access data from local docs, retrieve code from a GitHub repo, and so on. MCP, developed by Anthropic, is based on a client-server architecture that defines the communication details between an MCP client, hosted inside the AI application, and an MCP server that exposes tools, resources, and prompt templates. The server can be a subprocess launched by the client that runs locally or an independent process running remotely. In this hands-on course, you'll learn the core architecture behind MCP. You’ll create an MCP-compatible chatbot, build and deploy an MCP server, and connect the chatbot to your MCP server and other open-source servers. Here’s what you’ll do: - Understand why MCP makes AI development less fragmented and standardizes connections between AI applications and external data sources - Learn the core components of the client-server architecture of MCP and the underlying communication mechanism - Build a chatbot with custom tools for searching academic papers, and transform it into an MCP-compatible application - Build a local MCP server that exposes tools, resources, and prompt templates using FastMCP, and test it using MCP Inspector - Create an MCP client inside your chatbot to dynamically connect to your server - Connect your chatbot to reference servers built by Anthropic’s MCP team, such as filesystem, which implements filesystem operations, and fetch, which extracts contents from the web as markdown - Configure Claude Desktop to connect to your server and others, and explore how it abstracts away the low-level logic of MCP clients - Deploy your MCP server remotely and test it with the Inspector or other MCP-compatible applications - Learn about the roadmap for future MCP development, such as multi-agent architecture, MCP registry API, server discovery, authorization, and authentication MCP is an exciting and important technology that lets you build rich-context AI applications that connect to a growing ecosystem of MCP servers, with minimal integration work. Please sign up here!

Andrew Ng

142,010 次观看 • 1 年前

We’re excited to announce AI SDR-Kit, a comprehensive suite of app integrations and starter templates to build highly customizable AI sales agents. As a developer, I only realized how tough it can be to crack sales after founding my startup. It’s a lot of grunt work, prospecting, qualifying, outreaching and whatnot. And the complexity only goes up as you grow. But it doesn’t have to. AI agents can effectively optimize many of these routine tasks; for instance, they can find leads from Apollo, enrich their data using People Data Labs, manage them in Salesforce, send targeted emails via SendGrid, and schedule meetings in Calendly without any manual involvement. But the biggest challenge is building these app integrations for agents. A single Salesforce integration may take 100s of engineering hours, let alone other integrations across CRMs, email platforms, and data enrichment tools. Considering this, we Composio built AI SDR-Kit to enable developers to build full-fledged sales automation agents. You will get • Over 60 app integrations optimized for SDR/BDR agents. • Seamless handling of complex auth flows (OAuth, API Key, Basic). • Compatibility with 15+ Agentic Frameworks, including LangChain, LlamaIndex 🦙, CrewAI, Letta etc. • Self-hosting option for greater control. • Enterprise-ready with SOC 2 Type 2 compliance, SSO, and RBAC support. Here's a brief list of popular integrations by category that you get with SDR-Kit: • CRM: Salesforce, HubSpot, Attio, and more. • Contact Data: Apollo, ZoomInfo, People Data Labs, etc. • Email deliverability: Gmail, Outlook, Mailchimp, SendGrid, Klaviyo. • Comms & Collabs: Slack, Discord, Intercom & Zoom. • Social Media: LinkedIn, Facebook, Twitter & Reddit. • Meeting: Google Calendar, Calendly, Cal dot com, etc. • … and many more. All the integrations have been improved, keeping agents' real-world readiness in mind. We’ve also made it straightforward to connect these apps with your agents in a few lines of code. Making it easy to create AI sales agents. This is ideal for developers and companies building automated sales solutions - whether for internal SalesOps or AI SDR services. Checkout AI SDR-Kit now👇
1:21

We’re excited to announce AI SDR-Kit, a comprehensive suite of app integrations and starter templates to build highly customizable AI sales agents. As a developer, I only realized how tough it can be to crack sales after founding my startup. It’s a lot of grunt work, prospecting, qualifying, outreaching and whatnot. And the complexity only goes up as you grow. But it doesn’t have to. AI agents can effectively optimize many of these routine tasks; for instance, they can find leads from Apollo, enrich their data using People Data Labs, manage them in Salesforce, send targeted emails via SendGrid, and schedule meetings in Calendly without any manual involvement. But the biggest challenge is building these app integrations for agents. A single Salesforce integration may take 100s of engineering hours, let alone other integrations across CRMs, email platforms, and data enrichment tools. Considering this, we Composio built AI SDR-Kit to enable developers to build full-fledged sales automation agents. You will get • Over 60 app integrations optimized for SDR/BDR agents. • Seamless handling of complex auth flows (OAuth, API Key, Basic). • Compatibility with 15+ Agentic Frameworks, including LangChain, LlamaIndex 🦙, CrewAI, Letta etc. • Self-hosting option for greater control. • Enterprise-ready with SOC 2 Type 2 compliance, SSO, and RBAC support. Here's a brief list of popular integrations by category that you get with SDR-Kit: • CRM: Salesforce, HubSpot, Attio, and more. • Contact Data: Apollo, ZoomInfo, People Data Labs, etc. • Email deliverability: Gmail, Outlook, Mailchimp, SendGrid, Klaviyo. • Comms & Collabs: Slack, Discord, Intercom & Zoom. • Social Media: LinkedIn, Facebook, Twitter & Reddit. • Meeting: Google Calendar, Calendly, Cal dot com, etc. • … and many more. All the integrations have been improved, keeping agents' real-world readiness in mind. We’ve also made it straightforward to connect these apps with your agents in a few lines of code. Making it easy to create AI sales agents. This is ideal for developers and companies building automated sales solutions - whether for internal SalesOps or AI SDR services. Checkout AI SDR-Kit now👇

Karan Vaidya

54,217 次观看 • 1 年前

THIS GUY OPENED A TINY PC AND SHOWED HOW A $1,800 GMKTEC EVO-X2 TURNS INTO A LOCAL AI SERVER not a cloud dashboard. not another $200/mo subscription. just one box on a desk, upgraded with storage, cleaner airflow, faster networking, and pointed at real local models. the numbers are what make it interesting. Claude Code Max is $200/mo, ChatGPT Pro is $200/mo, Cursor is another $20/mo. that is $5,040/year just to rent tools most people use for coding, research, summaries, and agents. EVO-X2 flips the setup because it has 128GB unified memory, Ryzen AI Max+ 395, and up to 110GB usable as VRAM on Linux. enough room for Qwen3-Coder 30B, Llama 70B, and 200B-class local models when speed is not the bottleneck. so the workflow becomes simple: buy the box once, install Linux, run the model locally, connect the CLI, and let the machine handle jobs from your own network instead of sending every request to the cloud. this is where the real edge starts. overnight code audits, private document search, local RAG, scraping agents, transcription, file cleanup, all running while the token meter stays at zero. frontier models still win for the hardest prompts. but for daily automation, coding, research, and private workflows, local AI is already moving from nerd setup to obvious business math. the next flex won’t be “which AI plan do you pay for?” it’ll be “what’s running under your desk?”
1:09

THIS GUY OPENED A TINY PC AND SHOWED HOW A $1,800 GMKTEC EVO-X2 TURNS INTO A LOCAL AI SERVER not a cloud dashboard. not another $200/mo subscription. just one box on a desk, upgraded with storage, cleaner airflow, faster networking, and pointed at real local models. the numbers are what make it interesting. Claude Code Max is $200/mo, ChatGPT Pro is $200/mo, Cursor is another $20/mo. that is $5,040/year just to rent tools most people use for coding, research, summaries, and agents. EVO-X2 flips the setup because it has 128GB unified memory, Ryzen AI Max+ 395, and up to 110GB usable as VRAM on Linux. enough room for Qwen3-Coder 30B, Llama 70B, and 200B-class local models when speed is not the bottleneck. so the workflow becomes simple: buy the box once, install Linux, run the model locally, connect the CLI, and let the machine handle jobs from your own network instead of sending every request to the cloud. this is where the real edge starts. overnight code audits, private document search, local RAG, scraping agents, transcription, file cleanup, all running while the token meter stays at zero. frontier models still win for the hardest prompts. but for daily automation, coding, research, and private workflows, local AI is already moving from nerd setup to obvious business math. the next flex won’t be “which AI plan do you pay for?” it’ll be “what’s running under your desk?”

Gipp 🦅

13,271 次观看 • 1 个月前

how to set up hermes agent step by step. built-in memory, 40+ tools, works on your phone, and what to think of hermes vs openclaw: 1. hermes is a personal AI agent that runs in your terminal. think of it like open claw but with built-in memory, 40+ tools out of the box, and 90% cheaper token costs. you install it with one command. 2. the 3 problems with open claw that hermes solves: no memory (you keep repeating yourself), constant gateway restarts, and zero visibility into what you're spending on tokens. 3. hermes remembers everything. every completed task gets saved to memory. it searches through past logs to find solutions. over time it literally gets smarter at your specific workflows. 4. connect it to open router. you see exact costs per model per task. free models rotate weekly. one founder went from $130 every five days on open claw to $10 on hermes. same output. 5. it comes preloaded with skills. apple notes, imessage, find my, browser, web search, image generation, cron jobs. no hunting for plugins. 6. connect it to obsidian so it reads your entire vault. connect it to gstack for your dev environment. create custom skills for your specific workflows. 7. the biggest money saver: have it write code once for recurring tasks. then it runs without burning tokens every time. stop paying an LLM to do the same scrape or report daily. 8. run it on android via telegram. name your agents. talk to them like coworkers. in this episode imran shows you how to set this up. 9. you can run it bare metal, in docker, or serverless on modal. pick your risk level. i begged imran to come on The Startup Ideas Podcast (SIP) 🧃 and walk through the full installation live. he made it impossibly clear. if you've heard of Hermes Agent and want the clearest explanation of how to get set up like a pro let me know what you want me to cover on the next ep this is the best personal agent setup video on the internet right now. watch
37:00

how to set up hermes agent step by step. built-in memory, 40+ tools, works on your phone, and what to think of hermes vs openclaw: 1. hermes is a personal AI agent that runs in your terminal. think of it like open claw but with built-in memory, 40+ tools out of the box, and 90% cheaper token costs. you install it with one command. 2. the 3 problems with open claw that hermes solves: no memory (you keep repeating yourself), constant gateway restarts, and zero visibility into what you're spending on tokens. 3. hermes remembers everything. every completed task gets saved to memory. it searches through past logs to find solutions. over time it literally gets smarter at your specific workflows. 4. connect it to open router. you see exact costs per model per task. free models rotate weekly. one founder went from $130 every five days on open claw to $10 on hermes. same output. 5. it comes preloaded with skills. apple notes, imessage, find my, browser, web search, image generation, cron jobs. no hunting for plugins. 6. connect it to obsidian so it reads your entire vault. connect it to gstack for your dev environment. create custom skills for your specific workflows. 7. the biggest money saver: have it write code once for recurring tasks. then it runs without burning tokens every time. stop paying an LLM to do the same scrape or report daily. 8. run it on android via telegram. name your agents. talk to them like coworkers. in this episode imran shows you how to set this up. 9. you can run it bare metal, in docker, or serverless on modal. pick your risk level. i begged imran to come on The Startup Ideas Podcast (SIP) 🧃 and walk through the full installation live. he made it impossibly clear. if you've heard of Hermes Agent and want the clearest explanation of how to get set up like a pro let me know what you want me to cover on the next ep this is the best personal agent setup video on the internet right now. watch

GREG ISENBERG

618,145 次观看 • 3 个月前

Maple is preparing for the release of a co-working agent. You install it locally and it works with your files, whether it's office work or building websites and apps. It's a turnkey solution, as easy as Claude Code, that keeps your data secure and private, no data sharing with closed AI labs. This is THE sovereign AI app for individuals and businesses who want powerful AI while retaining ownership of their information. Why build an agent into the Maple app when other agents already exist? Easy, we want to give you control over your work. We don't have a business plan that incorporates making money off our users' data. In the age of AI, your information, whether it's personal or company trade secrets, is the single thing that differentiates you from everyone else. We all have access to AI that can build a professional website for selling shoes. But your strategy and network for how you sell shoes should not be shared with your competitors. Sovereignty is the path to protecting what makes you, you. Maple sits at the intersection of Usability and Sovereignty. Maple gives you the best tools that are both easy to use and maintain your data sovereignty. Sovereign for one, sovereign for all. It has been a journey to get here. We brought to market the very first personal chatbot with end-to-end encryption using TEEs in late 2024. Prior to that there were proofs of concept but no full product offerings. Every other AI chat product on the market handled your data in plain text, either selling you a service to get your data or asking you to trust that they won't snoop on you. Quickly people found Maple and latched onto its open-source code and verifiable encryption. We didn't stop there. You may remember earlier this year we teased a product called "Maple Agent" and opened up a waiting list. That product is a mobile app that acts as your AI "friend", maintaining one long continuous chat, and getting to know you over time. I dislike using the word "friend" there, but it's the best way to convey the UX in a few words. AI is a tool, always has been, always will be. Any kind of friendly personality on top is just synthetic. In our testing, the UX of Maple Agent is really powerful for what it does. Think about the many short AI chats you have in your favorite app, whether it's looking up a historical fact or asking advice about a topic. With Maple Agent, those all go away in favor of the long-running chat with the friendly agent. It's like you have your own personal assistant who knows you so well and can look up anything for you. When I ask AI certain questions, I want to ask an expert who already understands my situation so I'm not repeating myself for the 100th time. That's the amazing value the personal agent brings to the table. We still see great utility for a personal agent like the "Maple Agent". Thousands of people on the waiting list, hoping to get their hands on it, agree that the concept is worth exploring and trying out. We were constrained in launching it due to a few circumstances, one of them being access to the scale of compute needed to power it. We have a clear path laid out for how to get there, but today is not the day to execute on that. It will be in the near future. Instead we have a different agent ready to go that we think is also incredible. We now have an agentic harness inside of the Maple Research app. This thing is a powerhouse. It even builds and publishes its own software releases. The agent in Maple Research works with your local filesystem, speaks to the largest open models running in TEEs, utilizes local models for certain tasks, is compatible with MCP tools, has an API for connecting to anything you need, and also supports the ACP protocol, which means it can be extended in the future to speak to other tools like Claude Code, Codex, and local models running on your own hardware. A big unlock for us was the Goose Development Kit, which powers the core of our agent harness. More on that to come as we publish articles and documentation later about the agent. The agent inside Maple Research doesn't have a name. At least not yet, not sure if it ever will. For now we call it "Chat Mode" and "Agent Mode". Think of this as the workhorse, the truck, the heavy lifter. Our other "Agent", the phone app, is your sidekick in your pocket, ready to help with quick things and ongoing conversations about life. I am incredibly excited about the Maple Research Agent. While I'm already seeing great results using it for internal work items, I'm especially thrilled about the personal health and wellness work it's doing for me. I know there are plenty of apps out there for compiling wellness data, but I'm having it build a tool tailored specifically for what I need, without the extra fluff. And none of my health data is being donated to the closed AI labs or sent to advertisers. I know that the AI logic is not being silently adjusted to fit the whims of a large corporation that has paid for product placement. It's me, state of the art AI, and my data. That's how I want it. Maple's new agent makes that possible. We can't wait for you to try it out. If you want early access, comment here, email us, reach out in some way. To those on the other agent waitlist, you're already in the queue. Thanks for reading this lengthy update. :)
0:49

Maple is preparing for the release of a co-working agent. You install it locally and it works with your files, whether it's office work or building websites and apps. It's a turnkey solution, as easy as Claude Code, that keeps your data secure and private, no data sharing with closed AI labs. This is THE sovereign AI app for individuals and businesses who want powerful AI while retaining ownership of their information. Why build an agent into the Maple app when other agents already exist? Easy, we want to give you control over your work. We don't have a business plan that incorporates making money off our users' data. In the age of AI, your information, whether it's personal or company trade secrets, is the single thing that differentiates you from everyone else. We all have access to AI that can build a professional website for selling shoes. But your strategy and network for how you sell shoes should not be shared with your competitors. Sovereignty is the path to protecting what makes you, you. Maple sits at the intersection of Usability and Sovereignty. Maple gives you the best tools that are both easy to use and maintain your data sovereignty. Sovereign for one, sovereign for all. It has been a journey to get here. We brought to market the very first personal chatbot with end-to-end encryption using TEEs in late 2024. Prior to that there were proofs of concept but no full product offerings. Every other AI chat product on the market handled your data in plain text, either selling you a service to get your data or asking you to trust that they won't snoop on you. Quickly people found Maple and latched onto its open-source code and verifiable encryption. We didn't stop there. You may remember earlier this year we teased a product called "Maple Agent" and opened up a waiting list. That product is a mobile app that acts as your AI "friend", maintaining one long continuous chat, and getting to know you over time. I dislike using the word "friend" there, but it's the best way to convey the UX in a few words. AI is a tool, always has been, always will be. Any kind of friendly personality on top is just synthetic. In our testing, the UX of Maple Agent is really powerful for what it does. Think about the many short AI chats you have in your favorite app, whether it's looking up a historical fact or asking advice about a topic. With Maple Agent, those all go away in favor of the long-running chat with the friendly agent. It's like you have your own personal assistant who knows you so well and can look up anything for you. When I ask AI certain questions, I want to ask an expert who already understands my situation so I'm not repeating myself for the 100th time. That's the amazing value the personal agent brings to the table. We still see great utility for a personal agent like the "Maple Agent". Thousands of people on the waiting list, hoping to get their hands on it, agree that the concept is worth exploring and trying out. We were constrained in launching it due to a few circumstances, one of them being access to the scale of compute needed to power it. We have a clear path laid out for how to get there, but today is not the day to execute on that. It will be in the near future. Instead we have a different agent ready to go that we think is also incredible. We now have an agentic harness inside of the Maple Research app. This thing is a powerhouse. It even builds and publishes its own software releases. The agent in Maple Research works with your local filesystem, speaks to the largest open models running in TEEs, utilizes local models for certain tasks, is compatible with MCP tools, has an API for connecting to anything you need, and also supports the ACP protocol, which means it can be extended in the future to speak to other tools like Claude Code, Codex, and local models running on your own hardware. A big unlock for us was the Goose Development Kit, which powers the core of our agent harness. More on that to come as we publish articles and documentation later about the agent. The agent inside Maple Research doesn't have a name. At least not yet, not sure if it ever will. For now we call it "Chat Mode" and "Agent Mode". Think of this as the workhorse, the truck, the heavy lifter. Our other "Agent", the phone app, is your sidekick in your pocket, ready to help with quick things and ongoing conversations about life. I am incredibly excited about the Maple Research Agent. While I'm already seeing great results using it for internal work items, I'm especially thrilled about the personal health and wellness work it's doing for me. I know there are plenty of apps out there for compiling wellness data, but I'm having it build a tool tailored specifically for what I need, without the extra fluff. And none of my health data is being donated to the closed AI labs or sent to advertisers. I know that the AI logic is not being silently adjusted to fit the whims of a large corporation that has paid for product placement. It's me, state of the art AI, and my data. That's how I want it. Maple's new agent makes that possible. We can't wait for you to try it out. If you want early access, comment here, email us, reach out in some way. To those on the other agent waitlist, you're already in the queue. Thanks for reading this lengthy update. :)

Mark

30,227 次观看 • 15 天前

🤖🔬 Can AI actually do science end-to-end? 🧠📈 And how would we know when it matches, or surpasses, humans? ⚡🧪 AI is rapidly automating scientific discovery, but benchmarking full-cycle discovery, from 💡 ideation → 🧑‍💻 execution → 📊 conclusions, remains unsolved: 🧐🧐🧐 ❌🛠️ Open-ended discovery → manual validation (costly, unscalable) ❌📏 Metric-driven benchmarks (e.g., MLE-Bench) → convenient but narrow (is higher accuracy really enough?) ❌🤖⚖️ LLM-as-judge → useful, but fundamentally risky if used alone 🔥🚀 Introducing FIRE-Bench🔥: Fullcycle Insight Rediscovery Evaluation 👉🌐 📚✨ A benchmark that turns fresh, human-verified insights from recent 🏆 NeurIPS / ICLR / ICML papers into masked, end-to-end discovery challenges 🧩 🌍🔐 Constrained open-ended discovery–backed by ground truth. 📌 Key takeaways: 1⃣ 📖🧱 Reference-based evaluation still matters: constrained LLM judging helps, but human-grounded references remain essential until agents can consistently match human conclusions 2⃣ 🏆🧠 Expert-validated ground truth: all tasks come from recent NeurIPS / ICLR / ICML papers, with contamination carefully controlled 3⃣ 🔁🎭 Rediscovery, not reproduction: original 🧪 methods, 📊 experiments, 💻 implementations, and 📈 analyses are fully masked to create real discovery challenges 🔑 Key empirical findings: 💡 The "Science Gap" is Real: Even the best setup (Claude Code + Sonnet-4) caps out at an F1 score of 46.7. On hard tasks, agents struggle to break 30 💡 Success is a "Lottery": Performance has incredibly high variance. Reliability is a major unsolved issue. 💡 Coding is no longer the bottleneck; high-level reasoning and analysis are: ~74% of errors stem from flawed planning, not coding ⚙️ How it works: 🔹 Research-Problem Trees: We parse papers into trees (from broad roots to concrete leaves). This allows us to select intermediate nodes that perfectly balance open-ended exploration with verifiable ground truth. 🔹 Claim-Level Evaluation: We match AI conclusions against human conclusions using granular claim decomposition (F1 score). 🔹 Creativity Check: We score false positives to see if agents are finding novel truths (Spoiler🚨: they aren’t creative yet). 🔹 New Diagnostic Taxonomy: failures traced across four stages: 🧠 Planning → 🛠️ Implementation → ▶️ Execution → 🧾 Conclusion 🔹 Additional Analyses: cost efficiency, contamination checks, and more. 👀 The Future: 🚀 Live-FIRE-Bench: a live, continuously updated FIRE-Bench to track real-time progress on the latest research (Newest LLMs should be benchmarked with the newest research) 🚀 Stronger scaffolding (search + planning + coding) 🧠🧰 and converting FIRE-Bench into interactive environments for training research agents 🚀 Toward real creativity: We want better systems that can produce genuinely novel conclusions toward creativity 🎨⏳ 🚀 Better systems 🧠✨ and better benchmarks 📏 must co-evolve 🔄 over time 📜🎥 Paper, video, demo, and research trees: 👉🌐 #AI 🤖 #MachineLearning 📚 #AI4Science 🔬 #LLMs 🧠 #Research 🧪 #AgenticAI 🚀 #FireBench 🔥
0:42

🤖🔬 Can AI actually do science end-to-end? 🧠📈 And how would we know when it matches, or surpasses, humans? ⚡🧪 AI is rapidly automating scientific discovery, but benchmarking full-cycle discovery, from 💡 ideation → 🧑‍💻 execution → 📊 conclusions, remains unsolved: 🧐🧐🧐 ❌🛠️ Open-ended discovery → manual validation (costly, unscalable) ❌📏 Metric-driven benchmarks (e.g., MLE-Bench) → convenient but narrow (is higher accuracy really enough?) ❌🤖⚖️ LLM-as-judge → useful, but fundamentally risky if used alone 🔥🚀 Introducing FIRE-Bench🔥: Fullcycle Insight Rediscovery Evaluation 👉🌐 📚✨ A benchmark that turns fresh, human-verified insights from recent 🏆 NeurIPS / ICLR / ICML papers into masked, end-to-end discovery challenges 🧩 🌍🔐 Constrained open-ended discovery–backed by ground truth. 📌 Key takeaways: 1⃣ 📖🧱 Reference-based evaluation still matters: constrained LLM judging helps, but human-grounded references remain essential until agents can consistently match human conclusions 2⃣ 🏆🧠 Expert-validated ground truth: all tasks come from recent NeurIPS / ICLR / ICML papers, with contamination carefully controlled 3⃣ 🔁🎭 Rediscovery, not reproduction: original 🧪 methods, 📊 experiments, 💻 implementations, and 📈 analyses are fully masked to create real discovery challenges 🔑 Key empirical findings: 💡 The "Science Gap" is Real: Even the best setup (Claude Code + Sonnet-4) caps out at an F1 score of 46.7. On hard tasks, agents struggle to break 30 💡 Success is a "Lottery": Performance has incredibly high variance. Reliability is a major unsolved issue. 💡 Coding is no longer the bottleneck; high-level reasoning and analysis are: ~74% of errors stem from flawed planning, not coding ⚙️ How it works: 🔹 Research-Problem Trees: We parse papers into trees (from broad roots to concrete leaves). This allows us to select intermediate nodes that perfectly balance open-ended exploration with verifiable ground truth. 🔹 Claim-Level Evaluation: We match AI conclusions against human conclusions using granular claim decomposition (F1 score). 🔹 Creativity Check: We score false positives to see if agents are finding novel truths (Spoiler🚨: they aren’t creative yet). 🔹 New Diagnostic Taxonomy: failures traced across four stages: 🧠 Planning → 🛠️ Implementation → ▶️ Execution → 🧾 Conclusion 🔹 Additional Analyses: cost efficiency, contamination checks, and more. 👀 The Future: 🚀 Live-FIRE-Bench: a live, continuously updated FIRE-Bench to track real-time progress on the latest research (Newest LLMs should be benchmarked with the newest research) 🚀 Stronger scaffolding (search + planning + coding) 🧠🧰 and converting FIRE-Bench into interactive environments for training research agents 🚀 Toward real creativity: We want better systems that can produce genuinely novel conclusions toward creativity 🎨⏳ 🚀 Better systems 🧠✨ and better benchmarks 📏 must co-evolve 🔄 over time 📜🎥 Paper, video, demo, and research trees: 👉🌐 #AI 🤖 #MachineLearning 📚 #AI4Science 🔬 #LLMs 🧠 #Research 🧪 #AgenticAI 🚀 #FireBench 🔥

Zhen Wang

13,450 次观看 • 6 个月前

HERMES AGENT LEARNS FROM ITS OWN MISTAKES. UPDATES ITS MEMORY. CREATES ITS OWN SKILLS. NO CLOUD. EVERYTHING STORED LOCALLY. THIS IS HOW THE SELF-IMPROVING LOOP WORKS. most agents start from zero every session. Hermes carries forward what it learned. THREE MEMORY SYSTEMS: 1. PROCEDURAL MEMORY (how to act) stored in ~/.hermes/skills/ as SKILL.md files. when the agent repeats a complex workflow, it saves the procedure as a reusable skill. next time the same task comes up, it follows the skill instead of figuring it out again. you can also create skills explicitly: "create a skill called video-prep that captures how I format my video scripts. spoken english, define jargon inline, no em-dashes, close with a catchphrase." the agent writes the SKILL.md. available as a slash command from that moment. Hermes ships with 90+ skills. the number grows the longer you use it. 2. SEMANTIC MEMORY (durable facts about you) stored in ~/.hermes/memory/memory.md the agent scans conversations for facts worth remembering. preferences, habits, corrections, project details. real example from the video: agent tried to scrape a YouTube channel. URL was wrong. it failed. it updated memory.md with the correct URL pattern so it never makes the same mistake again. you can also save explicitly: "save to memory that my favorite testing framework is pytest" the agent updates memory.md immediately. this file loads into context on every session. the agent knows you better every week. 3. EPISODIC MEMORY (chat history) stored in ~/.hermes/state.db (local SQLite). every conversation. every tool call. every result. searchable with FTS5 full-text search. "search our past sessions. what was the first thing I ever said to you?" the agent queries state.db and finds it. over time, auxiliary models consolidate episodic memory into semantic memory. distilling recurring patterns into durable facts. THE SELF-IMPROVING LOOP: every agent run follows this cycle: → you send a prompt → working memory loads: SOUL.md + memory.md + relevant skills + chat history → agent calls tools (terminal, browser, delegate_task) → agent completes the task, replies to you → AFTER the reply: agent checks "did I learn something worth saving?" → if yes: updates memory.md or creates a new skill → next session starts smarter than the last this happens automatically. you don't ask the agent to learn. it decides what to remember on its own. WHAT MAKES THIS DIFFERENT FROM CLAUDE CODE: Claude Code has memory too. but Hermes stores everything locally. no cloud. your data never leaves your machine. Claude Code doesn't auto-create skills from experience. Hermes turns repeated workflows into reusable procedures. Claude Code memory is instruction-based. Hermes memory is conversational and self-updating. over months of usage, Hermes builds a knowledge base of your preferences, your projects, your mistakes, and the procedures that work for your specific workflow. the agent that remembers your birthday also remembers why your last deploy failed. NO EMBEDDINGS. PLAIN TEXT. Hermes does not use embeddings or RAG for memory. skill and memory search runs on plain text keyword matching. simpler. faster. no vector database to maintain. works entirely offline on your local machine. DELEGATE TO CLAUDE CODE: Hermes can spawn a sub-agent that runs Claude Code in headless mode: "spawn a sub-agent using Claude CLI to build a Python script that fetches the top 5 Hacker News stories to markdown." Hermes delegates. Claude Code writes the code. result returns to Hermes. Hermes runs the script and delivers the output. use Hermes for orchestration. use Claude Code for heavy coding. both tools. not competitors. WHAT HERMES DOES NOT HAVE: no built-in eval or LMOps system. no LangSmith, no LangFuse integration out of the box. trajectory export and logs exist but there is no automated quality tracking. if you need eval, build it yourself or connect external tools. the loop is self-improving. measuring how well it improves is on you. comment LOOP and I'll send you the configs that control how fast Hermes learns and what it remembers. memory limits, skill auto-creation triggers, and the auxiliary model that runs the learning. Replace your entire team with 8 hermes agents👇
20:54

HERMES AGENT LEARNS FROM ITS OWN MISTAKES. UPDATES ITS MEMORY. CREATES ITS OWN SKILLS. NO CLOUD. EVERYTHING STORED LOCALLY. THIS IS HOW THE SELF-IMPROVING LOOP WORKS. most agents start from zero every session. Hermes carries forward what it learned. THREE MEMORY SYSTEMS: 1. PROCEDURAL MEMORY (how to act) stored in ~/.hermes/skills/ as SKILL.md files. when the agent repeats a complex workflow, it saves the procedure as a reusable skill. next time the same task comes up, it follows the skill instead of figuring it out again. you can also create skills explicitly: "create a skill called video-prep that captures how I format my video scripts. spoken english, define jargon inline, no em-dashes, close with a catchphrase." the agent writes the SKILL.md. available as a slash command from that moment. Hermes ships with 90+ skills. the number grows the longer you use it. 2. SEMANTIC MEMORY (durable facts about you) stored in ~/.hermes/memory/memory.md the agent scans conversations for facts worth remembering. preferences, habits, corrections, project details. real example from the video: agent tried to scrape a YouTube channel. URL was wrong. it failed. it updated memory.md with the correct URL pattern so it never makes the same mistake again. you can also save explicitly: "save to memory that my favorite testing framework is pytest" the agent updates memory.md immediately. this file loads into context on every session. the agent knows you better every week. 3. EPISODIC MEMORY (chat history) stored in ~/.hermes/state.db (local SQLite). every conversation. every tool call. every result. searchable with FTS5 full-text search. "search our past sessions. what was the first thing I ever said to you?" the agent queries state.db and finds it. over time, auxiliary models consolidate episodic memory into semantic memory. distilling recurring patterns into durable facts. THE SELF-IMPROVING LOOP: every agent run follows this cycle: → you send a prompt → working memory loads: SOUL.md + memory.md + relevant skills + chat history → agent calls tools (terminal, browser, delegate_task) → agent completes the task, replies to you → AFTER the reply: agent checks "did I learn something worth saving?" → if yes: updates memory.md or creates a new skill → next session starts smarter than the last this happens automatically. you don't ask the agent to learn. it decides what to remember on its own. WHAT MAKES THIS DIFFERENT FROM CLAUDE CODE: Claude Code has memory too. but Hermes stores everything locally. no cloud. your data never leaves your machine. Claude Code doesn't auto-create skills from experience. Hermes turns repeated workflows into reusable procedures. Claude Code memory is instruction-based. Hermes memory is conversational and self-updating. over months of usage, Hermes builds a knowledge base of your preferences, your projects, your mistakes, and the procedures that work for your specific workflow. the agent that remembers your birthday also remembers why your last deploy failed. NO EMBEDDINGS. PLAIN TEXT. Hermes does not use embeddings or RAG for memory. skill and memory search runs on plain text keyword matching. simpler. faster. no vector database to maintain. works entirely offline on your local machine. DELEGATE TO CLAUDE CODE: Hermes can spawn a sub-agent that runs Claude Code in headless mode: "spawn a sub-agent using Claude CLI to build a Python script that fetches the top 5 Hacker News stories to markdown." Hermes delegates. Claude Code writes the code. result returns to Hermes. Hermes runs the script and delivers the output. use Hermes for orchestration. use Claude Code for heavy coding. both tools. not competitors. WHAT HERMES DOES NOT HAVE: no built-in eval or LMOps system. no LangSmith, no LangFuse integration out of the box. trajectory export and logs exist but there is no automated quality tracking. if you need eval, build it yourself or connect external tools. the loop is self-improving. measuring how well it improves is on you. comment LOOP and I'll send you the configs that control how fast Hermes learns and what it remembers. memory limits, skill auto-creation triggers, and the auxiliary model that runs the learning. Replace your entire team with 8 hermes agents👇

YanXbt

22,720 次观看 • 1 个月前

$AMD $5 Trillion is Inevitable LT| Agentic AI🧵 Agentic AI is the new $5 Trillion TAM 🚨🚨🚨 This thead will do Comp with $INTC and how to quantify this massive Agentic AI demand spike, and forcing Jensen to rush a CPU design. Global Agentic AI Market size is estimated to be $3-$5Trillion TAM by 2030(McKinsey) Quantifying the demand from agentic AI for AMD involves assessing the broader market growth for agentic systems, their unique computational requirements (particularly for CPUs in orchestration and reasoning tasks), and AMD's positioning very well through products like EPYC processors and partnerships. AMD EPYC Venice is the most superior choice in 2026-2027 for most Agentic AI workloads Agentic AI refers to autonomous AI agents that perform multi-step tasks, involving sequential logic, tool integration, and decision-making workloads that heavily rely on CPUs for handling orchestration, memory management, and context switching, rather than just GPU-parallelized training or batch inference. Agentic AI is often cited as 40-100x more "hungry" than traditional AI due to its continuous, 24/7 operation and complex workflows. This stems from factors like chain-of-thought reasoning (multiple LLM calls per query), API/tool interactions, memory management, and orchestration loops, which can generate 10-100x more tokens and require real-time responsiveness. For example, a single agentic query might trigger 5-20 model inferences, making it 10-20x more compute-intensive than simple chatbots, and the always-on nature compounds this to 40-100x overall. Nvidia's CEO has highlighted this as driving "easily 100x more computation" for inference in agentic/reasoning setups. AMD's EPYC Venice (6th Gen EPYC, codenamed "Venice") and Intel's Xeon 7 Diamond Rapids represent the pinnacle of server CPU technology in 2026, both targeting high-performance data center workloads like AI inference, agentic AI orchestration, cloud computing, and HPC. Venice builds on AMD's Zen 6 architecture, emphasizing core density and efficiency, while Diamond Rapids leverages Intel's Panther Cove P-cores for balanced performance. Both chips adopt similar advancements like 16-channel DDR5 memory and PCIe Gen 6, but differ in core counts, process nodes, and overall design philosophy. Intel has faced acute supply constraints across its Xeon lineup, including legacy nodes (Intel 7/3) and the ramping 18A process for next-gen parts. Intel shortage is expected with lead times up to 6 months or longer. 1. AMD EPYC Venice vs Intel Xeon 7 Diamond Rapids Architecture AMD: Zen 6 chiplet design with 8 CCDs and dual IODs Intel: Panther Cove P-cores; multi-die architecture with 4 compute tiles Core/Thread Count AMD: Up to 256 cores / 512 threads (Zen 6c variant) Intel: Up to 192 cores / 192 threads Process Node AMD: TSMC N2 (2nm) Intel: Intel 18A (1.8nm-class); in-house fab Memory Support AMD: 16-channel DDR5; up to 1.6 TB/s bandwidth. Intel: 16-channel DDR5 ; up to 1.6 TB/s bandwidth I/O and Connectivity AMD: PCIe Gen 6 (up to 128 lanes); twice the CPU-to-GPU bandwidth Intel: PCIe Gen 6 (up to 128 lanes); LGA 9324 socket Power (TDP) AMD: Starting 400-500W, potentially lower due to efficiency gains from TSMC 2nm Intel: Starting 400-500W, as it targets competitive efficiency Performance Projections AMD: Up to 70% uplift vs. 5th Gen Turin (1.7x in multi-threaded/AI tasks) Intel: ~40% faster than Granite Rapids (Xeon 6, 128-core). Lags AMD in per-core perf and 40-50% behind Venice core-for-core comp Target Workloads AMD: AI inference/orchestration, HPC, cloud virtualization. Partnerships Intel: Hyperscale AI, general enterprise. Custom silicon Pricing: AMD: estimated $10k-$20k for top SKUs Intel: estimated $8-$18k Availability: AMD: Significant Ramp H2 2026 due to higher allocation from TSMC Intel: H1-H2 2026 delayed, but trying to catch up Overall: ~Venice's 256 cores provide a 33% edge over Diamond Rapids' 192, making it superior for massively parallel tasks like AI training/inference or virtualization ~TSMC's N2 vs. Intel 18A debates rage on which is "better," but AMD's mature chiplet approach yields better density ( 32 cores/CCD vs. Intel's 48/tile). Venice's redesign reduces latency, aiding agentic AI where CPUs handle orchestration ~ Early projections show Venice widening AMD's lead matching or exceeding Diamond Rapids' perf with fewer watts in multi-threaded benchmarks. Intel's no-SMT design (to prioritize AI) handicaps it vs. AMD's 512 threads, though Clearwater Forest (E-core) could compete in density-focused niches. ~Power & Cooling: Both push above 400-500W, demanding liquid cooling. ~AMD been taking market share now above 40%. AMD EPYC Venice emerges as the superior choice in 2026 for most server workloads. Its higher core/thread count (256/512 vs. 192/192), stronger per-core performance, and architecture optimized for AI-driven tasks (agentic orchestration with GPU integration) provide decisive advantages in throughput, scalability, and efficiency. Projections indicate Venice delivering 1.7x the performance of prior gens while widening the gap over Intel ( 40-70% leads in multi-threaded benchmarks). AMD's fabless model with TSMC ensures reliable scaling, and its ecosystem ( open ROCm) appeals to AI adopters. Intel's Diamond Rapids is competitive in single-threaded enterprise apps and custom hyperscale ( NVLink), with potential fab advantages for supply/security. However, without SMT and lower density, it falls short in core-for-core battles—exposing Intel to another generation of AMD dominance unless 18A yields surprise efficiency gains. For data centers prioritizing raw compute ( AI, HPC), Venice wins; for Intel-centric ecosystems or specialized I/O, Diamond Rapids holds ground. Real benchmarks post-launch will confirm, but logic points to AMD pulling ahead. 2. Market size , Potential Revenue and Supply Global Agentic AI market size is projected to be $3-$5 Trillion by 2030 according to McKinsey, where consensus points to 40-50% CAGR driven by small to large enterprise demand. I also wrote a full thread on how and why Agentic AI is so explosive that AMD will blow all anlaysts estimate for subscribers. Link below if you are interested. AMD's data center segment hit a record $5.4B in Q4 2025 (up 39% YoY), with EPYC shipments ramping due to agentic demand. With 2GW of deployment in H2 2026, AMD AI data center revenue has $40-$50B+ at the lowest or most conservative projection; or Total Revenue in the $77-$94B For FY2026. However, Agentic AI massive demand spike could send EPYC revenue 3x to 4x in the next few years, potentially surpassing MI series GPU demand as enterprises prioritize CPU-dense Rack setups. This is pushing $NVDA Jensen to rush a CPU design and acquired Groq, a new CPU player due to this massive TAM. Noted that this is just popping just in weeks, highlighting we are just so early in this AI Supercycle and the pace of adoption is insane, and clearly productivity will skyrocket. Why? Because Agentic AI is 24/7 Smart AI agent working for you or your businesses is a mad compelling, and it is estimated to be 40-100x more Inference Hugnry! Many experts already said it is impossible to project this kind of Inference Demand. AI CapEx is expected to ramp up even more in 2027-2028-2029 and 2030 as Global Agentic AI is going to scale to $3-$5 Trillion TAM by 2030. The nature of Agentic is driving higher CPU/GPU ratio, with CPUs handling 50-90% of Agentic workflows. For example, The current Helios Rack: 18 compute trays per rack with 72 GPUs + 18 CPUs. The beauty of this $META and $AMD long term partnership is, that it is absolutely flexible to adjust racks to higher CPU rato or equal to service different needs. Helios rack can be easily swap to 2 GPUs 2CPUs or even CPUs only trays for dedicated orchestration/head nodes. You see, the beauty of this open rack-scale is flexibility and evolvability. If Agentic AI demand pushes much higher, AMD should be able to adjust variant trays without abandoning Heilos Rack. We can't talk just about massive Agentic AI demand without talking about the Supply side or TSMC. TSMC, AMD's primary foundry for advanced nodes ( Zen 6/Venice on N2/2nm), is addressing AI-driven shortages through massive expansions. TSMC accelerates fab construction with up to 10 facilities targeted for 2026. TSMC is accelerating its domestic manufacturing expansion, with industry sources indicating that as many as ten fabs could be under construction or preparing to begin operations across Taiwan’s major science parks. TSMC Capex: $52-56B in 2026 (up 37% YoY), with $45B already approved for new/upgraded capacities. 70-80% for advanced processes (2nm/A16), 10-20% for packaging (CoWoS quadrupling to 120-140K wafers/month by late 2026). In addition, Taiwanese companies (led by TSMC) commit to at least $250B in direct investments in US-based advanced semiconductor, AI, and energy production/innovation capacity.Taiwan provides $250B in government credit guarantees to facilitate additional investments and build a full US semiconductor ecosystem (including industrial parks). TSMC completed a second land purchase in Arizona (January 2026) for gigafab scaling, with an additional $100B+ (potentially four more modules) to further expand and qualify for tariff exemptions. AMD with secured 12GW from OpenAI and $META and massive Agentic AI will mean higher priority acess to 20-30% more wafers on TSMC advanced nodes, as TSMC has multi-year agreements with AMD for AI chips. Dr. C. C. Wei, CEO of TSMC quote: "I spend a lot of time in the last three or four months talking to my customer and then customers. Customer. I want to make sure that my customers demand are real. I talk to those cloud service providers, all of them. Their answer is. I'm quite satisfied with their answer. Actually they show me the evidence that the AI really help their business. So they grow their business successfully and he or she in their financial return. So I also double check their financial status. They are very rich." Amid shortages, the US buildout ensures AMD can ramp production of Instinct GPUs and EPYC CPUs without the constraints hitting competitors like Intel. By diversifying away from Taiwan (85% of advanced nodes today), the agreement mitigates supply disruptions, ensuring stable flows for AMD's chips. Scaling production and securing supply will matter for AMD the most in the next 5-10 years growth. The growth could be 80-100% YoY or higher; or it could be in the 60%. The aggressive TSMC supply ramp is reassuring the higher growth point. Conclusion: AMD stands at a pivotal inflection point in 2026, where the explosive rise of agentic AI demanding 40-100x more inference compute through its 24/7, multi-step orchestration positions the company to potentially triple its EPYC CPU revenue to $45-60B+ by 2028 while scaling Instinct GPUs to tens of billions annually by 2027. Agentic AI demand could push AI CapEx closer to $1 Trillion in 2027, far higher than most estimates. Dr. Lisa Su, AMD's visionary CEO, is masterfully securing supply to harness this massive demand by prioritizing operational execution and deep TSMC collaboration, ensuring readiness for the second-half 2026 AI ramp. Dr. Su has explicitly called out surging EPYC demand for agentic tasks where CPUs power head nodes and traditional workloads alongside GPUs while guiding for data center dominance through proactive capacity planning and partnerships like Nutanix ($150M investment for open agentic platforms) or providing tens of millions CPUs for OpenAI, $META, $ORCL, $AMZN, $MSFT, $GOOGL and others. Her strategy includes multi-year TSMC agreements for advanced nodes (N2 for Venice CPUs and future Instincts), diversifying beyond Taiwan to mitigate risks, and unveiling innovations like the MI455X GPU at CES 2026, which she touted as enabling "the next trillion-dollar market opportunity" in physical AI. Dr. Su's forward-looking vision predicting AI reaching 5 billion users emphasizes "AI everywhere," backed by hardware like Ryzen AI chips, all while declaring demand "going through the roof" and committing to scale without bottlenecks. TSMC's aggressive ramp-up, fueled by $52-56B in 2026 capex (up 37% YoY) and 10+ new fabs across Taiwan, the US (Arizona cluster expanding to 6+ modules with $165B+ investment), Japan, and Europe, provides profound reassurance for AMD's supply stability. The January 2026 US-Taiwan agreement committing $250B in investments and credit guarantees for US reshoring accelerates this, granting tariff relief (15% rates with 1.5-2.5x exemptions) tied to capacity buildouts, enabling TSMC to potentially double output over the decade to meet AI wafer hunger. This translates to 20-30% higher wafer allocations on key nodes, sidestepping Intel-like shortages and empowering Dr. Su's team to deliver on hyperscaler demands without disruption. Ultimately, this synergy cements AMD's leadership in the agentic era, promising sustained growth, $5T+ valuations at scale, and a resilient path forward as AI reshapes the world. This is NOT Financial Advice! Video source: AMD CES 2026
2:49

$AMD $5 Trillion is Inevitable LT| Agentic AI🧵 Agentic AI is the new $5 Trillion TAM 🚨🚨🚨 This thead will do Comp with $INTC and how to quantify this massive Agentic AI demand spike, and forcing Jensen to rush a CPU design. Global Agentic AI Market size is estimated to be $3-$5Trillion TAM by 2030(McKinsey) Quantifying the demand from agentic AI for AMD involves assessing the broader market growth for agentic systems, their unique computational requirements (particularly for CPUs in orchestration and reasoning tasks), and AMD's positioning very well through products like EPYC processors and partnerships. AMD EPYC Venice is the most superior choice in 2026-2027 for most Agentic AI workloads Agentic AI refers to autonomous AI agents that perform multi-step tasks, involving sequential logic, tool integration, and decision-making workloads that heavily rely on CPUs for handling orchestration, memory management, and context switching, rather than just GPU-parallelized training or batch inference. Agentic AI is often cited as 40-100x more "hungry" than traditional AI due to its continuous, 24/7 operation and complex workflows. This stems from factors like chain-of-thought reasoning (multiple LLM calls per query), API/tool interactions, memory management, and orchestration loops, which can generate 10-100x more tokens and require real-time responsiveness. For example, a single agentic query might trigger 5-20 model inferences, making it 10-20x more compute-intensive than simple chatbots, and the always-on nature compounds this to 40-100x overall. Nvidia's CEO has highlighted this as driving "easily 100x more computation" for inference in agentic/reasoning setups. AMD's EPYC Venice (6th Gen EPYC, codenamed "Venice") and Intel's Xeon 7 Diamond Rapids represent the pinnacle of server CPU technology in 2026, both targeting high-performance data center workloads like AI inference, agentic AI orchestration, cloud computing, and HPC. Venice builds on AMD's Zen 6 architecture, emphasizing core density and efficiency, while Diamond Rapids leverages Intel's Panther Cove P-cores for balanced performance. Both chips adopt similar advancements like 16-channel DDR5 memory and PCIe Gen 6, but differ in core counts, process nodes, and overall design philosophy. Intel has faced acute supply constraints across its Xeon lineup, including legacy nodes (Intel 7/3) and the ramping 18A process for next-gen parts. Intel shortage is expected with lead times up to 6 months or longer. 1. AMD EPYC Venice vs Intel Xeon 7 Diamond Rapids Architecture AMD: Zen 6 chiplet design with 8 CCDs and dual IODs Intel: Panther Cove P-cores; multi-die architecture with 4 compute tiles Core/Thread Count AMD: Up to 256 cores / 512 threads (Zen 6c variant) Intel: Up to 192 cores / 192 threads Process Node AMD: TSMC N2 (2nm) Intel: Intel 18A (1.8nm-class); in-house fab Memory Support AMD: 16-channel DDR5; up to 1.6 TB/s bandwidth. Intel: 16-channel DDR5 ; up to 1.6 TB/s bandwidth I/O and Connectivity AMD: PCIe Gen 6 (up to 128 lanes); twice the CPU-to-GPU bandwidth Intel: PCIe Gen 6 (up to 128 lanes); LGA 9324 socket Power (TDP) AMD: Starting 400-500W, potentially lower due to efficiency gains from TSMC 2nm Intel: Starting 400-500W, as it targets competitive efficiency Performance Projections AMD: Up to 70% uplift vs. 5th Gen Turin (1.7x in multi-threaded/AI tasks) Intel: ~40% faster than Granite Rapids (Xeon 6, 128-core). Lags AMD in per-core perf and 40-50% behind Venice core-for-core comp Target Workloads AMD: AI inference/orchestration, HPC, cloud virtualization. Partnerships Intel: Hyperscale AI, general enterprise. Custom silicon Pricing: AMD: estimated $10k-$20k for top SKUs Intel: estimated $8-$18k Availability: AMD: Significant Ramp H2 2026 due to higher allocation from TSMC Intel: H1-H2 2026 delayed, but trying to catch up Overall: ~Venice's 256 cores provide a 33% edge over Diamond Rapids' 192, making it superior for massively parallel tasks like AI training/inference or virtualization ~TSMC's N2 vs. Intel 18A debates rage on which is "better," but AMD's mature chiplet approach yields better density ( 32 cores/CCD vs. Intel's 48/tile). Venice's redesign reduces latency, aiding agentic AI where CPUs handle orchestration ~ Early projections show Venice widening AMD's lead matching or exceeding Diamond Rapids' perf with fewer watts in multi-threaded benchmarks. Intel's no-SMT design (to prioritize AI) handicaps it vs. AMD's 512 threads, though Clearwater Forest (E-core) could compete in density-focused niches. ~Power & Cooling: Both push above 400-500W, demanding liquid cooling. ~AMD been taking market share now above 40%. AMD EPYC Venice emerges as the superior choice in 2026 for most server workloads. Its higher core/thread count (256/512 vs. 192/192), stronger per-core performance, and architecture optimized for AI-driven tasks (agentic orchestration with GPU integration) provide decisive advantages in throughput, scalability, and efficiency. Projections indicate Venice delivering 1.7x the performance of prior gens while widening the gap over Intel ( 40-70% leads in multi-threaded benchmarks). AMD's fabless model with TSMC ensures reliable scaling, and its ecosystem ( open ROCm) appeals to AI adopters. Intel's Diamond Rapids is competitive in single-threaded enterprise apps and custom hyperscale ( NVLink), with potential fab advantages for supply/security. However, without SMT and lower density, it falls short in core-for-core battles—exposing Intel to another generation of AMD dominance unless 18A yields surprise efficiency gains. For data centers prioritizing raw compute ( AI, HPC), Venice wins; for Intel-centric ecosystems or specialized I/O, Diamond Rapids holds ground. Real benchmarks post-launch will confirm, but logic points to AMD pulling ahead. 2. Market size , Potential Revenue and Supply Global Agentic AI market size is projected to be $3-$5 Trillion by 2030 according to McKinsey, where consensus points to 40-50% CAGR driven by small to large enterprise demand. I also wrote a full thread on how and why Agentic AI is so explosive that AMD will blow all anlaysts estimate for subscribers. Link below if you are interested. AMD's data center segment hit a record $5.4B in Q4 2025 (up 39% YoY), with EPYC shipments ramping due to agentic demand. With 2GW of deployment in H2 2026, AMD AI data center revenue has $40-$50B+ at the lowest or most conservative projection; or Total Revenue in the $77-$94B For FY2026. However, Agentic AI massive demand spike could send EPYC revenue 3x to 4x in the next few years, potentially surpassing MI series GPU demand as enterprises prioritize CPU-dense Rack setups. This is pushing $NVDA Jensen to rush a CPU design and acquired Groq, a new CPU player due to this massive TAM. Noted that this is just popping just in weeks, highlighting we are just so early in this AI Supercycle and the pace of adoption is insane, and clearly productivity will skyrocket. Why? Because Agentic AI is 24/7 Smart AI agent working for you or your businesses is a mad compelling, and it is estimated to be 40-100x more Inference Hugnry! Many experts already said it is impossible to project this kind of Inference Demand. AI CapEx is expected to ramp up even more in 2027-2028-2029 and 2030 as Global Agentic AI is going to scale to $3-$5 Trillion TAM by 2030. The nature of Agentic is driving higher CPU/GPU ratio, with CPUs handling 50-90% of Agentic workflows. For example, The current Helios Rack: 18 compute trays per rack with 72 GPUs + 18 CPUs. The beauty of this $META and $AMD long term partnership is, that it is absolutely flexible to adjust racks to higher CPU rato or equal to service different needs. Helios rack can be easily swap to 2 GPUs 2CPUs or even CPUs only trays for dedicated orchestration/head nodes. You see, the beauty of this open rack-scale is flexibility and evolvability. If Agentic AI demand pushes much higher, AMD should be able to adjust variant trays without abandoning Heilos Rack. We can't talk just about massive Agentic AI demand without talking about the Supply side or TSMC. TSMC, AMD's primary foundry for advanced nodes ( Zen 6/Venice on N2/2nm), is addressing AI-driven shortages through massive expansions. TSMC accelerates fab construction with up to 10 facilities targeted for 2026. TSMC is accelerating its domestic manufacturing expansion, with industry sources indicating that as many as ten fabs could be under construction or preparing to begin operations across Taiwan’s major science parks. TSMC Capex: $52-56B in 2026 (up 37% YoY), with $45B already approved for new/upgraded capacities. 70-80% for advanced processes (2nm/A16), 10-20% for packaging (CoWoS quadrupling to 120-140K wafers/month by late 2026). In addition, Taiwanese companies (led by TSMC) commit to at least $250B in direct investments in US-based advanced semiconductor, AI, and energy production/innovation capacity.Taiwan provides $250B in government credit guarantees to facilitate additional investments and build a full US semiconductor ecosystem (including industrial parks). TSMC completed a second land purchase in Arizona (January 2026) for gigafab scaling, with an additional $100B+ (potentially four more modules) to further expand and qualify for tariff exemptions. AMD with secured 12GW from OpenAI and $META and massive Agentic AI will mean higher priority acess to 20-30% more wafers on TSMC advanced nodes, as TSMC has multi-year agreements with AMD for AI chips. Dr. C. C. Wei, CEO of TSMC quote: "I spend a lot of time in the last three or four months talking to my customer and then customers. Customer. I want to make sure that my customers demand are real. I talk to those cloud service providers, all of them. Their answer is. I'm quite satisfied with their answer. Actually they show me the evidence that the AI really help their business. So they grow their business successfully and he or she in their financial return. So I also double check their financial status. They are very rich." Amid shortages, the US buildout ensures AMD can ramp production of Instinct GPUs and EPYC CPUs without the constraints hitting competitors like Intel. By diversifying away from Taiwan (85% of advanced nodes today), the agreement mitigates supply disruptions, ensuring stable flows for AMD's chips. Scaling production and securing supply will matter for AMD the most in the next 5-10 years growth. The growth could be 80-100% YoY or higher; or it could be in the 60%. The aggressive TSMC supply ramp is reassuring the higher growth point. Conclusion: AMD stands at a pivotal inflection point in 2026, where the explosive rise of agentic AI demanding 40-100x more inference compute through its 24/7, multi-step orchestration positions the company to potentially triple its EPYC CPU revenue to $45-60B+ by 2028 while scaling Instinct GPUs to tens of billions annually by 2027. Agentic AI demand could push AI CapEx closer to $1 Trillion in 2027, far higher than most estimates. Dr. Lisa Su, AMD's visionary CEO, is masterfully securing supply to harness this massive demand by prioritizing operational execution and deep TSMC collaboration, ensuring readiness for the second-half 2026 AI ramp. Dr. Su has explicitly called out surging EPYC demand for agentic tasks where CPUs power head nodes and traditional workloads alongside GPUs while guiding for data center dominance through proactive capacity planning and partnerships like Nutanix ($150M investment for open agentic platforms) or providing tens of millions CPUs for OpenAI, $META, $ORCL, $AMZN, $MSFT, $GOOGL and others. Her strategy includes multi-year TSMC agreements for advanced nodes (N2 for Venice CPUs and future Instincts), diversifying beyond Taiwan to mitigate risks, and unveiling innovations like the MI455X GPU at CES 2026, which she touted as enabling "the next trillion-dollar market opportunity" in physical AI. Dr. Su's forward-looking vision predicting AI reaching 5 billion users emphasizes "AI everywhere," backed by hardware like Ryzen AI chips, all while declaring demand "going through the roof" and committing to scale without bottlenecks. TSMC's aggressive ramp-up, fueled by $52-56B in 2026 capex (up 37% YoY) and 10+ new fabs across Taiwan, the US (Arizona cluster expanding to 6+ modules with $165B+ investment), Japan, and Europe, provides profound reassurance for AMD's supply stability. The January 2026 US-Taiwan agreement committing $250B in investments and credit guarantees for US reshoring accelerates this, granting tariff relief (15% rates with 1.5-2.5x exemptions) tied to capacity buildouts, enabling TSMC to potentially double output over the decade to meet AI wafer hunger. This translates to 20-30% higher wafer allocations on key nodes, sidestepping Intel-like shortages and empowering Dr. Su's team to deliver on hyperscaler demands without disruption. Ultimately, this synergy cements AMD's leadership in the agentic era, promising sustained growth, $5T+ valuations at scale, and a resilient path forward as AI reshapes the world. This is NOT Financial Advice! Video source: AMD CES 2026

Mike

44,460 次观看 • 5 个月前

Deepseek V4 Flash 0731 (Q2) - 12 tokens/sec - Single RTX 4090 - 650+ tokens/sec prefill - 250k context - no kv cache quantization! DeepSeek just dropped the official V4 Flash 0731 two days ago with a massive agent capabilities upgrade. The official benchmarks are literally crushing their own V4-Pro-Preview on agentic tasks like Terminal Bench 2.1 and DeepSWE. Unsloth AI said they couldn't wait to bring it to local devices, and they delivered. If you thought my 118B Poolside Laguna S 2.1 MoE run last week on a single GPU was wild, hold onto your hardware. I just successfully ran Unsloth’s brand new 91GB DeepSeek-V4-Flash-0731 (UD-IQ2_M) GGUF entirely locally. And I pushed it to a mind-bending 250,000 context window. The VRAM ceiling is an illusion if you know how to optimize llama.cpp. Here are the benchmarks and the cheat codes to run a local frontier class model yourself. For the hardware and setup, I used a single NVIDIA RTX 4090 (24GB VRAM) hooked up via a PCIe 4 bus, running Ubuntu 22.04 LTS and CUDA 13.0. You don't need a massive enterprise server for this, if you have more than 80 GB of standard DDR4 RAM and a 24GB card like an RTX 3090 or 4090, you can run this exact stack yourself. All benchmarks were run using a massive 28k token prompt to truly stress test the prefill limits. no kv cache quantization THE BENCHMARKS (Scaling Context): # 80k Context (Baseline: -b 2048 -ub 2048): Prefill: 465.43 t/s | Decode: 13.00 t/s | VRAM: 22.87 GB # 80k Context (Optimized: -b 4096 -ub 4096): Prefill: 643.15 t/s | Decode: 12.20 t/s | VRAM: 23.00 GB (Notice how doubling the batch flags spiked my prefill throughput by nearly 200 t/s with almost zero VRAM penalty) # 180k Context (-b 4096 -ub 4096): Prefill: 629.18 t/s | Decode: 11.92 t/s | VRAM: 23.40 GB # 250k Context MAXIMUM (-b 4096 -ub 4096): Prefill: 619.02 t/s | Decode: 11.54 t/s | VRAM: 23.40 GB # THE SECRET SAUCE (Why this works): Unsloth’s UD-IQ2_M quant is ~91GB across 3 files. Since I only have 24GB of VRAM, the PCIe 4 bus and system RAM have to do the heavy lifting. The magic bullet is the --no-mmap flag. By completely bypassing OS disk paging, I forced llama.cpp to load the massive model weights directly into the system RAM upfront. Combined with Flash Attention (-fa on) and exactly 12 CPU threads (--threads 12), I maintained an incredibly stable 11.5+ tokens/sec decode speed even at a quarter million token context. # THE EXACT COMMAND: ./build/bin/llama-server -m /workspace/models/DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf -c 250000 -fa on --port 8080 --threads 12 -b 4096 -ub 4096 --no-mmap -v Local conversational and agentic coding AI is fully here. You don’t need an API or an H100 cluster. Qwen 3.8 27b drops next week making the 24GB VRAM tier even more worthwhile. What does your current local AI rig look like, and what's the craziest model you've managed to squeeze into it? Official huggingface GGUF links from Unsloth and performance graphs are dropped in the replies below!
0:21

Deepseek V4 Flash 0731 (Q2) - 12 tokens/sec - Single RTX 4090 - 650+ tokens/sec prefill - 250k context - no kv cache quantization! DeepSeek just dropped the official V4 Flash 0731 two days ago with a massive agent capabilities upgrade. The official benchmarks are literally crushing their own V4-Pro-Preview on agentic tasks like Terminal Bench 2.1 and DeepSWE. Unsloth AI said they couldn't wait to bring it to local devices, and they delivered. If you thought my 118B Poolside Laguna S 2.1 MoE run last week on a single GPU was wild, hold onto your hardware. I just successfully ran Unsloth’s brand new 91GB DeepSeek-V4-Flash-0731 (UD-IQ2_M) GGUF entirely locally. And I pushed it to a mind-bending 250,000 context window. The VRAM ceiling is an illusion if you know how to optimize llama.cpp. Here are the benchmarks and the cheat codes to run a local frontier class model yourself. For the hardware and setup, I used a single NVIDIA RTX 4090 (24GB VRAM) hooked up via a PCIe 4 bus, running Ubuntu 22.04 LTS and CUDA 13.0. You don't need a massive enterprise server for this, if you have more than 80 GB of standard DDR4 RAM and a 24GB card like an RTX 3090 or 4090, you can run this exact stack yourself. All benchmarks were run using a massive 28k token prompt to truly stress test the prefill limits. no kv cache quantization THE BENCHMARKS (Scaling Context): # 80k Context (Baseline: -b 2048 -ub 2048): Prefill: 465.43 t/s | Decode: 13.00 t/s | VRAM: 22.87 GB # 80k Context (Optimized: -b 4096 -ub 4096): Prefill: 643.15 t/s | Decode: 12.20 t/s | VRAM: 23.00 GB (Notice how doubling the batch flags spiked my prefill throughput by nearly 200 t/s with almost zero VRAM penalty) # 180k Context (-b 4096 -ub 4096): Prefill: 629.18 t/s | Decode: 11.92 t/s | VRAM: 23.40 GB # 250k Context MAXIMUM (-b 4096 -ub 4096): Prefill: 619.02 t/s | Decode: 11.54 t/s | VRAM: 23.40 GB # THE SECRET SAUCE (Why this works): Unsloth’s UD-IQ2_M quant is ~91GB across 3 files. Since I only have 24GB of VRAM, the PCIe 4 bus and system RAM have to do the heavy lifting. The magic bullet is the --no-mmap flag. By completely bypassing OS disk paging, I forced llama.cpp to load the massive model weights directly into the system RAM upfront. Combined with Flash Attention (-fa on) and exactly 12 CPU threads (--threads 12), I maintained an incredibly stable 11.5+ tokens/sec decode speed even at a quarter million token context. # THE EXACT COMMAND: ./build/bin/llama-server -m /workspace/models/DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf -c 250000 -fa on --port 8080 --threads 12 -b 4096 -ub 4096 --no-mmap -v Local conversational and agentic coding AI is fully here. You don’t need an API or an H100 cluster. Qwen 3.8 27b drops next week making the 24GB VRAM tier even more worthwhile. What does your current local AI rig look like, and what's the craziest model you've managed to squeeze into it? Official huggingface GGUF links from Unsloth and performance graphs are dropped in the replies below!

Alok

35,450 次观看 • 3 天前

$NVDA $GFS NVIDIA’s reported agreement to acquire Groq for $20B in cash (per CNBC, amplified via Reuters and other wire coverage) represents a materially different strategic posture than NVIDIA’s prior M&A pattern, given both the headline size (largest reported NVIDIA acquisition to date) and the unusual carve-out that Groq’s early-stage cloud business would not be included. Public reporting indicates the information originated from Alex Davis, CEO of Disruptive (lead investor in Groq’s latest financing), and that neither NVIDIA nor Groq had issued an immediate confirmation at the time of publication. The same reporting frames the transaction as coming together quickly, only months after Groq raised $750M at a ~$6.9B valuation, and highlights Groq’s positioning as a high-performance inference chip vendor founded by ex-Google TPU engineers. Groq is best understood as a vertically integrated inference acceleration company whose core asset is an application-specific processor optimized for deterministic, low-latency execution of transformer-style workloads, paired with a compiler-led software stack and a distribution layer (GroqCloud) designed to reduce developer friction via OpenAI-compatible APIs and integrations. Groq brands its architecture as a Language Processing Unit (LPU) and consistently emphasizes that the design target is inference, not training. The company’s own architecture description centers on 1-core execution, large on-chip SRAM used as primary storage (explicitly not cache), a custom compiler that statically schedules compute and communication, and direct chip-to-chip connectivity intended to coordinate multi-chip execution without relying on conventional caching hierarchies or dynamic runtime scheduling. The technical premise is a deliberate inversion of the conventional GPU approach. GPUs deliver throughput via massively parallel, multi-core execution with dynamic scheduling, complex memory hierarchies, and heavy reliance on off-chip HBM bandwidth and sophisticated runtime/kernel optimization. Groq instead argues that inference bottlenecks are driven by latency variance (tail latency), synchronization overhead, and memory access unpredictability inherent in dynamically scheduled, cache-heavy architectures, particularly when workloads are latency sensitive and batch sizes cannot be inflated. Groq’s solution is to move “control” into the compiler: the full execution graph and inter-chip communication schedule are computed ahead of time down to clock-cycle granularity, with deterministic execution designed to reduce run-to-run variance. In Groq’s framing, the removal of caches, reorder buffers, speculative execution overhead, and other sources of contention enables predictable latency and high utilization without per-model kernel engineering typical of GPU tuning cycles. A critical nuance is that Groq’s determinism is not merely a software claim; it is tightly coupled to architectural constraints and system design choices that trade flexibility for predictability. Third-party technical commentary indicates Groq’s chip uses a fully deterministic VLIW-style approach with minimal buffering, no external memory, and heavy dependence on sharding models across many chips because on-chip SRAM capacity is limited. SemiAnalysis describes a ~725 mm^2 die on GlobalFoundries 14nm with ~230MB of SRAM and notes that “no useful models” fit on a single chip, forcing multi-chip partitioning for modern LLMs and driving a system-level design where networking and compilation are first-class scheduling problems rather than ancillary infrastructure. This is consistent with Groq’s own messaging that tensor parallelism across chips is a primary design goal, enabled by large on-chip SRAM and compile-time coordination of compute plus interconnect. The on-chip SRAM emphasis is central to Groq’s latency story and also its most constraining trade-off. Groq claims on-chip SRAM bandwidth “upwards of 80 TB/s” and contrasts that with off-chip HBM bandwidth “about 8 TB/s,” asserting a potential 10x advantage from bandwidth plus reduced trips across chip-to-memory boundaries. While these comparisons are marketing-oriented and depend on workload specifics, the architectural implication is clear: Groq prioritizes ultra-fast local weight/activation access and then scales capacity by adding chips, not by attaching large off-chip memory pools. This design can reduce latency for sequential inference layers and minimize unpredictable stalls, but it pushes complexity into partitioning strategy, interconnect topology, and compiler scheduling, and it increases the number of chips needed for very large parameter counts and large KV-cache footprints. Groq also highlights numeric formats and compiler-driven precision management as a performance lever. In its 2025 technical blog, Groq describes “TruePoint numerics,” including 100-bit intermediate accumulation and selective quantization choices (FP32 for attention-sensitive operations, block floating point for MoE weights, FP8 storage in error-tolerant layers), and claims 2-4x speedups versus BF16 without measurable accuracy degradation on benchmarks such as MMLU and HumanEval. Even if the absolute uplift is workload dependent, the strategic point is that Groq is pursuing performance via end-to-end co-design: precision policy is not just hardware capability (FP8/BF16) but compiler-enforced mapping of precision to error sensitivity, which can matter materially for inference cost-per-token if it reduces memory traffic and boosts throughput without forcing aggressive, accuracy-damaging quantization. Independent performance datapoints indicate Groq has been credible on latency-oriented inference speed, at least for certain regimes. EE Times reported in 2023 that Groq demonstrated Llama-2 70B inference at ~240 tokens/s per user on a cloud-based dev system described as 10 racks and 64 chips, using the company’s 1st-gen silicon introduced several years earlier. Separate Groq commentary around independent benchmarking cites results showing ~241 tokens/s throughput and ~0.8s time to receive 100 output tokens for a Llama-2 70B API configuration, positioning the platform as a step-change in “available speed” for certain interactive use cases. These figures do not settle total cost-of-ownership versus GPUs or hyperscaler ASICs, but they establish that Groq’s system-level architecture can deliver strong single-user throughput and latency on large models when properly partitioned and scheduled. GroqCloud is the commercial wrapper that packages this hardware/software stack as “tokens-as-a-service,” aiming to make Groq adoption feel like switching API endpoints rather than adopting new silicon. Groq’s documentation states its API is designed to be “mostly compatible” with OpenAI client libraries, and its pricing page provides model-specific token rates, published speeds (tokens/s), prompt caching discounts, and batch processing discounts. For example, pricing lists inputs as low as $0.05 per 1M tokens and outputs as low as $0.08 per 1M tokens for certain smaller LLM configurations, with higher prices for larger models and long-context or MoE variants; it also advertises prompt caching with a 50% discount on cached input tokens for certain models and a batch API offering 50% lower cost for asynchronous processing windows. These mechanics are economically important because they demonstrate Groq’s go-to-market is not simply “sell chips,” but “sell predictable unit economics per token,” with tooling (batch, caching) that directly targets inference cost drivers (reused prompts, throughput smoothing, and asynchronous workloads). The cloud footprint and distribution partnerships indicate Groq has been building an inference-native “edge within the cloud” strategy rather than competing head-on with hyperscalers on breadth of services. A 2025 Groq newsroom release describes a European deployment in Helsinki with Equinix, positioned as latency reduction and data governance for European customers, and explicitly references Equinix Fabric enabling private connectivity to GroqCloud over public, private, or sovereign infrastructure. The same release enumerates additional capacity in the U.S. (Equinix, DataBank), Canada (Bell Canada), and Saudi Arabia (HUMAIN), and states these sites collectively served more than 20M tokens/s across Groq’s global network at that time. That supply-side metric matters because it provides a directional sense that Groq is scaling capacity as a network, not merely as a chip vendor. Customer disclosure is inherently limited because Groq is private and many enterprise deployments are not public, but Groq’s marketing materials and partnerships provide signals about demand vectors. The company’s public website displays logos of large consumer and enterprise brands (e.g., Dropbox, Vercel, Chevron, Volkswagen, Canva, Robinhood, Riot Games, Workday, Ramp) and includes a published customer quote claiming a 7.41x chat speed increase and an 89% cost reduction after moving to GroqCloud, followed by a tripling of token consumption. While marketing claims should be treated as case-specific and not generalized, they indicate that Groq is targeting both AI-native developers (who measure success by latency and cost-per-token) and enterprise buyers (who care about predictable performance and governance). Supplier and dependency mapping for Groq spans 3 layers: silicon production, system integration, and cloud infrastructure. On silicon, third-party analysis indicates GlobalFoundries 14nm for the 1st-gen Groq chip, implying a supply chain less constrained by the most capacity-tight leading-edge nodes and advanced packaging bottlenecks that dominate high-end GPU supply (HBM stacks, CoWoS-type packaging constraints). If accurate, this is strategically meaningful because it suggests Groq capacity expansion could be gated more by conventional wafer supply, board assembly, and data center power than by the same HBM/advanced packaging scarcity that has constrained top-tier GPU ramp cycles. On systems and cloud, Groq’s own releases identify colocation and connectivity partners (Equinix, DataBank, Bell Canada) and a Middle East partner (HUMAIN), implying dependencies on data center real estate, power availability, and network connectivity, alongside procurement of standard server components, NICs/switching, racks, and cooling infrastructure. The Groq design narrative also emphasizes air cooling and reduced need for complex power/cooling infrastructure, which—if realized in deployments—can widen the set of feasible hosting locations and lower deployment friction relative to liquid-cooled, very high power density GPU racks. Against that backdrop, the strategic rationale for NVIDIA acquiring Groq can be framed as a set of overlapping objectives: inference silicon optionality, architectural hedging, competitive defense, and supply chain diversification, with the carve-out of GroqCloud signaling a preference to avoid direct cloud competition and to focus on IP and product portfolio control rather than operating a capital-intensive token-serving business. The deal, if confirmed, would occur at a valuation step-up of ~190% versus Groq’s reported ~$6.9B private valuation in the September $750M round, reinforcing that any acquisition logic would be predominantly strategic rather than a conventional financial multiple arbitrage. The most compelling strategic driver is inference. Training has historically been the center of gravity for cutting-edge GPU demand, but inference volume is structurally larger and more distributed as deployments scale, with economics dominated by cost-per-token, latency guarantees, and utilization under spiky demand. Inference workloads also create a strategic vulnerability for NVIDIA: hyperscalers and large platforms can justify bespoke ASICs (TPU, Trainium/Inferentia, Maia-class efforts) because inference is stable, repeatable, and can amortize software investment at massive scale. Groq’s core proposition—deterministic, compiler-scheduled inference with predictable latency—aligns directly with the segment where GPU generality is least valued and where “good enough” programmability plus superior unit economics can win share. Acquiring Groq would allow NVIDIA to own a credible inference-native architecture rather than relying solely on GPUs and software optimization to defend that segment. Competitive defense logic is also plausible. Groq occupies a specific competitive wedge: low-latency, high-throughput interactive inference, delivered via a simple API abstraction that reduces switching cost. That wedge directly pressures GPU inference margins in the long run because it makes inference price/performance comparisons more transparent at the token level, and it targets a developer persona that historically defaulted to CUDA-first ecosystems. Even if NVIDIA’s current-generation systems can achieve very high tokens/s per user with extensive optimization, the strategic risk is that competing architectures normalize the idea that inference is best served by special-purpose silicon with a simpler programming model, weakening CUDA lock-in at the application layer. NVIDIA has actively demonstrated that Blackwell-era systems can exceed 1,000 tokens/s per user in benchmarked configurations, but that performance leadership does not automatically translate to lowest cost-per-token across the full range of batch sizes, latency targets, and deployment environments. Groq’s existence as a credible alternative architecture forces NVIDIA to keep defending inference economics rather than only raw performance leadership. The “technology acquisition” rationale is unusually strong in this specific case because Groq’s differentiator is not a single block of silicon IP but an end-to-end methodology: compiler-led static scheduling, deterministic networking, and a system architecture designed around tensor-parallel inference rather than throughput-maximizing batch inference. NVIDIA’s stack is already compiler-heavy (TensorRT, Triton, CUDA graphs, kernel fusion, speculative decoding techniques), but GPUs remain dynamically scheduled devices with complex memory hierarchies and stochastic latency behaviors under contention. Groq’s approach provides an alternate design point: treating the entire inference execution (compute plus communication) as a statically schedulable program. In principle, that IP could be valuable even if Groq silicon itself is not adopted at massive scale, because it can inform how NVIDIA builds future inference-optimized products, compilers, and networking fabrics, especially as distributed inference with large models makes communication a first-order performance determinant. Supply chain diversification is a non-obvious but potentially important driver. If Groq’s mainstream product generation is truly based on a mature process node and avoids HBM, then the scaling constraints look different than those of state-of-the-art GPUs. NVIDIA’s ability to meet incremental demand has been tightly coupled to advanced packaging and HBM supply, and those constraints can remain binding even when wafer supply is available. An inference ASIC architecture that relies primarily on on-chip SRAM and scales by adding chips—while not costless—could reduce dependence on HBM availability and advanced packaging capacity, enabling NVIDIA to ship “inference capacity” in higher absolute volumes or into geographies and customer segments where the highest-end GPUs are economically or logistically difficult to deploy. This could be particularly relevant for latency-sensitive inference deployed in regional colocation footprints rather than centralized hyperscale campuses. The carve-out of GroqCloud, if accurate, is itself a strategic signal about NVIDIA’s priorities. Operating a token-serving cloud at scale is capital intensive, structurally lower margin than silicon IP rents, and creates channel conflict with hyperscalers and CSP partners who are core NVIDIA customers. NVIDIA has generally positioned its cloud offerings through partnerships rather than as a direct hyperscale competitor. Excluding GroqCloud would preserve neutrality with CSPs and avoid inheriting multi-region data residency obligations and partner contracts, while still allowing NVIDIA to acquire Groq’s silicon, compiler technology, and engineering talent. At the same time, excluding GroqCloud would also mean NVIDIA would not automatically acquire the commercial proof-point of Groq’s unit economics or the customer contracts that validate product-market fit at scale, increasing the importance of diligence on whether Groq’s cloud pricing is structurally profitable or partially subsidized by fundraising. There is also a “preemptive acquisition” angle. The reporting identifies recent investors in Groq’s latest round including large financial institutions and strategic/industry players. In that context, Groq represents an asset that could plausibly have been acquired by a competitor (AMD/Intel) or by a hyperscaler seeking to accelerate inference independence. NVIDIA acquiring Groq could be a defensive move to prevent a credible inference-native architecture from being weaponized by a rival with deep distribution. Even if GroqCloud is carved out, controlling the silicon roadmap and compiler IP would meaningfully constrain Groq’s ability to evolve into a standalone competitor, unless the carved-out entity retains long-term rights to the hardware and software stack. However, the strategic case is not one-sided; there are meaningful risks and potential contradictions that would need to be reconciled for the transaction to be value-accretive on a multi-year horizon. 1st, Groq’s architecture appears to rely on scaling out chip count to achieve capacity, which introduces system cost, networking complexity, and physical footprint considerations. The absence of external memory and limited on-chip SRAM implies very large models require substantial chip parallelism, and the economics then depend heavily on chip cost, yield, power efficiency, and interconnect overhead. SemiAnalysis explicitly frames Groq as trading space for time and raises questions about token economics and whether publicly advertised pricing reflects fully loaded costs or market share capture. 2nd, integration risk is non-trivial. Groq’s compiler-led deterministic model is philosophically and practically different from CUDA’s dominant programming and execution model. A poorly executed integration could create internal product confusion, dilute engineering focus, or alienate developers if the combined stack fragments. 3rd, there is cannibalization risk. If Groq-class inference silicon undercuts GPU inference economics, NVIDIA could face internal margin trade-offs, even if the goal is to defend share against hyperscaler ASICs. Cannibalization can still be rational if it prevents larger share loss, but it would require crisp portfolio segmentation and go-to-market discipline. The presence of NVIDIA’s own rapidly improving inference performance complicates the “need” for Groq but does not eliminate the “option value.” NVIDIA has demonstrated benchmark-leading tokens/s per user on Blackwell-based systems, suggesting that raw interactive throughput is not necessarily the limiting factor for NVIDIA’s product line. The more enduring strategic question is unit economics and architectural control: whether future inference demand is better monetized through general-purpose GPUs plus software optimization, or whether a bifurcated product portfolio (training GPUs plus inference-native ASICs) becomes necessary to defend total AI compute wallet share as hyperscaler ASIC penetration increases. Acquiring Groq could be a decisive move to ensure NVIDIA participates in both regimes rather than betting exclusively on GPUs to win inference forever. What is “special” about Groq’s technology relative to a typical accelerator roadmap is the tight coupling of determinism, compilation, and networking into a single scheduling problem. The LPU narrative emphasizes deterministic compute and networking, static scheduling, and direct chip-to-chip coordination that allows “hundreds” (more precisely, 100s) of chips to behave like a single scheduled resource. The architecture also explicitly targets tensor-parallel, latency-optimized distribution rather than pure data-parallel throughput scaling, which matters for real-time applications where a single response must arrive quickly rather than many requests being processed in bulk. The implication is that Groq is optimized for the time-to-first-token and steady token streaming behavior that defines user experience in interactive LLMs, and it attempts to achieve that without relying on large batch sizes that can degrade latency. From a portfolio manager’s perspective, the most important interpretation is that an NVIDIA-Groq combination would likely be less about “NVIDIA needs more inference speed” and more about controlling the architectural trajectory of inference acceleration and removing a fast-improving, developer-friendly competitor from the market. The carve-out of GroqCloud would reinforce that the transaction is aimed at IP, talent, and product optionality, not acquiring a cloud revenue stream. The valuation step-up implied by $20B versus $6.9B would therefore be justified only if the acquired assets materially reduce long-term competitive risk (hyperscaler ASIC displacement, inference margin compression) or enable new monetization vectors (inference ASIC product line, supply chain de-bottlenecking, improved software determinism) that would be difficult to achieve on a comparable timeline via internal R&D.
10:20

$NVDA $GFS NVIDIA’s reported agreement to acquire Groq for $20B in cash (per CNBC, amplified via Reuters and other wire coverage) represents a materially different strategic posture than NVIDIA’s prior M&A pattern, given both the headline size (largest reported NVIDIA acquisition to date) and the unusual carve-out that Groq’s early-stage cloud business would not be included. Public reporting indicates the information originated from Alex Davis, CEO of Disruptive (lead investor in Groq’s latest financing), and that neither NVIDIA nor Groq had issued an immediate confirmation at the time of publication. The same reporting frames the transaction as coming together quickly, only months after Groq raised $750M at a ~$6.9B valuation, and highlights Groq’s positioning as a high-performance inference chip vendor founded by ex-Google TPU engineers. Groq is best understood as a vertically integrated inference acceleration company whose core asset is an application-specific processor optimized for deterministic, low-latency execution of transformer-style workloads, paired with a compiler-led software stack and a distribution layer (GroqCloud) designed to reduce developer friction via OpenAI-compatible APIs and integrations. Groq brands its architecture as a Language Processing Unit (LPU) and consistently emphasizes that the design target is inference, not training. The company’s own architecture description centers on 1-core execution, large on-chip SRAM used as primary storage (explicitly not cache), a custom compiler that statically schedules compute and communication, and direct chip-to-chip connectivity intended to coordinate multi-chip execution without relying on conventional caching hierarchies or dynamic runtime scheduling. The technical premise is a deliberate inversion of the conventional GPU approach. GPUs deliver throughput via massively parallel, multi-core execution with dynamic scheduling, complex memory hierarchies, and heavy reliance on off-chip HBM bandwidth and sophisticated runtime/kernel optimization. Groq instead argues that inference bottlenecks are driven by latency variance (tail latency), synchronization overhead, and memory access unpredictability inherent in dynamically scheduled, cache-heavy architectures, particularly when workloads are latency sensitive and batch sizes cannot be inflated. Groq’s solution is to move “control” into the compiler: the full execution graph and inter-chip communication schedule are computed ahead of time down to clock-cycle granularity, with deterministic execution designed to reduce run-to-run variance. In Groq’s framing, the removal of caches, reorder buffers, speculative execution overhead, and other sources of contention enables predictable latency and high utilization without per-model kernel engineering typical of GPU tuning cycles. A critical nuance is that Groq’s determinism is not merely a software claim; it is tightly coupled to architectural constraints and system design choices that trade flexibility for predictability. Third-party technical commentary indicates Groq’s chip uses a fully deterministic VLIW-style approach with minimal buffering, no external memory, and heavy dependence on sharding models across many chips because on-chip SRAM capacity is limited. SemiAnalysis describes a ~725 mm^2 die on GlobalFoundries 14nm with ~230MB of SRAM and notes that “no useful models” fit on a single chip, forcing multi-chip partitioning for modern LLMs and driving a system-level design where networking and compilation are first-class scheduling problems rather than ancillary infrastructure. This is consistent with Groq’s own messaging that tensor parallelism across chips is a primary design goal, enabled by large on-chip SRAM and compile-time coordination of compute plus interconnect. The on-chip SRAM emphasis is central to Groq’s latency story and also its most constraining trade-off. Groq claims on-chip SRAM bandwidth “upwards of 80 TB/s” and contrasts that with off-chip HBM bandwidth “about 8 TB/s,” asserting a potential 10x advantage from bandwidth plus reduced trips across chip-to-memory boundaries. While these comparisons are marketing-oriented and depend on workload specifics, the architectural implication is clear: Groq prioritizes ultra-fast local weight/activation access and then scales capacity by adding chips, not by attaching large off-chip memory pools. This design can reduce latency for sequential inference layers and minimize unpredictable stalls, but it pushes complexity into partitioning strategy, interconnect topology, and compiler scheduling, and it increases the number of chips needed for very large parameter counts and large KV-cache footprints. Groq also highlights numeric formats and compiler-driven precision management as a performance lever. In its 2025 technical blog, Groq describes “TruePoint numerics,” including 100-bit intermediate accumulation and selective quantization choices (FP32 for attention-sensitive operations, block floating point for MoE weights, FP8 storage in error-tolerant layers), and claims 2-4x speedups versus BF16 without measurable accuracy degradation on benchmarks such as MMLU and HumanEval. Even if the absolute uplift is workload dependent, the strategic point is that Groq is pursuing performance via end-to-end co-design: precision policy is not just hardware capability (FP8/BF16) but compiler-enforced mapping of precision to error sensitivity, which can matter materially for inference cost-per-token if it reduces memory traffic and boosts throughput without forcing aggressive, accuracy-damaging quantization. Independent performance datapoints indicate Groq has been credible on latency-oriented inference speed, at least for certain regimes. EE Times reported in 2023 that Groq demonstrated Llama-2 70B inference at ~240 tokens/s per user on a cloud-based dev system described as 10 racks and 64 chips, using the company’s 1st-gen silicon introduced several years earlier. Separate Groq commentary around independent benchmarking cites results showing ~241 tokens/s throughput and ~0.8s time to receive 100 output tokens for a Llama-2 70B API configuration, positioning the platform as a step-change in “available speed” for certain interactive use cases. These figures do not settle total cost-of-ownership versus GPUs or hyperscaler ASICs, but they establish that Groq’s system-level architecture can deliver strong single-user throughput and latency on large models when properly partitioned and scheduled. GroqCloud is the commercial wrapper that packages this hardware/software stack as “tokens-as-a-service,” aiming to make Groq adoption feel like switching API endpoints rather than adopting new silicon. Groq’s documentation states its API is designed to be “mostly compatible” with OpenAI client libraries, and its pricing page provides model-specific token rates, published speeds (tokens/s), prompt caching discounts, and batch processing discounts. For example, pricing lists inputs as low as $0.05 per 1M tokens and outputs as low as $0.08 per 1M tokens for certain smaller LLM configurations, with higher prices for larger models and long-context or MoE variants; it also advertises prompt caching with a 50% discount on cached input tokens for certain models and a batch API offering 50% lower cost for asynchronous processing windows. These mechanics are economically important because they demonstrate Groq’s go-to-market is not simply “sell chips,” but “sell predictable unit economics per token,” with tooling (batch, caching) that directly targets inference cost drivers (reused prompts, throughput smoothing, and asynchronous workloads). The cloud footprint and distribution partnerships indicate Groq has been building an inference-native “edge within the cloud” strategy rather than competing head-on with hyperscalers on breadth of services. A 2025 Groq newsroom release describes a European deployment in Helsinki with Equinix, positioned as latency reduction and data governance for European customers, and explicitly references Equinix Fabric enabling private connectivity to GroqCloud over public, private, or sovereign infrastructure. The same release enumerates additional capacity in the U.S. (Equinix, DataBank), Canada (Bell Canada), and Saudi Arabia (HUMAIN), and states these sites collectively served more than 20M tokens/s across Groq’s global network at that time. That supply-side metric matters because it provides a directional sense that Groq is scaling capacity as a network, not merely as a chip vendor. Customer disclosure is inherently limited because Groq is private and many enterprise deployments are not public, but Groq’s marketing materials and partnerships provide signals about demand vectors. The company’s public website displays logos of large consumer and enterprise brands (e.g., Dropbox, Vercel, Chevron, Volkswagen, Canva, Robinhood, Riot Games, Workday, Ramp) and includes a published customer quote claiming a 7.41x chat speed increase and an 89% cost reduction after moving to GroqCloud, followed by a tripling of token consumption. While marketing claims should be treated as case-specific and not generalized, they indicate that Groq is targeting both AI-native developers (who measure success by latency and cost-per-token) and enterprise buyers (who care about predictable performance and governance). Supplier and dependency mapping for Groq spans 3 layers: silicon production, system integration, and cloud infrastructure. On silicon, third-party analysis indicates GlobalFoundries 14nm for the 1st-gen Groq chip, implying a supply chain less constrained by the most capacity-tight leading-edge nodes and advanced packaging bottlenecks that dominate high-end GPU supply (HBM stacks, CoWoS-type packaging constraints). If accurate, this is strategically meaningful because it suggests Groq capacity expansion could be gated more by conventional wafer supply, board assembly, and data center power than by the same HBM/advanced packaging scarcity that has constrained top-tier GPU ramp cycles. On systems and cloud, Groq’s own releases identify colocation and connectivity partners (Equinix, DataBank, Bell Canada) and a Middle East partner (HUMAIN), implying dependencies on data center real estate, power availability, and network connectivity, alongside procurement of standard server components, NICs/switching, racks, and cooling infrastructure. The Groq design narrative also emphasizes air cooling and reduced need for complex power/cooling infrastructure, which—if realized in deployments—can widen the set of feasible hosting locations and lower deployment friction relative to liquid-cooled, very high power density GPU racks. Against that backdrop, the strategic rationale for NVIDIA acquiring Groq can be framed as a set of overlapping objectives: inference silicon optionality, architectural hedging, competitive defense, and supply chain diversification, with the carve-out of GroqCloud signaling a preference to avoid direct cloud competition and to focus on IP and product portfolio control rather than operating a capital-intensive token-serving business. The deal, if confirmed, would occur at a valuation step-up of ~190% versus Groq’s reported ~$6.9B private valuation in the September $750M round, reinforcing that any acquisition logic would be predominantly strategic rather than a conventional financial multiple arbitrage. The most compelling strategic driver is inference. Training has historically been the center of gravity for cutting-edge GPU demand, but inference volume is structurally larger and more distributed as deployments scale, with economics dominated by cost-per-token, latency guarantees, and utilization under spiky demand. Inference workloads also create a strategic vulnerability for NVIDIA: hyperscalers and large platforms can justify bespoke ASICs (TPU, Trainium/Inferentia, Maia-class efforts) because inference is stable, repeatable, and can amortize software investment at massive scale. Groq’s core proposition—deterministic, compiler-scheduled inference with predictable latency—aligns directly with the segment where GPU generality is least valued and where “good enough” programmability plus superior unit economics can win share. Acquiring Groq would allow NVIDIA to own a credible inference-native architecture rather than relying solely on GPUs and software optimization to defend that segment. Competitive defense logic is also plausible. Groq occupies a specific competitive wedge: low-latency, high-throughput interactive inference, delivered via a simple API abstraction that reduces switching cost. That wedge directly pressures GPU inference margins in the long run because it makes inference price/performance comparisons more transparent at the token level, and it targets a developer persona that historically defaulted to CUDA-first ecosystems. Even if NVIDIA’s current-generation systems can achieve very high tokens/s per user with extensive optimization, the strategic risk is that competing architectures normalize the idea that inference is best served by special-purpose silicon with a simpler programming model, weakening CUDA lock-in at the application layer. NVIDIA has actively demonstrated that Blackwell-era systems can exceed 1,000 tokens/s per user in benchmarked configurations, but that performance leadership does not automatically translate to lowest cost-per-token across the full range of batch sizes, latency targets, and deployment environments. Groq’s existence as a credible alternative architecture forces NVIDIA to keep defending inference economics rather than only raw performance leadership. The “technology acquisition” rationale is unusually strong in this specific case because Groq’s differentiator is not a single block of silicon IP but an end-to-end methodology: compiler-led static scheduling, deterministic networking, and a system architecture designed around tensor-parallel inference rather than throughput-maximizing batch inference. NVIDIA’s stack is already compiler-heavy (TensorRT, Triton, CUDA graphs, kernel fusion, speculative decoding techniques), but GPUs remain dynamically scheduled devices with complex memory hierarchies and stochastic latency behaviors under contention. Groq’s approach provides an alternate design point: treating the entire inference execution (compute plus communication) as a statically schedulable program. In principle, that IP could be valuable even if Groq silicon itself is not adopted at massive scale, because it can inform how NVIDIA builds future inference-optimized products, compilers, and networking fabrics, especially as distributed inference with large models makes communication a first-order performance determinant. Supply chain diversification is a non-obvious but potentially important driver. If Groq’s mainstream product generation is truly based on a mature process node and avoids HBM, then the scaling constraints look different than those of state-of-the-art GPUs. NVIDIA’s ability to meet incremental demand has been tightly coupled to advanced packaging and HBM supply, and those constraints can remain binding even when wafer supply is available. An inference ASIC architecture that relies primarily on on-chip SRAM and scales by adding chips—while not costless—could reduce dependence on HBM availability and advanced packaging capacity, enabling NVIDIA to ship “inference capacity” in higher absolute volumes or into geographies and customer segments where the highest-end GPUs are economically or logistically difficult to deploy. This could be particularly relevant for latency-sensitive inference deployed in regional colocation footprints rather than centralized hyperscale campuses. The carve-out of GroqCloud, if accurate, is itself a strategic signal about NVIDIA’s priorities. Operating a token-serving cloud at scale is capital intensive, structurally lower margin than silicon IP rents, and creates channel conflict with hyperscalers and CSP partners who are core NVIDIA customers. NVIDIA has generally positioned its cloud offerings through partnerships rather than as a direct hyperscale competitor. Excluding GroqCloud would preserve neutrality with CSPs and avoid inheriting multi-region data residency obligations and partner contracts, while still allowing NVIDIA to acquire Groq’s silicon, compiler technology, and engineering talent. At the same time, excluding GroqCloud would also mean NVIDIA would not automatically acquire the commercial proof-point of Groq’s unit economics or the customer contracts that validate product-market fit at scale, increasing the importance of diligence on whether Groq’s cloud pricing is structurally profitable or partially subsidized by fundraising. There is also a “preemptive acquisition” angle. The reporting identifies recent investors in Groq’s latest round including large financial institutions and strategic/industry players. In that context, Groq represents an asset that could plausibly have been acquired by a competitor (AMD/Intel) or by a hyperscaler seeking to accelerate inference independence. NVIDIA acquiring Groq could be a defensive move to prevent a credible inference-native architecture from being weaponized by a rival with deep distribution. Even if GroqCloud is carved out, controlling the silicon roadmap and compiler IP would meaningfully constrain Groq’s ability to evolve into a standalone competitor, unless the carved-out entity retains long-term rights to the hardware and software stack. However, the strategic case is not one-sided; there are meaningful risks and potential contradictions that would need to be reconciled for the transaction to be value-accretive on a multi-year horizon. 1st, Groq’s architecture appears to rely on scaling out chip count to achieve capacity, which introduces system cost, networking complexity, and physical footprint considerations. The absence of external memory and limited on-chip SRAM implies very large models require substantial chip parallelism, and the economics then depend heavily on chip cost, yield, power efficiency, and interconnect overhead. SemiAnalysis explicitly frames Groq as trading space for time and raises questions about token economics and whether publicly advertised pricing reflects fully loaded costs or market share capture. 2nd, integration risk is non-trivial. Groq’s compiler-led deterministic model is philosophically and practically different from CUDA’s dominant programming and execution model. A poorly executed integration could create internal product confusion, dilute engineering focus, or alienate developers if the combined stack fragments. 3rd, there is cannibalization risk. If Groq-class inference silicon undercuts GPU inference economics, NVIDIA could face internal margin trade-offs, even if the goal is to defend share against hyperscaler ASICs. Cannibalization can still be rational if it prevents larger share loss, but it would require crisp portfolio segmentation and go-to-market discipline. The presence of NVIDIA’s own rapidly improving inference performance complicates the “need” for Groq but does not eliminate the “option value.” NVIDIA has demonstrated benchmark-leading tokens/s per user on Blackwell-based systems, suggesting that raw interactive throughput is not necessarily the limiting factor for NVIDIA’s product line. The more enduring strategic question is unit economics and architectural control: whether future inference demand is better monetized through general-purpose GPUs plus software optimization, or whether a bifurcated product portfolio (training GPUs plus inference-native ASICs) becomes necessary to defend total AI compute wallet share as hyperscaler ASIC penetration increases. Acquiring Groq could be a decisive move to ensure NVIDIA participates in both regimes rather than betting exclusively on GPUs to win inference forever. What is “special” about Groq’s technology relative to a typical accelerator roadmap is the tight coupling of determinism, compilation, and networking into a single scheduling problem. The LPU narrative emphasizes deterministic compute and networking, static scheduling, and direct chip-to-chip coordination that allows “hundreds” (more precisely, 100s) of chips to behave like a single scheduled resource. The architecture also explicitly targets tensor-parallel, latency-optimized distribution rather than pure data-parallel throughput scaling, which matters for real-time applications where a single response must arrive quickly rather than many requests being processed in bulk. The implication is that Groq is optimized for the time-to-first-token and steady token streaming behavior that defines user experience in interactive LLMs, and it attempts to achieve that without relying on large batch sizes that can degrade latency. From a portfolio manager’s perspective, the most important interpretation is that an NVIDIA-Groq combination would likely be less about “NVIDIA needs more inference speed” and more about controlling the architectural trajectory of inference acceleration and removing a fast-improving, developer-friendly competitor from the market. The carve-out of GroqCloud would reinforce that the transaction is aimed at IP, talent, and product optionality, not acquiring a cloud revenue stream. The valuation step-up implied by $20B versus $6.9B would therefore be justified only if the acquired assets materially reduce long-term competitive risk (hyperscaler ASIC displacement, inference margin compression) or enable new monetization vectors (inference ASIC product line, supply chain de-bottlenecking, improved software determinism) that would be difficult to achieve on a comparable timeline via internal R&D.

TheValueist

102,145 次观看 • 7 个月前

Use this prompt in OpenClaw to create your own AI agent command center that syncs up your life like Tony Stark's Jarvis in Iron Man. Adapt the specifics (agent names, data sources, branding) below to your own setup. Prompt: Build me a mission control dashboard for my OpenClaw AI agent system. Stack: Next.js 15 (App Router) + Convex (real-time backend) + Tailwind CSS v4 + Framer Motion + ShadCN UI + Lucide icons. TypeScript throughout. This is the command center where I monitor and control my autonomous AI agent(s) running on OpenClaw. The agent operates 24/7 on a Mac Mini, connected to Telegram/Discord, running cron jobs, spawning sub-agents, and reading/writing to a filesystem-based memory and state system. Dark mode only. Ultra-premium aesthetic, think Iron Man's JARVIS HUD meets a Bloomberg terminal. Subtle glass effects (backdrop-blur-xl, bg-white/[0.03]), no heavy gradients or glow. Rounded corners (16-20px on cards). Framer Motion for page transitions, stagger animations on card grids, spring physics on interactions. Mobile-first responsive. Never cookie-cutter. ## Architecture The dashboard reads live data from TWO sources: 1. **Convex**: real-time database for structured data (tasks, contacts, content drafts, calendar events, activity logs) 2. **Local API routes** (`/api/*`): read files from the agent's workspace filesystem at `~/.openclaw/workspace/` and return JSON. This is how live system state flows into the dashboard. ## Pages & Views (8 nav items, some with tab sub-views) ### 1. HOME (`/`) Dashboard overview. Grid of live status cards: - **System Health**: read from `/api/system-state` (parses `state/servers.json`). Show each service with UP/DOWN indicator, port, last check time. - **Agent Status**: read from `/api/agents` (parses `agents/registry.json` + agent workspace files). Show active agent count, healthy/unhealthy ratio, active sub-agent count from OpenClaw sessions API. - **Cron Health**: read from `/api/cron-health` (parses `state/crons.json`). Table of all scheduled jobs with name, schedule, last status (green/red dot), consecutive errors. - **Revenue Tracker**: read from `/api/revenue` (parses `state/revenue.json`). Current revenue, monthly burn, net. - **Content Pipeline**: read from `/api/content-pipeline` (parses `content/queue.md`). Kanban-style: Draft | Review | Approved | Published counts. - **Quick Stats**: total tasks, pending approvals, active sessions, uptime. All panels auto-refresh every 15 seconds. Live indicator dot + "AUTO 15S" badge in header. ### 2. OPS (`/ops`) with 3 tabs: Operations | Tasks | Calendar **Operations tab:** Full operational view. Server health table, branch status (from `state/branch-check.json`), observations feed (from `state/observations.md`), system priorities (from `shared-context/priorities.md`). **Tasks tab:** Strategic task suggestion system. API route `/api/suggested-tasks` reads/writes `state/suggested-tasks.json`. Cards grouped by category (Revenue, Product, Community, Content, Operations, Clients, Trading, Brand) with emoji headers. Each card shows title, reasoning, next action, priority badge, effort badge, approve/reject buttons. Filter bar by status and category. **Calendar tab:** Weekly calendar view from Convex `calendarEvents` table. Drag-to-create, color-coded by type, time slots. ### 3. AGENTS (`/agents`) with 2 tabs: Agents | Models **Agents tab:** Card grid of all registered agents from `/api/agents`. Each card shows name, role, model, level (L1-L4), status. Cards are CLICKABLE: expanding into a detail panel showing: - Agent personality (reads their SOUL .md) - Capabilities and rules (reads their RULES .md) - Sub-agents they can spawn - Recent outputs (reads from `shared-context/agent-outputs/`) **Models tab:** Model inventory table showing all available models, their routing (which tasks go to which model), costs, and failover chains. ### 4. CHAT (`/chat`): 2 tabs: Chat | Command **Chat tab:** Chat interface to communicate with the agent. Left sidebar shows session list (from `/api/chat-history` reading .jsonl transcript files). Main area shows messages with role-aligned bubbles (user right, assistant left), date separators, channel badges (telegram/discord/webchat). Input bar with send button + voice input (Web Speech API with SpeechRecognition). Messages sent via `/api/chat-send` which queues to a file the agent reads. **Command tab:** Quick command interface for common operations. ### 5. CONTENT (`/content`) Content pipeline management. Read from Convex `contentDrafts` table AND `/api/content-pipeline`. Show drafts in kanban columns. Each card shows title, platform target, draft text preview, status, created date. Edit/approve/reject actions. ### 6. COMMS (`/comms`) with 2 tabs: Comms | CRM **Comms tab:** Communication hub showing recent Discord digest, Telegram messages, notification history. **CRM tab:** Client pipeline kanban (Prospect → Contacted → Meeting → Proposal → Active). API route `/api/clients` reads markdown files from `clients/` directory. Each card shows client name, status, contacts, last interaction, next action. ### 7. KNOWLEDGE (`/knowledge`) with 2 tabs: Knowledge | Ecosystem **Knowledge tab:** Searchable knowledge base. Global search across all workspace files using `/api/knowledge` endpoint. **Ecosystem tab:** Product grid showing all products/apps in the ecosystem. Each card shows product name, status (Active/Development/Concept), health indicator, key metrics. Cards link to `/ecosystem/[slug]` detail pages with tabbed views (Overview, Brand, Community, Content, Legal, Product, Website, Actions). Detail pages read from `/api/ecosystem/[slug]` which parses workspace memory files. ### 8. CODE (`/code`) Code pipeline view. Shows repositories from `/api/repos` (scans ~/Desktop/Projects/ for git repos). Each repo card shows name, branch, last commit, dirty file count, language breakdown. Detail view at `/api/repos/detail` shows recent commits, file tree, open PRs. ## Navigation Top horizontal nav bar, NOT sidebar. All 8 items visible at all viewport widths. Use `flex` layout with `flex-1` items. Text size uses `clamp(0.45rem, 0.75vw, 0.6875rem)` for fluid scaling. Active item gets `text-primary bg-primary/[0.06]` static highlight (no sliding animation). Agent/app name visible at md+ breakpoints (`hidden md:inline`). Tab sub-views use a reusable `TabBar` component with pill/glass styling and Framer Motion `layoutId` transitions. Tab state stored in URL via `?tab=` search params. ## API Routes (all under `src/app/api/`) Each API route reads from the agent's workspace filesystem and returns JSON: - `/api/system-state` → reads `state/servers.json`, `state/branch-check.json` - `/api/agents` → reads `agents/registry.json`, agent SOUL .md files - `/api/agents/[id]` → reads specific agent's SOUL .md, RULES .md, outputs - `/api/cron-health` → reads `state/crons.json` - `/api/revenue` → reads `state/revenue.json` - `/api/content-pipeline` → parses `content/queue.md` (markdown with status markers) - `/api/suggested-tasks` → GET (read) / POST (approve/reject) on `state/suggested-tasks.json` - `/api/observations` → reads `state/observations.md` - `/api/priorities` → reads `shared-context/priorities.md` - `/api/chat-history` → reads .jsonl transcript files with pagination/search/channel filter - `/api/chat-send` → writes to queue file - `/api/clients` → reads markdown files from `clients/` directory - `/api/ecosystem/[slug]` → reads memory files for specific ecosystem - `/api/repos` → scans project directories for git repos - `/api/health` → returns status, uptime, memory usage, Convex connectivity All filesystem paths should be configurable via environment variable (default: `~/.openclaw/workspace/`). ## Convex Schema Define tables for: activities, calendarEvents, tasks, contacts, contentDrafts, ecosystemProducts. Include seed scripts (`convex/seed.ts`) to populate initial data. ## Key Design Rules - Mobile-first, test at 320px minimum - Font sizes 10-14px for body text, everything must fit naturally at small viewports - Cards use consistent border radius (16-20px) - Glass cards: `bg-white/[0.03] backdrop-blur-xl border border-white/[0.06]` - No heavy blur blobs or grain overlays - Stagger animations on card grids (0.05s delay per item) - Skeleton loading states for all async data - Custom scrollbar styling - Empty states with helpful messaging - All text must use Inter or system font stack - Never mix sharp and rounded corners in the same view - Premium = lighter feel, more whitespace, less visual noise ## File Structure ``` src/ app/ page.tsx, layout.tsx, providers.tsx agents/page.tsx calendar/page.tsx chat/page.tsx code/page.tsx comms/page.tsx content/page.tsx ecosystem/page.tsx, ecosystem/[slug]/page.tsx knowledge/page.tsx ops/page.tsx api/[...all routes above] components/ nav.tsx tab-bar.tsx dashboard-overview.tsx ops-view.tsx, suggested-tasks-view.tsx agents-view.tsx, models-view.tsx chat-center-view.tsx, voice-input.tsx content-view.tsx comms-view.tsx, crm-view.tsx knowledge-base.tsx, ecosystem-view.tsx code-pipeline.tsx activity-feed.tsx, calendar-view.tsx ui/ (ShadCN primitives) hooks/ lib/ convex/ schema.ts functions for each table seed.ts ``` Build the complete application. Every component, every API route, every Convex function. Production-quality code and premium design, not stubs. Dark mode only. Make it look incredibly beautiful and premium, no cookie cutter UI / AI slop.
1:14

Use this prompt in OpenClaw to create your own AI agent command center that syncs up your life like Tony Stark's Jarvis in Iron Man. Adapt the specifics (agent names, data sources, branding) below to your own setup. Prompt: Build me a mission control dashboard for my OpenClaw AI agent system. Stack: Next.js 15 (App Router) + Convex (real-time backend) + Tailwind CSS v4 + Framer Motion + ShadCN UI + Lucide icons. TypeScript throughout. This is the command center where I monitor and control my autonomous AI agent(s) running on OpenClaw. The agent operates 24/7 on a Mac Mini, connected to Telegram/Discord, running cron jobs, spawning sub-agents, and reading/writing to a filesystem-based memory and state system. Dark mode only. Ultra-premium aesthetic, think Iron Man's JARVIS HUD meets a Bloomberg terminal. Subtle glass effects (backdrop-blur-xl, bg-white/[0.03]), no heavy gradients or glow. Rounded corners (16-20px on cards). Framer Motion for page transitions, stagger animations on card grids, spring physics on interactions. Mobile-first responsive. Never cookie-cutter. ## Architecture The dashboard reads live data from TWO sources: 1. **Convex**: real-time database for structured data (tasks, contacts, content drafts, calendar events, activity logs) 2. **Local API routes** (`/api/*`): read files from the agent's workspace filesystem at `~/.openclaw/workspace/` and return JSON. This is how live system state flows into the dashboard. ## Pages & Views (8 nav items, some with tab sub-views) ### 1. HOME (`/`) Dashboard overview. Grid of live status cards: - **System Health**: read from `/api/system-state` (parses `state/servers.json`). Show each service with UP/DOWN indicator, port, last check time. - **Agent Status**: read from `/api/agents` (parses `agents/registry.json` + agent workspace files). Show active agent count, healthy/unhealthy ratio, active sub-agent count from OpenClaw sessions API. - **Cron Health**: read from `/api/cron-health` (parses `state/crons.json`). Table of all scheduled jobs with name, schedule, last status (green/red dot), consecutive errors. - **Revenue Tracker**: read from `/api/revenue` (parses `state/revenue.json`). Current revenue, monthly burn, net. - **Content Pipeline**: read from `/api/content-pipeline` (parses `content/queue.md`). Kanban-style: Draft | Review | Approved | Published counts. - **Quick Stats**: total tasks, pending approvals, active sessions, uptime. All panels auto-refresh every 15 seconds. Live indicator dot + "AUTO 15S" badge in header. ### 2. OPS (`/ops`) with 3 tabs: Operations | Tasks | Calendar **Operations tab:** Full operational view. Server health table, branch status (from `state/branch-check.json`), observations feed (from `state/observations.md`), system priorities (from `shared-context/priorities.md`). **Tasks tab:** Strategic task suggestion system. API route `/api/suggested-tasks` reads/writes `state/suggested-tasks.json`. Cards grouped by category (Revenue, Product, Community, Content, Operations, Clients, Trading, Brand) with emoji headers. Each card shows title, reasoning, next action, priority badge, effort badge, approve/reject buttons. Filter bar by status and category. **Calendar tab:** Weekly calendar view from Convex `calendarEvents` table. Drag-to-create, color-coded by type, time slots. ### 3. AGENTS (`/agents`) with 2 tabs: Agents | Models **Agents tab:** Card grid of all registered agents from `/api/agents`. Each card shows name, role, model, level (L1-L4), status. Cards are CLICKABLE: expanding into a detail panel showing: - Agent personality (reads their SOUL .md) - Capabilities and rules (reads their RULES .md) - Sub-agents they can spawn - Recent outputs (reads from `shared-context/agent-outputs/`) **Models tab:** Model inventory table showing all available models, their routing (which tasks go to which model), costs, and failover chains. ### 4. CHAT (`/chat`): 2 tabs: Chat | Command **Chat tab:** Chat interface to communicate with the agent. Left sidebar shows session list (from `/api/chat-history` reading .jsonl transcript files). Main area shows messages with role-aligned bubbles (user right, assistant left), date separators, channel badges (telegram/discord/webchat). Input bar with send button + voice input (Web Speech API with SpeechRecognition). Messages sent via `/api/chat-send` which queues to a file the agent reads. **Command tab:** Quick command interface for common operations. ### 5. CONTENT (`/content`) Content pipeline management. Read from Convex `contentDrafts` table AND `/api/content-pipeline`. Show drafts in kanban columns. Each card shows title, platform target, draft text preview, status, created date. Edit/approve/reject actions. ### 6. COMMS (`/comms`) with 2 tabs: Comms | CRM **Comms tab:** Communication hub showing recent Discord digest, Telegram messages, notification history. **CRM tab:** Client pipeline kanban (Prospect → Contacted → Meeting → Proposal → Active). API route `/api/clients` reads markdown files from `clients/` directory. Each card shows client name, status, contacts, last interaction, next action. ### 7. KNOWLEDGE (`/knowledge`) with 2 tabs: Knowledge | Ecosystem **Knowledge tab:** Searchable knowledge base. Global search across all workspace files using `/api/knowledge` endpoint. **Ecosystem tab:** Product grid showing all products/apps in the ecosystem. Each card shows product name, status (Active/Development/Concept), health indicator, key metrics. Cards link to `/ecosystem/[slug]` detail pages with tabbed views (Overview, Brand, Community, Content, Legal, Product, Website, Actions). Detail pages read from `/api/ecosystem/[slug]` which parses workspace memory files. ### 8. CODE (`/code`) Code pipeline view. Shows repositories from `/api/repos` (scans ~/Desktop/Projects/ for git repos). Each repo card shows name, branch, last commit, dirty file count, language breakdown. Detail view at `/api/repos/detail` shows recent commits, file tree, open PRs. ## Navigation Top horizontal nav bar, NOT sidebar. All 8 items visible at all viewport widths. Use `flex` layout with `flex-1` items. Text size uses `clamp(0.45rem, 0.75vw, 0.6875rem)` for fluid scaling. Active item gets `text-primary bg-primary/[0.06]` static highlight (no sliding animation). Agent/app name visible at md+ breakpoints (`hidden md:inline`). Tab sub-views use a reusable `TabBar` component with pill/glass styling and Framer Motion `layoutId` transitions. Tab state stored in URL via `?tab=` search params. ## API Routes (all under `src/app/api/`) Each API route reads from the agent's workspace filesystem and returns JSON: - `/api/system-state` → reads `state/servers.json`, `state/branch-check.json` - `/api/agents` → reads `agents/registry.json`, agent SOUL .md files - `/api/agents/[id]` → reads specific agent's SOUL .md, RULES .md, outputs - `/api/cron-health` → reads `state/crons.json` - `/api/revenue` → reads `state/revenue.json` - `/api/content-pipeline` → parses `content/queue.md` (markdown with status markers) - `/api/suggested-tasks` → GET (read) / POST (approve/reject) on `state/suggested-tasks.json` - `/api/observations` → reads `state/observations.md` - `/api/priorities` → reads `shared-context/priorities.md` - `/api/chat-history` → reads .jsonl transcript files with pagination/search/channel filter - `/api/chat-send` → writes to queue file - `/api/clients` → reads markdown files from `clients/` directory - `/api/ecosystem/[slug]` → reads memory files for specific ecosystem - `/api/repos` → scans project directories for git repos - `/api/health` → returns status, uptime, memory usage, Convex connectivity All filesystem paths should be configurable via environment variable (default: `~/.openclaw/workspace/`). ## Convex Schema Define tables for: activities, calendarEvents, tasks, contacts, contentDrafts, ecosystemProducts. Include seed scripts (`convex/seed.ts`) to populate initial data. ## Key Design Rules - Mobile-first, test at 320px minimum - Font sizes 10-14px for body text, everything must fit naturally at small viewports - Cards use consistent border radius (16-20px) - Glass cards: `bg-white/[0.03] backdrop-blur-xl border border-white/[0.06]` - No heavy blur blobs or grain overlays - Stagger animations on card grids (0.05s delay per item) - Skeleton loading states for all async data - Custom scrollbar styling - Empty states with helpful messaging - All text must use Inter or system font stack - Never mix sharp and rounded corners in the same view - Premium = lighter feel, more whitespace, less visual noise ## File Structure ``` src/ app/ page.tsx, layout.tsx, providers.tsx agents/page.tsx calendar/page.tsx chat/page.tsx code/page.tsx comms/page.tsx content/page.tsx ecosystem/page.tsx, ecosystem/[slug]/page.tsx knowledge/page.tsx ops/page.tsx api/[...all routes above] components/ nav.tsx tab-bar.tsx dashboard-overview.tsx ops-view.tsx, suggested-tasks-view.tsx agents-view.tsx, models-view.tsx chat-center-view.tsx, voice-input.tsx content-view.tsx comms-view.tsx, crm-view.tsx knowledge-base.tsx, ecosystem-view.tsx code-pipeline.tsx activity-feed.tsx, calendar-view.tsx ui/ (ShadCN primitives) hooks/ lib/ convex/ schema.ts functions for each table seed.ts ``` Build the complete application. Every component, every API route, every Convex function. Production-quality code and premium design, not stubs. Dark mode only. Make it look incredibly beautiful and premium, no cookie cutter UI / AI slop.

klöss

201,471 次观看 • 5 个月前