Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Developers have been using the Decisions API to: • Route requests to the right model, tool, or agent. • Turn scaled inputs into useful labels, rankings, and scores. • Analyze images, compare visual content, or identify key video frames. • Choose buttons, navigate forms, or determine actions from screenshots.... show more

OpenAI Developers

459,383 subscribers

160,290 views • 13 days ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

9 Comments

OpenAI Developers13 days ago

Powered by GPT-6 Luna, it accepts text and image inputs and supports 3 kinds of outputs: Predicates: Estimate the probability that a statement is true. Choices: Select from predefined options, with confidence scores. Scores: Evaluate an input against a numeric range.

OpenAI Developers13 days ago

Let your app choose the right model, tool, or action in near real-time with Decisions API, now available to all developers in public beta. The Decisions API makes decisions up to 10x faster than GPT-6 Luna through the Responses API.

OpenAI Developers13 days ago

You can now try the Decisions API in public beta:

Patrick Schaub13 days ago

"Route requests to the right model, tool, or agent." @theo it's time to challenge this again?

Adam13 days ago

Please don't force me to use this. This will never work well.

Ishu Agrawal13 days ago

How would Luna have any idea of the “right model” to route to

Jason Rai13 days ago

Video input support please

John Herzog13 days ago

flagging risky tool calls before they run could save agent builders a lot of cleanup

Hidden Hall13 days ago

Will this be available in aws?

Related Videos

Claude Code just shipped v2.1.69 with a new built-in skill: /claude-api It detects your language, picks the right surface (direct API, tool use, or Agent SDK), and loads the relevant reference docs directly into context. Covers streaming, batches, files API, structured outputs, tool use patterns, MCP, also sets model defaults so you're not guessing. Useful when you're building on top of Claude and don't want to context-switch to the docs every 5 minutes.

Claude Code just shipped v2.1.69 with a new built-in skill: /claude-api It detects your language, picks the right surface (direct API, tool use, or Agent SDK), and loads the relevant reference docs directly into context. Covers streaming, batches, files API, structured outputs, tool use patterns, MCP, also sets model defaults so you're not guessing. Useful when you're building on top of Claude and don't want to context-switch to the docs every 5 minutes.

Daniel San

152,831 views • 7 months ago

here's another little tool i've been working on called "typoverse." i have thousands of fonts but i find myself always using the same ones. and i am simply not going to manually tag or categorize fonts either. so i built a tool that compares all my fonts to each other and maps them based on similarity. would this be useful to anyone?

here's another little tool i've been working on called "typoverse." i have thousands of fonts but i find myself always using the same ones. and i am simply not going to manually tag or categorize fonts either. so i built a tool that compares all my fonts to each other and maps them based on similarity. would this be useful to anyone?

River Marchand

28,465 views • 7 months ago

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find trending ETH tokens in just a few simple steps! Whether you're new to Ethereum or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on ETH and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next ETH gem! Cherry Trending:

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find trending ETH tokens in just a few simple steps! Whether you're new to Ethereum or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on ETH and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next ETH gem! Cherry Trending:

Cherry Trending

79,157 views • 2 years ago

Chrome DevTools now supports *individual* network request throttling! Folks have been asking for this for years! DevTools now allows developers to simulate slow network conditions for specific requests rather than the entire page. This helps in testing how a web application performs and handles issues when specific resources (like images, scripts, or API calls) are slow to load.

Chrome DevTools now supports individual network request throttling! Folks have been asking for this for years! DevTools now allows developers to simulate slow network conditions for specific requests rather than the entire page. This helps in testing how a web application performs and handles issues when specific resources (like images, scripts, or API calls) are slow to load.

Addy Osmani

281,774 views • 9 months ago

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big SUI tokens in just a few simple steps! Whether you're new to SUI or already familiar with the blockchain, this guide will help you discover the hottest tokens on the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on SUI and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next SUI gem!

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big SUI tokens in just a few simple steps! Whether you're new to SUI or already familiar with the blockchain, this guide will help you discover the hottest tokens on the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on SUI and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next SUI gem!

Cherry Trending

54,183 views • 2 years ago

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big Tron tokens in just a few simple steps! Whether you're new to Tron or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on Tron and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next Tron gem! Cherry Trending:

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big Tron tokens in just a few simple steps! Whether you're new to Tron or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on Tron and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next Tron gem! Cherry Trending:

Cherry Trending

88,376 views • 2 years ago

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big BSC (Binance Smart Chain) tokens in just a few simple steps! Whether you're new to BSC or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on BSC and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next big BSC gem!

🍒Good evening, #Cherry Team and Questers! In this video, we're going to show you how to find the next big BSC (Binance Smart Chain) tokens in just a few simple steps! Whether you're new to BSC or a seasoned pro, this guide will help you discover the hottest tokens in the market. We'll walk you through using Cherry Trending, a powerful tool that tracks trending tokens on BSC and other major blockchains. Plus, you'll learn how to analyze key data to identify potential winners. Stay tuned and get ready to uncover the next big BSC gem!

Cherry Trending

47,042 views • 2 years ago

How to use 50+ API keys (models) for FREE on OpenClaw API??? - go to - login or register your account - click on "more models" - click on "use case" and select what you need it for - choose the model and open it - click on "view code" → "Generate API key" many models don't allow direct deploy, so use the "view code" button to generate API access basically Nvidia NIM gives you the ability to test almost any model from their list for FREE some of them are not worse than GPT 5.2 or Claude Opus 4.6, some might even perform better depending on the task how to understand if a model is efficient and compare it with others??? - go to - type the model name in search - click on "benchmarks" - you’ll see performance tests and rankings this way you can easily compare free models with paid ones of course there are RPM limits, on many models it’s around ~40 requests per minute each model is different, after generating the API key, RPM limits are shown in the top-right corner nothing stops you from using them, many work perfectly fine, super solid option for first tests and for learning OpenClaw or any other system where you need an AI API model

How to use 50+ API keys (models) for FREE on OpenClaw API??? - go to - login or register your account - click on "more models" - click on "use case" and select what you need it for - choose the model and open it - click on "view code" → "Generate API key" many models don't allow direct deploy, so use the "view code" button to generate API access basically Nvidia NIM gives you the ability to test almost any model from their list for FREE some of them are not worse than GPT 5.2 or Claude Opus 4.6, some might even perform better depending on the task how to understand if a model is efficient and compare it with others??? - go to - type the model name in search - click on "benchmarks" - you’ll see performance tests and rankings this way you can easily compare free models with paid ones of course there are RPM limits, on many models it’s around ~40 requests per minute each model is different, after generating the API key, RPM limits are shown in the top-right corner nothing stops you from using them, many work perfectly fine, super solid option for first tests and for learning OpenClaw or any other system where you need an AI API model

Ronin

58,974 views • 7 months ago

10 agent evals every AI engineer should know 1) golden set a frozen set of cases you run after every prompt model or tool change use it as your baseline to see whether the system improved or quietly broke OpenAI Evals helps you build repeatable benchmark sets and compare model changes → 2) llm as judge a second model scores open ended answers against a written rubric use it when there is no exact output to compare against OpenEvals provides ready made evaluators for LLM applications → 3) rubric scoring score correctness tone safety and cost separately one quality number hides the problem DeepEval helps you create custom metrics and score every dimension independently → 4) trajectory eval grade the path the agent took not only the final answer AgentEvals checks agent actions decisions and tool calls across the full trajectory → 5) tool unit tests test every tool with fixed inputs and outputs no model in the loop MCP Inspector helps you inspect and test MCP servers tools and responses separately → 6) regression suite replay previous runs against every new prompt model or toolset then compare the results Promptfoo helps you run repeatable eval suites catch regressions and add checks to CI → 7) a b in prod split real traffic between two versions and compare actual outcomes GrowthBook provides feature flags controlled experiments and product analytics → 8) human review sample real runs and let a person grade them honestly use it to calibrate your automated judge Argilla helps teams collect human feedback review outputs and build better datasets → 9) shadow run let the candidate run on real traffic while its output is shown to nobody use it before a risky rollout Langfuse helps you trace production runs compare candidates and monitor eval results → 10) red team attack the system before somebody else does jailbreaks prompt injection data leaks and tool abuse Garak scans LLM systems for vulnerabilities and unsafe behavior → offline evals tell you it works online evals tell you it still works you probably do not need all ten today start with the two that would have caught your last outage bookmark this

10 agent evals every AI engineer should know 1) golden set a frozen set of cases you run after every prompt model or tool change use it as your baseline to see whether the system improved or quietly broke OpenAI Evals helps you build repeatable benchmark sets and compare model changes → 2) llm as judge a second model scores open ended answers against a written rubric use it when there is no exact output to compare against OpenEvals provides ready made evaluators for LLM applications → 3) rubric scoring score correctness tone safety and cost separately one quality number hides the problem DeepEval helps you create custom metrics and score every dimension independently → 4) trajectory eval grade the path the agent took not only the final answer AgentEvals checks agent actions decisions and tool calls across the full trajectory → 5) tool unit tests test every tool with fixed inputs and outputs no model in the loop MCP Inspector helps you inspect and test MCP servers tools and responses separately → 6) regression suite replay previous runs against every new prompt model or toolset then compare the results Promptfoo helps you run repeatable eval suites catch regressions and add checks to CI → 7) a b in prod split real traffic between two versions and compare actual outcomes GrowthBook provides feature flags controlled experiments and product analytics → 8) human review sample real runs and let a person grade them honestly use it to calibrate your automated judge Argilla helps teams collect human feedback review outputs and build better datasets → 9) shadow run let the candidate run on real traffic while its output is shown to nobody use it before a risky rollout Langfuse helps you trace production runs compare candidates and monitor eval results → 10) red team attack the system before somebody else does jailbreaks prompt injection data leaks and tool abuse Garak scans LLM systems for vulnerabilities and unsafe behavior → offline evals tell you it works online evals tell you it still works you probably do not need all ten today start with the two that would have caught your last outage bookmark this

elune

155,026 views • 2 months ago

He’s right and it’s nothing wrong with what he said Black Americans don’t have the option to pick and choose when to identify as Black, it’s a slap in the face for immigrants to only identify as Black when they can benefit from our sacrifices Or when they need our political backing, immigrants are using democrat tactics to say we must not be Black because we aren’t speaking on what ICE is doing to immigrants

He’s right and it’s nothing wrong with what he said Black Americans don’t have the option to pick and choose when to identify as Black, it’s a slap in the face for immigrants to only identify as Black when they can benefit from our sacrifices Or when they need our political backing, immigrants are using democrat tactics to say we must not be Black because we aren’t speaking on what ICE is doing to immigrants

•

42,901 views • 10 months ago

I wanted to compare three images-to-video tools for a real job: turning a product photo into a short video. I gave the task to my agent with Aident Loadout. Aident AI helped my agent find, select, and use the right services automatically. Exa found the official pages for Runway, Pika, and Luma Dream Machine. Firecrawl read their docs and pricing. Loadout turned the verified details into a one-page guide, then used Fal's fal-ai/nano-banana-2 model to generate the comparison image after showing me the generation details for approval. I did not need separate Firecrawl or Fal subscriptions for this run. Aident credits covered those calls. The output kept the useful constraints visible: image-to-video support, maximum length and resolution, queue or credit limits, published entry price, and source links. That is the part I liked: I stayed inside the agent I was already using while Loadout handled the on-platform tools, the approval checkpoint, and the Audit record.

I wanted to compare three images-to-video tools for a real job: turning a product photo into a short video. I gave the task to my agent with Aident Loadout. Aident AI helped my agent find, select, and use the right services automatically. Exa found the official pages for Runway, Pika, and Luma Dream Machine. Firecrawl read their docs and pricing. Loadout turned the verified details into a one-page guide, then used Fal's fal-ai/nano-banana-2 model to generate the comparison image after showing me the generation details for approval. I did not need separate Firecrawl or Fal subscriptions for this run. Aident credits covered those calls. The output kept the useful constraints visible: image-to-video support, maximum length and resolution, queue or credit limits, published entry price, and source links. That is the part I liked: I stayed inside the agent I was already using while Loadout handled the on-platform tools, the approval checkpoint, and the Audit record.

Shoaib AI

14,536 views • 1 month ago

Here's how to identify a SCAM memecoin using 1 tool: We'll take $SACHI as an example ($8M market cap right now) You need to check just 2 things: • Top holders • Top traders If you see that the Top 10 wallets are empty (low $SOL balance) or marked with ghosts, run away. This means that developers sniped the majority of supply before going live and are dumping their tokens right now. This simple trick will save your money. Trading and analyzing here:

Here's how to identify a SCAM memecoin using 1 tool: We'll take $SACHI as an example ($8M market cap right now) You need to check just 2 things: • Top holders • Top traders If you see that the Top 10 wallets are empty (low $SOL balance) or marked with ghosts, run away. This means that developers sniped the majority of supply before going live and are dumping their tokens right now. This simple trick will save your money. Trading and analyzing here:

Oracle Boar

49,304 views • 10 months ago

HERMES AGENT WITHOUT TOOLS IS A CHATBOT. WITH THEM IT BUILDS 3D TOWERS IN BLENDER, CHECKS STOCK PRICES, AND DRIVES VS CODE. tonbi JUST DROPPED THE FULL GUIDE. module 6 of his 10-part Hermes masterclass. best breakdown of the tool layer anyone has published. what you need to know: TOOLS vs SKILLS vs MCP skills = instructions (markdown, loaded into context) tools = callable functions (Python, agent emits call, Hermes executes) MCP = adapters to external systems (Blender, Stripe, Linear, Notion) every tool has three parts: → the function (does the real work) → the schema (what the model sees to decide when to call it) → the registry (makes the tool exist in the agent) the model never runs the function directly. it emits a structured request (tool name + JSON args). Hermes executes and returns the result. if the tool fails, the error goes back as JSON. the agent recovers instead of crashing. TOOL SETS CONTROL THE SURFACE hermes chat --tool-sets web # only web tools loaded. no files, no terminal. hermes chat --tool-sets safe # read-only: web search, vision, image gen. # no file writes. no terminal. no code execution. mid-session: /tools enable video /tools disable terminal or toggle in the dashboard: hermes dashboard → Skills → Tool Sets MCP SERVERS two transport types: STDIO (local subprocess) or HTTP (remote endpoint). add a server: hermes mcp # or ask: "add the MCP server for Blender" filter tools with include/exclude per server. keep only what you trust. security: → OAuth 2.1 PKCE (no long-lived tokens in config) → package scanning via api.osv. dev before launch → all MCP calls go through approval gates HERMES AS MCP SERVER hermes mcp serve exposes 10 tools via FastMCP. connect VS Code Copilot or Cursor to your running Hermes instance. BUILD YOUR OWN TOOL He built a stock price tool live: → Python function calling Finnhub API → schema with name, description, parameters → registered in tool_sets.py → agent calls it automatically when relevant any repeating API call in your workflow can become a native tool. full Hermes architecture deep-dive in the article 👇

HERMES AGENT WITHOUT TOOLS IS A CHATBOT. WITH THEM IT BUILDS 3D TOWERS IN BLENDER, CHECKS STOCK PRICES, AND DRIVES VS CODE. tonbi JUST DROPPED THE FULL GUIDE. module 6 of his 10-part Hermes masterclass. best breakdown of the tool layer anyone has published. what you need to know: TOOLS vs SKILLS vs MCP skills = instructions (markdown, loaded into context) tools = callable functions (Python, agent emits call, Hermes executes) MCP = adapters to external systems (Blender, Stripe, Linear, Notion) every tool has three parts: → the function (does the real work) → the schema (what the model sees to decide when to call it) → the registry (makes the tool exist in the agent) the model never runs the function directly. it emits a structured request (tool name + JSON args). Hermes executes and returns the result. if the tool fails, the error goes back as JSON. the agent recovers instead of crashing. TOOL SETS CONTROL THE SURFACE hermes chat --tool-sets web # only web tools loaded. no files, no terminal. hermes chat --tool-sets safe # read-only: web search, vision, image gen. # no file writes. no terminal. no code execution. mid-session: /tools enable video /tools disable terminal or toggle in the dashboard: hermes dashboard → Skills → Tool Sets MCP SERVERS two transport types: STDIO (local subprocess) or HTTP (remote endpoint). add a server: hermes mcp # or ask: "add the MCP server for Blender" filter tools with include/exclude per server. keep only what you trust. security: → OAuth 2.1 PKCE (no long-lived tokens in config) → package scanning via api.osv. dev before launch → all MCP calls go through approval gates HERMES AS MCP SERVER hermes mcp serve exposes 10 tools via FastMCP. connect VS Code Copilot or Cursor to your running Hermes instance. BUILD YOUR OWN TOOL He built a stock price tool live: → Python function calling Finnhub API → schema with name, description, parameters → registered in tool_sets.py → agent calls it automatically when relevant any repeating API call in your workflow can become a native tool. full Hermes architecture deep-dive in the article 👇

YanXbt

32,680 views • 4 months ago

Claude Code: Auto mode Auto mode lets Claude safely complete long-running tasks without asking permission for every action. But how does it work? 4 key terms: • Classifier: a separate model that approves or denies each action based on your intent and scope • Filtering: the classifier only sees your messages and Claude's tool calls, not Claude’s reasoning or tool outputs, so it isn't influenced when making decisions • Tier check: read-only and easily-undone actions (like edits inside your repo) skip the classifier; everything else goes through it • Probe: a server-side check that scans incoming content, like web pages, for prompt injection and flags it to Claude I'm working on a longer video that explains how auto mode works and how to configure it. What questions do you have about it?

Claude Code: Auto mode Auto mode lets Claude safely complete long-running tasks without asking permission for every action. But how does it work? 4 key terms: • Classifier: a separate model that approves or denies each action based on your intent and scope • Filtering: the classifier only sees your messages and Claude's tool calls, not Claude’s reasoning or tool outputs, so it isn't influenced when making decisions • Tier check: read-only and easily-undone actions (like edits inside your repo) skip the classifier; everything else goes through it • Probe: a server-side check that scans incoming content, like web pages, for prompt injection and flags it to Claude I'm working on a longer video that explains how auto mode works and how to configure it. What questions do you have about it?

Delba

43,073 views • 4 months ago

Box is a launch partner for Microsoft’s Apps in Copilot launch. The Box AI Agent for Microsoft365 Copilot will be able to surface rich visual previews like slides, graphs, and images right in the conversation and have access to all of the Box MCP server tools. Coming soon to the Box AI Agent in Microsoft 365 Copilot: 🔹 MCP-powered workflows — metadata extraction, document generation, and more 🔹 Visual-first search — view complicated graphs or rich visuals that are buried in your Box content, without having to open files. 🔹 Fewer clarifications, faster decisions - move from chat to action Enable agents to bring apps into the flow of work, while keeping IT in control:

Box is a launch partner for Microsoft’s Apps in Copilot launch. The Box AI Agent for Microsoft365 Copilot will be able to surface rich visual previews like slides, graphs, and images right in the conversation and have access to all of the Box MCP server tools. Coming soon to the Box AI Agent in Microsoft 365 Copilot: 🔹 MCP-powered workflows — metadata extraction, document generation, and more 🔹 Visual-first search — view complicated graphs or rich visuals that are buried in your Box content, without having to open files. 🔹 Fewer clarifications, faster decisions - move from chat to action Enable agents to bring apps into the flow of work, while keeping IT in control:

Box

335,591 views • 7 months ago

Karpathy’s Agentic Engineering finally has proper DevTools! When an agent stops working, the model is only one possible cause. The problem could be a failed tool, a lost connection, an interface update that never appeared, or something earlier in the conversation. CopilotKit🪁 has rebuilt its open-source Inspector around this problem. It sits inside the application and watches the full interaction between the user, the interface, and the agent. When something fails, the Inspector button turns red, names the failure, and opens the spot where it happened. But even a much harder problem is reproducing that failure. This is because agents do not always follow the same path twice. Running the same prompt again may trigger another tool, produce another response, or start with a different application state. Inspector handles this through saved Threads and an isolated Agent Playground. A developer can open the conversation where the problem occurred, choose "Try from here," and copy everything up to that point into the Playground. The original conversation remains unchanged while another response or tool path is tested. Threads can open inside the live application, while assistant messages can jump back to their matching context in Inspector. Everything runs underneath AG-UI, the protocol that carries messages, tool activity, state changes, and other events between the agent and the interface. Those interactions can also feed CopilotKit Intelligence so that when the same issue or useful behavior appears repeatedly, it can become an evidence-backed Insight and a proposed SKILL(.)md file. Developers can review, edit, and approve the improvement before the agent inherits it. The tool also helps developers find a problem, recreate its surrounding context, test another path, and turn repeated lessons into agent improvements. It is open source and included with CopilotKit development builds. Intelligence setup starts with one prompt. Here are the docs: I am testing this extensively and will cover this in more detail soon with a hands-on demo.

Karpathy’s Agentic Engineering finally has proper DevTools! When an agent stops working, the model is only one possible cause. The problem could be a failed tool, a lost connection, an interface update that never appeared, or something earlier in the conversation. CopilotKit🪁 has rebuilt its open-source Inspector around this problem. It sits inside the application and watches the full interaction between the user, the interface, and the agent. When something fails, the Inspector button turns red, names the failure, and opens the spot where it happened. But even a much harder problem is reproducing that failure. This is because agents do not always follow the same path twice. Running the same prompt again may trigger another tool, produce another response, or start with a different application state. Inspector handles this through saved Threads and an isolated Agent Playground. A developer can open the conversation where the problem occurred, choose "Try from here," and copy everything up to that point into the Playground. The original conversation remains unchanged while another response or tool path is tested. Threads can open inside the live application, while assistant messages can jump back to their matching context in Inspector. Everything runs underneath AG-UI, the protocol that carries messages, tool activity, state changes, and other events between the agent and the interface. Those interactions can also feed CopilotKit Intelligence so that when the same issue or useful behavior appears repeatedly, it can become an evidence-backed Insight and a proposed SKILL(.)md file. Developers can review, edit, and approve the improvement before the agent inherits it. The tool also helps developers find a problem, recreate its surrounding context, test another path, and turn repeated lessons into agent improvements. It is open source and included with CopilotKit development builds. Intelligence setup starts with one prompt. Here are the docs: I am testing this extensively and will cover this in more detail soon with a hands-on demo.

Akshay 🚀

12,206 views • 1 month ago

I made a video breaking down AI agent observability and evals, and ran the whole thing on open source. If you build agents, sooner or later you watch one run and realise you have no idea what it just did, or what it cost you. That gap is what observability and evals are for. Least glamorous part of the stack, and the part you end up needing most. 🛠️ treg, OpenRouter for agent tools (4.2k): 🔑 Try treg: 🌟 Github Repo (2k): 💻 Portable memory: ☕ Join our community: Observability is a camera. It records what ran, how long, what it cost. Eval is a referee. It decides whether the run was good enough to ship. I ran it on Treg for the tool calls and Waku Agent for the memory. treg is an OpenRouter for agent tool calls, so your agent carries one API key instead of one per tool. Both open source, so you can read the code rather than take my word for it. Full video in the first reply 👇 You Can Build Anything. You Can Learn Anything. 💪

I made a video breaking down AI agent observability and evals, and ran the whole thing on open source. If you build agents, sooner or later you watch one run and realise you have no idea what it just did, or what it cost you. That gap is what observability and evals are for. Least glamorous part of the stack, and the part you end up needing most. 🛠️ treg, OpenRouter for agent tools (4.2k): 🔑 Try treg: 🌟 Github Repo (2k): 💻 Portable memory: ☕ Join our community: Observability is a camera. It records what ran, how long, what it cost. Eval is a referee. It decides whether the run was good enough to ship. I ran it on Treg for the tool calls and Waku Agent for the memory. treg is an OpenRouter for agent tool calls, so your agent carries one API key instead of one per tool. Both open source, so you can read the code rather than take my word for it. Full video in the first reply 👇 You Can Build Anything. You Can Learn Anything. 💪

Shen Sean Chen

33,494 views • 13 days ago