正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

We just released the Databricks synthetic evaluation SDK! 🎉 We’ve found that synthesizing evals is a great way to hill climb your AI system before you’re able to get labels from domain experts. With this SDK, and our fancy diffing UI, users are able to improve the inner loop... show more

Nikhil Thorat

9,449 subscribers

10,446 次观看 • 1 年前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

3 条评论

Hamel Husain 的头像

Hamel Husain1 年前

@databricks Looks cool! Do you need to be using MLFlow to take advantage of this? BTW I didn't even know until now that MLFlow had LLM tools in it!!!

floating point 的头像

floating point1 年前

@databricks I don't understand the workflow. So, say I see the agent made a wrong step, I see the diff and where. What is my next step? With agents I can't just add positive and negative demonstration to the training set, I have to debug the prompts, your tool does not solve that, right?

czverse.𝕏 的头像

czverse.𝕏1 年前

@databricks Game changer! Synth evals + diffing UI = faster, smarter iterations. Can't wait to try this out!

相关视频

Grok3 Integration Now Available in AI Agent SDK! We’re thrilled to announce the integration of Grok3 into the AI Agent SDK, brining developers more AI capabilities! 🎉 🚀 Grok3 Integration: Easy integration with Grok3 for your projects. 🚀 AI Provider Enum: Quickly switch between multiple AI providers, including Grok3. 🔖 Check out the doc for the guide and more details: 🎬 Watch demos on setting up Grok3 and transacting via a Telegram powered by Grok3 with the SDK 👇 Dive in and start building with Grok3 today!

Grok3 Integration Now Available in AI Agent SDK! We’re thrilled to announce the integration of Grok3 into the AI Agent SDK, brining developers more AI capabilities! 🎉 🚀 Grok3 Integration: Easy integration with Grok3 for your projects. 🚀 AI Provider Enum: Quickly switch between multiple AI providers, including Grok3. 🔖 Check out the doc for the guide and more details: 🎬 Watch demos on setting up Grok3 and transacting via a Telegram powered by Grok3 with the SDK 👇 Dive in and start building with Grok3 today!

Crypto.com AI Agent

23,363 次观看 • 1 年前

Our AI Agent Launchpad is LIVE. Check out is a quick tutorial video. We are leveraging SuperHype 's Hyperliquid native token launchpad SDK and all AI Agent token will get bonded to SuperHype 's HyperEVM AMM DEX. A new era has begun for The Farm😈

Our AI Agent Launchpad is LIVE. Check out is a quick tutorial video. We are leveraging SuperHype 's Hyperliquid native token launchpad SDK and all AI Agent token will get bonded to SuperHype 's HyperEVM AMM DEX. A new era has begun for The Farm😈

thefarm.fun

27,393 次观看 • 1 年前

We worked with OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor. With just a few lines of code, you can trace all your agent workflows and debug them much faster. Check out this quick demo to get started:

We worked with OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor. With just a few lines of code, you can trace all your agent workflows and debug them much faster. Check out this quick demo to get started:

Respan

58,055 次观看 • 1 年前

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.

Runway

35,040 次观看 • 7 个月前

🔥 We fine-tuned Gemma 3 using Unsloth AI on a custom instruction dataset scraped and built with . The model is now able to answer questions about OpenAI Agents SDK documentation. Check out the demo below👇

🔥 We fine-tuned Gemma 3 using Unsloth AI on a custom instruction dataset scraped and built with . The model is now able to answer questions about OpenAI Agents SDK documentation. Check out the demo below👇

Eric Ciarla (hiring)

28,794 次观看 • 1 年前

Now it’s time to onboard more builders. We’ve put together a short developer onboarding video. The video covers: - 1⃣: How to build your first agent with the Unibase Agent SDK and deploy it to Unibase AIP — from anywhere - 2⃣: How to call deployed agents and execute tasks via the Unibase AIP SDK — from anywhere Next up, we’ll show how to use Unibase Membase to persist the memory of the OpenClaw bot 🧠 🔗Useful Link ▫️Unibase Docs: ▫️Unibase Agent SDK: Your Fast way to spin up your first agent with AIP. ▫️Unibase AIP SDK & DOCS ▫️AIP 2.0 DOCS: *As the SDK is under active development, certain interfaces may be subject to change. Please refer to the latest GitHub source for the authoritative implementation.

Now it’s time to onboard more builders. We’ve put together a short developer onboarding video. The video covers: - 1⃣: How to build your first agent with the Unibase Agent SDK and deploy it to Unibase AIP — from anywhere - 2⃣: How to call deployed agents and execute tasks via the Unibase AIP SDK — from anywhere Next up, we’ll show how to use Unibase Membase to persist the memory of the OpenClaw bot 🧠 🔗Useful Link ▫️Unibase Docs: ▫️Unibase Agent SDK: Your Fast way to spin up your first agent with AIP. ▫️Unibase AIP SDK & DOCS ▫️AIP 2.0 DOCS: *As the SDK is under active development, certain interfaces may be subject to change. Please refer to the latest GitHub source for the authoritative implementation.

Unibase

19,712 次观看 • 6 个月前

We've been obsessing a lot about creating the best possible AI support experience Dub.co. And I think we've found the magic formula. The stack: ✦ Mintlify → docs + help articles ✦ Upstash vector → vector search ✦ AI SDK + Claude sonnet → LLM/retrieval ✦ Plain → support infra Here's how it works: We consolidated our docs + help articles on Mintlify (best-in-class docs platform) so all our support content is now in one place. On every push to main, we sync changes to our Upstash Vector DB. We also built a custom chat interface that retrieves content from Upstash Vector and generates responses via AI SDK + Claude sonnet. The best part: The AI agent uses 𝚝𝚘𝚘𝚕() from AI SDK to retrieve the user's workspace / partner data to provide accurate responses. Finally, if the user needs to speak to a human, we generate a 𝚃𝚒𝚌𝚔𝚎𝚝𝚄𝚙𝚕𝚘𝚊𝚍 UI that lets users upload screenshots + add more context, before creating a ticket on Plain. Cherry on top: The chat interface is embeddable, so we're able to embed it both in our Mintlify docs + support portal 👇 Since we launched this, it's been a gamechanger for scaling support – reducing support workload and creating a much better experience for our users. Highly recommend giving this a try!

We've been obsessing a lot about creating the best possible AI support experience Dub.co. And I think we've found the magic formula. The stack: ✦ Mintlify → docs + help articles ✦ Upstash vector → vector search ✦ AI SDK + Claude sonnet → LLM/retrieval ✦ Plain → support infra Here's how it works: We consolidated our docs + help articles on Mintlify (best-in-class docs platform) so all our support content is now in one place. On every push to main, we sync changes to our Upstash Vector DB. We also built a custom chat interface that retrieves content from Upstash Vector and generates responses via AI SDK + Claude sonnet. The best part: The AI agent uses 𝚝𝚘𝚘𝚕() from AI SDK to retrieve the user's workspace / partner data to provide accurate responses. Finally, if the user needs to speak to a human, we generate a 𝚃𝚒𝚌𝚔𝚎𝚝𝚄𝚙𝚕𝚘𝚊𝚍 UI that lets users upload screenshots + add more context, before creating a ticket on Plain. Cherry on top: The chat interface is embeddable, so we're able to embed it both in our Mintlify docs + support portal 👇 Since we launched this, it's been a gamechanger for scaling support – reducing support workload and creating a much better experience for our users. Highly recommend giving this a try!

Steven Tey

22,790 次观看 • 4 个月前

Also coming up next: s402 SDK To bring on new developers and servers to our platform we will be making a boilerplate/SDK for the ease of integrating s402 payments to their applications. Plus, should be updated and all new bugs are fixed. Feel free to try out an Agent! In our example below we query our BNB Chain memecoin insights tool with our agent. Agents have multi-tool capabilities by the way! Proof it works 👇

Also coming up next: s402 SDK To bring on new developers and servers to our platform we will be making a boilerplate/SDK for the ease of integrating s402 payments to their applications. Plus, should be updated and all new bugs are fixed. Feel free to try out an Agent! In our example below we query our BNB Chain memecoin insights tool with our agent. Agents have multi-tool capabilities by the way! Proof it works 👇

Sora 🔶

14,931 次观看 • 9 个月前

Anthropic just released 5 Claude workshops on the new skill every AI builder needs: Loop Engineering: 00:00 - Stop babysitting your agents 37:07 - Build a proactive Claude Code loop 1:11:23 - Build a production-ready Managed Agent 1:38:46 - Teach agents to learn from your team 2:07:15 - Hill-climb your agent with eval loops It is a 2 hour and 25 minute blueprint for Loop Engineering: proactive workflows, managed agents, team feedback, evals, and self-improvement. Watch it today, and read the article below to improve your skills with Fable 5.

Anthropic just released 5 Claude workshops on the new skill every AI builder needs: Loop Engineering: 00:00 - Stop babysitting your agents 37:07 - Build a proactive Claude Code loop 1:11:23 - Build a production-ready Managed Agent 1:38:46 - Teach agents to learn from your team 2:07:15 - Hill-climb your agent with eval loops It is a 2 hour and 25 minute blueprint for Loop Engineering: proactive workflows, managed agents, team feedback, evals, and self-improvement. Watch it today, and read the article below to improve your skills with Fable 5.

Alex

46,668 次观看 • 25 天前

🚨 “Tomorrow we’re going to have an announcement - one of the biggest Medical breakthroughs in the history of our Country” “We think we’ve found the answer to Autism & tomorrow Bobby & Health Experts are going to be talking about that” “Parents will be able to help their Child with Autism - their beautiful Child”

🚨 “Tomorrow we’re going to have an announcement - one of the biggest Medical breakthroughs in the history of our Country” “We think we’ve found the answer to Autism & tomorrow Bobby & Health Experts are going to be talking about that” “Parents will be able to help their Child with Autism - their beautiful Child”

Concerned Citizen

5,576,390 次观看 • 10 个月前

Anthropic engineer: "Over 90% of Anthropic engineers use a self-improving agentic loop for coding. The key is to close the loop: give the agent a way to verify and improve its own work" In this 12-minute talk, an Anthropic engineer explains how to build an agent that continuously improves itself. Claude + routines + loops + CLAUDE.md - that's the foundation. Skip Netflix tonight. Save this post!

Anthropic engineer: "Over 90% of Anthropic engineers use a self-improving agentic loop for coding. The key is to close the loop: give the agent a way to verify and improve its own work" In this 12-minute talk, an Anthropic engineer explains how to build an agent that continuously improves itself. Claude + routines + loops + CLAUDE.md - that's the foundation. Skip Netflix tonight. Save this post!

unicode

61,880 次观看 • 1 个月前

Introducing Durable Sessions — the key pattern for collaborative AI. As the world moves to getting things done through agents, the winners are going to be the products that combine AI with collaboration. Building AI apps on a Durable Session pattern is the best way to do that. The post linked in the 🧵below shows how you can build Durable Sessions into your existing stack, schema and AI SDK using Durable Streams from Electric and Tack And All That DB.

Introducing Durable Sessions — the key pattern for collaborative AI. As the world moves to getting things done through agents, the winners are going to be the products that combine AI with collaboration. Building AI apps on a Durable Session pattern is the best way to do that. The post linked in the 🧵below shows how you can build Durable Sessions into your existing stack, schema and AI SDK using Durable Streams from Electric and Tack And All That DB.

James Arthur

75,967 次观看 • 6 个月前

Is your Agent ready for real-world users? Learn how to run an evaluation using Agentverse’s Response QA Agent - a built-in tool that stress-tests your Agent with realistic queries and scores its performance. This is the first step to improving discoverability, trust, and performance. Run your first QA check and see how your Agent responds under pressure. Watch 👇

Is your Agent ready for real-world users? Learn how to run an evaluation using Agentverse’s Response QA Agent - a built-in tool that stress-tests your Agent with realistic queries and scores its performance. This is the first step to improving discoverability, trust, and performance. Run your first QA check and see how your Agent responds under pressure. Watch 👇

Agentverse by Fetch.ai

20,202 次观看 • 8 个月前

I rebuilt Posia using the Pi Agent SDK. The core agent is ~15 lines of code, and the harness is so flexible and hackable. Recorded a 17-min walkthrough on: - Pi’s extension system + how to build your own - extensions that cut tool tokens by 80–96% - the core Pi SDK, step by step - building hosted agents with it Adding my Pi extensions setup in AI Builder Club repo here:

I rebuilt Posia using the Pi Agent SDK. The core agent is ~15 lines of code, and the harness is so flexible and hackable. Recorded a 17-min walkthrough on: - Pi’s extension system + how to build your own - extensions that cut tool tokens by 80–96% - the core Pi SDK, step by step - building hosted agents with it Adding my Pi extensions setup in AI Builder Club repo here:

Jason Zhou

65,108 次观看 • 25 天前

🤖 HYTOPIA AI Agents Society (Start Building!) 🎮 We've just added the "ai-agents" example to our SDK, which is a starting point of a game with an AI Agent Society! You can use this example as a template to build your own AI agents for a HYTOPIA game:

🤖 HYTOPIA AI Agents Society (Start Building!) 🎮 We've just added the "ai-agents" example to our SDK, which is a starting point of a game with an AI Agent Society! You can use this example as a template to build your own AI agents for a HYTOPIA game:

HYTOPIA

15,732 次观看 • 1 年前

Today, Shopify is launching our AI store setup. Now, with just a simple description of your business, you can get a custom-designed theme that makes your website launch-ready. This is just one of the ways we are leveraging AI to help our merchants start and scale faster. We’ve once again reduced the time from having your first idea to getting your first sale.

Today, Shopify is launching our AI store setup. Now, with just a simple description of your business, you can get a custom-designed theme that makes your website launch-ready. This is just one of the ways we are leveraging AI to help our merchants start and scale faster. We’ve once again reduced the time from having your first idea to getting your first sale.

Harley Finkelstein

73,826 次观看 • 1 年前

It turns out that even AWS doesn't properly document all of their errors. So I built an Agent with Effect/AI to discover missing error tags by invoking the API. Our AWS SDK is Effect-native, so it was trivial to build. This is not sped up:

It turns out that even AWS doesn't properly document all of their errors. So I built an Agent with Effect/AI to discover missing error tags by invoking the API. Our AWS SDK is Effect-native, so it was trivial to build. This is not sped up:

sam

32,394 次观看 • 7 个月前

Microsoft Teams just got Generative UI But this is just the tip of the iceberg. AG-UI is the protocol connecting any agent to any user on any app, topped with Generative UI. That means your agent can render: - forms - buttons - charts - approval flows Right inside the chat, on the fly. But it's not limited to in-app copilots. What's special is that you can work with any agent using the same system and bring them into any surface, including Slack, WhatsApp, Discord and more. This is exactly why Hermes and OpenClaw took off. As users of these agents, we don't like walls of text, and it's a pain to get when you can have way better UX. CopilotKit🪁 + AG-UI handle that interaction layer. If you want to try out Agentic UI in Microsoft Teams, you can get early access here:

Microsoft Teams just got Generative UI But this is just the tip of the iceberg. AG-UI is the protocol connecting any agent to any user on any app, topped with Generative UI. That means your agent can render: - forms - buttons - charts - approval flows Right inside the chat, on the fly. But it's not limited to in-app copilots. What's special is that you can work with any agent using the same system and bring them into any surface, including Slack, WhatsApp, Discord and more. This is exactly why Hermes and OpenClaw took off. As users of these agents, we don't like walls of text, and it's a pain to get when you can have way better UX. CopilotKit🪁 + AG-UI handle that interaction layer. If you want to try out Agentic UI in Microsoft Teams, you can get early access here:

Atai Barkai

24,595 次观看 • 1 个月前

We Neo Sigma Ritvik Kapila are building the future of self-improving AI systems! By closing the feedback loop between production data and system improvements, we help teams capture failures, convert them into structured evaluation signals, and use them to drive continuous improvements in agent behavior. We show how our system works on Tau3 bench across retail, telecom, and airline domains. Agent performance on the validation set (with a fixed underlying model, GPT5.4) improves from 0.56 → 0.78 (~40% jump in accuracy).

We Neo Sigma Ritvik Kapila are building the future of self-improving AI systems! By closing the feedback loop between production data and system improvements, we help teams capture failures, convert them into structured evaluation signals, and use them to drive continuous improvements in agent behavior. We show how our system works on Tau3 bench across retail, telecom, and airline domains. Agent performance on the validation set (with a fixed underlying model, GPT5.4) improves from 0.56 → 0.78 (~40% jump in accuracy).

Gauri Gupta

95,994 次观看 • 4 个月前

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai

Martin Nebelong

594,503 次观看 • 2 年前