Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Introducing Respan Evals 2.0: Automation. Most AI production issues don’t show up as errors. They show up as answers that look valid but are wrong, incomplete, unsafe, or off-brand. Respan now runs automated online evals on live production logs to catch silent quality failures in real time.

Respan

1,869 subscribers

110,783 görüntüleme • 1 ay önce •via X (Twitter)

Bilim & Teknoloji Haberler & Politika

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Respan Launch Week, Day 2: Evals Evals are one of the hardest parts of building AI applications. It is not because teams cannot run them. It is because they are hard to structure, hard to compare, and even harder to improve over time. So teams end up guessing. Did this prompt actually get better? Is this model really an improvement? We built Evals in Respan to make this systematic. You define: - what you want to test, such as prompts, models, or configs - the dataset, from production logs or test cases - the evaluators, whether LLM, code, human, or a mix Then you run experiments and compare results side by side. Same inputs. Same evaluation. Clear answers. No more guessing. Start running your first eval.

Respan Launch Week, Day 2: Evals Evals are one of the hardest parts of building AI applications. It is not because teams cannot run them. It is because they are hard to structure, hard to compare, and even harder to improve over time. So teams end up guessing. Did this prompt actually get better? Is this model really an improvement? We built Evals in Respan to make this systematic. You define: - what you want to test, such as prompts, models, or configs - the dataset, from production logs or test cases - the evaluators, whether LLM, code, human, or a mix Then you run experiments and compare results side by side. Same inputs. Same evaluation. Clear answers. No more guessing. Start running your first eval.

Respan

127,656 görüntüleme • 3 ay önce

The hard part about LLM failures is that their outputs rarely look like failures. The demo “works.” The output sounds coherent. The user actively uses the product. And your dashboard looks normal. Meanwhile, the system can be wrong, unsafe, or quietly driving up token spend. And you won’t notice until the damage adds up. Prompts often serve as business logic (policies, safety, and product context). But many teams ship them without the basics, such as versioning, reviewable changes, end-to-end traces, and eval gates. In production, it doesn’t crash. It degrades via wrong answers, policy misses, and surprise spending. No crash. No error. No alert. I cover this exact issue in my Stanford University CS 224G guest lecture on AI Observability and Evaluations. Here are the core ideas: • If you only log the final output, you’re guessing. Full traces show where it broke. • Evals are feedback loops. Use clear pass/fail criteria tied to outcomes. • Run evals continuously on production traces and don’t wait for support tickets. The moat isn’t prompt cleverness. It’s a measured improvement. Full lecture + blog below 👇

The hard part about LLM failures is that their outputs rarely look like failures. The demo “works.” The output sounds coherent. The user actively uses the product. And your dashboard looks normal. Meanwhile, the system can be wrong, unsafe, or quietly driving up token spend. And you won’t notice until the damage adds up. Prompts often serve as business logic (policies, safety, and product context). But many teams ship them without the basics, such as versioning, reviewable changes, end-to-end traces, and eval gates. In production, it doesn’t crash. It degrades via wrong answers, policy misses, and surprise spending. No crash. No error. No alert. I cover this exact issue in my Stanford University CS 224G guest lecture on AI Observability and Evaluations. Here are the core ideas: • If you only log the final output, you’re guessing. Full traces show where it broke. • Evals are feedback loops. Use clear pass/fail criteria tied to outcomes. • Run evals continuously on production traces and don’t wait for support tickets. The moat isn’t prompt cleverness. It’s a measured improvement. Full lecture + blog below 👇

Arsh Shah Dilbagi

20,833 görüntüleme • 5 ay önce

So many people talking about AI evals. Hamel Husain and Shreya Shankar show me how to actually build one. Live. Hamel and Shreya teach the world’s most popular course on evals, and have long been at the forefront of this emerging and important new skill for AI product builders. I'm so excited to finally have them on the podcast. Learn: 🔸 WTF are evals 🔸 A step-by-step live walkthrough of how to create an eval 🔸 Why evals have become so important 🔸 The debate between “vibes” and systematic evals 🔸 Code-based evals vs. LLM-as-judge 🔸 Why you only need 4-7 evals (not hundreds) 🔸 Building an LLM judge in Google Sheets 🔸 Why you always want to start with error analysis 🔸 Much more Listen now 👇 • YouTube: • Spotify: • Apple: Thank you to our wonderful sponsors for supporting the podcast: 🏆 Fin — The #1 AI agent for customer service: 🏆 dscout — The UX platform to capture insights at every stage: from ideation to production: 🏆 Mercury — The art of simplified finances:

So many people talking about AI evals. Hamel Husain and Shreya Shankar show me how to actually build one. Live. Hamel and Shreya teach the world’s most popular course on evals, and have long been at the forefront of this emerging and important new skill for AI product builders. I'm so excited to finally have them on the podcast. Learn: 🔸 WTF are evals 🔸 A step-by-step live walkthrough of how to create an eval 🔸 Why evals have become so important 🔸 The debate between “vibes” and systematic evals 🔸 Code-based evals vs. LLM-as-judge 🔸 Why you only need 4-7 evals (not hundreds) 🔸 Building an LLM judge in Google Sheets 🔸 Why you always want to start with error analysis 🔸 Much more Listen now 👇 • YouTube: • Spotify: • Apple: Thank you to our wonderful sponsors for supporting the podcast: 🏆 Fin — The #1 AI agent for customer service: 🏆 dscout — The UX platform to capture insights at every stage: from ideation to production: 🏆 Mercury — The art of simplified finances:

Lenny Rachitsky

125,296 görüntüleme • 10 ay önce

Eval scores in 2026 are broken. MMLU at 91%+, GSM8K at 94%+, yet models still can't handle basic multi-step tasks. And reported scores don't even agree across model cards, papers, and platforms. We just shipped Community Evals on Hugging Face: - Benchmark datasets now host live leaderboards (MMLU-Pro, GPQA, HLE) - Scores live in model repos as versioned YAML - Anyone can submit evals to any model via PR without merging. - Verified badges for reproducible runs via Inspect AI This won't fix saturation or stop test set contamination. But it makes the game visible. What was evaluated, how, when, and by whom. Done trusting black-box leaderboards. Time to decentralize evals.

Eval scores in 2026 are broken. MMLU at 91%+, GSM8K at 94%+, yet models still can't handle basic multi-step tasks. And reported scores don't even agree across model cards, papers, and platforms. We just shipped Community Evals on Hugging Face: - Benchmark datasets now host live leaderboards (MMLU-Pro, GPQA, HLE) - Scores live in model repos as versioned YAML - Anyone can submit evals to any model via PR without merging. - Verified badges for reproducible runs via Inspect AI This won't fix saturation or stop test set contamination. But it makes the game visible. What was evaluated, how, when, and by whom. Done trusting black-box leaderboards. Time to decentralize evals.

Ben Burtenshaw

19,259 görüntüleme • 6 ay önce

We gave AI agents access to real production systems and watched them do things they were never authorized to do. SOC 2 and ISO can't catch it, but TrustAI (YC S26) can. We built new evals so companies can have a continuous, transparent record on key vulnerabilities, get suggested mitigative solutions, and run time/cost efficiency analysis. Built by two Massachusetts Institute of Technology (MIT) engineers, backed by Y Combinator. Considering or running an agent on sensitive systems? Building one that connects to other enterprises' private data? Reach out today at

We gave AI agents access to real production systems and watched them do things they were never authorized to do. SOC 2 and ISO can't catch it, but TrustAI (YC S26) can. We built new evals so companies can have a continuous, transparent record on key vulnerabilities, get suggested mitigative solutions, and run time/cost efficiency analysis. Built by two Massachusetts Institute of Technology (MIT) engineers, backed by Y Combinator. Considering or running an agent on sensitive systems? Building one that connects to other enterprises' private data? Reach out today at

Hannah Chung

34,674 görüntüleme • 18 gün önce

Over 1 billion PDFs are created every day, but your agents still can’t read them reliably. Today we’re releasing Parse 2.0, the most accurate document parsing API in the world. Extend already processes millions of pages daily for leading AI teams like Brex, Mercury, Opendoor, Flatiron Health, and hundreds of others. Now, its even better. Parse 2.0 is SOTA quality on RealDoc-Bench, our open source benchmark that measures agent success rate on real world docs that agents actually encounter in production. We trained Parse 2.0 on 1M+ pages of the hardest documents seen in production. Here’s how it stacks up: - #1 in healthcare, real estate, logistics, and financial services - 95.7% agent Q&A accuracy on 581 docs (next best: 92%) - 0.847 F1 on layout (next best: 0.759) Give it a try today and build production-ready document agents with Extend.

Over 1 billion PDFs are created every day, but your agents still can’t read them reliably. Today we’re releasing Parse 2.0, the most accurate document parsing API in the world. Extend already processes millions of pages daily for leading AI teams like Brex, Mercury, Opendoor, Flatiron Health, and hundreds of others. Now, its even better. Parse 2.0 is SOTA quality on RealDoc-Bench, our open source benchmark that measures agent success rate on real world docs that agents actually encounter in production. We trained Parse 2.0 on 1M+ pages of the hardest documents seen in production. Here’s how it stacks up: - #1 in healthcare, real estate, logistics, and financial services - 95.7% agent Q&A accuracy on 581 docs (next best: 92%) - 0.847 F1 on layout (next best: 0.759) Give it a try today and build production-ready document agents with Extend.

Kushal Byatnal

586,668 görüntüleme • 2 ay önce

Cybercab production line work continues and more vehicles seem to be leaving the factory and headed to the outbound lot. Today I saw another ~60 but this time they were spread all over the outbound lot, some driving to the Superchargers, others doing driving testing while even more look to be preparing for transportation off site. Now that we are in mid-April, I’m hoping we will begin to see even more of these showing up at the outbound lot and if that starts to happen, it will give a better idea how the production line is doing as far as ramping up.

Cybercab production line work continues and more vehicles seem to be leaving the factory and headed to the outbound lot. Today I saw another ~60 but this time they were spread all over the outbound lot, some driving to the Superchargers, others doing driving testing while even more look to be preparing for transportation off site. Now that we are in mid-April, I’m hoping we will begin to see even more of these showing up at the outbound lot and if that starts to happen, it will give a better idea how the production line is doing as far as ramping up.

Joe Tegtmeyer 🚀 🤠🛸😎

79,426 görüntüleme • 3 ay önce

I have been hyping Brian Scanlan and Intercom to every eng leader I've spoken to since getting a behind the scenes look at how they 2x'd their productivity in their R&D org. In this week's episode of How I AI, Brian pulls up dashboards and tools and shows us how it's done: - explicit targets + reporting on leading indicators of AI adoption - how they measured how quality increased with more AI usage - centralized skills repo for the entire company - telemetry on claude code + evals on internal success - building for an agent-first onboarding flow Sorry to be that bro, but truly: STOP what you are doing and watch this right now if you are looking to inflect your eng + product team with AI. As always, a huge thanks to our sponsors: 🧠Celigo - Intelligent automation built for AI: Cursor - The best (and my fave!) way to code with AI: On YT now 👉

I have been hyping Brian Scanlan and Intercom to every eng leader I've spoken to since getting a behind the scenes look at how they 2x'd their productivity in their R&D org. In this week's episode of How I AI, Brian pulls up dashboards and tools and shows us how it's done: - explicit targets + reporting on leading indicators of AI adoption - how they measured how quality increased with more AI usage - centralized skills repo for the entire company - telemetry on claude code + evals on internal success - building for an agent-first onboarding flow Sorry to be that bro, but truly: STOP what you are doing and watch this right now if you are looking to inflect your eng + product team with AI. As always, a huge thanks to our sponsors: 🧠Celigo - Intelligent automation built for AI: Cursor - The best (and my fave!) way to code with AI: On YT now 👉

claire vo 🖤

12,834 görüntüleme • 3 ay önce

Your inbox is full of AI-generated sales emails that you don’t care about. Especially in enterprise, you can’t afford to miss. Therefore, we’re taking a different approach to breaking into new accounts. - Identify and act on strategic insights in real-time - Write outbound that targets real problems - Show up to every meeting ready to speak credibly Here’s what your day as an enterprise seller will look like.

Your inbox is full of AI-generated sales emails that you don’t care about. Especially in enterprise, you can’t afford to miss. Therefore, we’re taking a different approach to breaking into new accounts. - Identify and act on strategic insights in real-time - Write outbound that targets real problems - Show up to every meeting ready to speak credibly Here’s what your day as an enterprise seller will look like.

Keith Peiris

290,809 görüntüleme • 1 yıl önce

Alright, this one’s worth your attention if you’re building or deploying agents. Future AGI just open-sourced their entire platform and i don’t mean a trimmed-down version. this is the full stack: UI, backend, simulation engine, evals, optimization loop, observability, guardrails, gateway, docs. all in one repo. Apache 2.0. I’ve been putting it through its paces on production agents, and what stands out isn’t just the breadth it’s the architecture. Most of the current “agent reliability” stack is fragmented. tracing lives in one tool, evals in another, guardrails somewhere else. you end up manually connecting dots, and the agent itself doesn’t really improve you just keep patching prompts and hoping for the best. This flips that model. It’s built as a closed feedback loop: simulate failures → evaluate in real time → detect production issues → learn from them → generate fixes → validate against real traffic → check regressions → redeploy → monitor again And when something new breaks, the loop just runs again. no manual glue. The simulation piece is especially strong. instead of static test cases, it generates adversarial, multi-turn conversations based on how your agent actually behaves basically hunting for the exact scenarios where your system fails confidently. ran a few thousand simulations on our side… caught things we definitely would’ve missed. Evals run fast (sub-50ms) across modalities. not LLM-as-judge trained classifiers. guardrails are built-in, not layered on top. observability gives you step-level visibility into reasoning, cost, latency, quality. But the real shift is the optimization loop. Most tools tell you *what* broke. this system actually fixes it, validates the fix, and ensures nothing else regresses. That’s the missing layer. It’s clearly built with production in mind not a research demo. and the fact that it’s self-hostable makes it even more relevant if you’re running serious workloads. If you’ve been duct-taping together infra around your agents, this is probably the closest thing to a unified system i’ve seen so far. Worth checking out. If you're serious about deploying reliable AI agents, this is worth a look: 👉 You can also try it instantly (no setup) via their cloud version:

Alright, this one’s worth your attention if you’re building or deploying agents. Future AGI just open-sourced their entire platform and i don’t mean a trimmed-down version. this is the full stack: UI, backend, simulation engine, evals, optimization loop, observability, guardrails, gateway, docs. all in one repo. Apache 2.0. I’ve been putting it through its paces on production agents, and what stands out isn’t just the breadth it’s the architecture. Most of the current “agent reliability” stack is fragmented. tracing lives in one tool, evals in another, guardrails somewhere else. you end up manually connecting dots, and the agent itself doesn’t really improve you just keep patching prompts and hoping for the best. This flips that model. It’s built as a closed feedback loop: simulate failures → evaluate in real time → detect production issues → learn from them → generate fixes → validate against real traffic → check regressions → redeploy → monitor again And when something new breaks, the loop just runs again. no manual glue. The simulation piece is especially strong. instead of static test cases, it generates adversarial, multi-turn conversations based on how your agent actually behaves basically hunting for the exact scenarios where your system fails confidently. ran a few thousand simulations on our side… caught things we definitely would’ve missed. Evals run fast (sub-50ms) across modalities. not LLM-as-judge trained classifiers. guardrails are built-in, not layered on top. observability gives you step-level visibility into reasoning, cost, latency, quality. But the real shift is the optimization loop. Most tools tell you what broke. this system actually fixes it, validates the fix, and ensures nothing else regresses. That’s the missing layer. It’s clearly built with production in mind not a research demo. and the fact that it’s self-hostable makes it even more relevant if you’re running serious workloads. If you’ve been duct-taping together infra around your agents, this is probably the closest thing to a unified system i’ve seen so far. Worth checking out. If you're serious about deploying reliable AI agents, this is worth a look: 👉 You can also try it instantly (no setup) via their cloud version:

Aakash Verma

22,910 görüntüleme • 3 ay önce

$AMZN CEO Andy Jassy describes AI adoption as very barbelled, with most existing enterprise workloads still not using inference though he expects that to change significantly over time. "I think it's actually kind of useful to look at at least our view of what we see in the demand and adoption curve right now, which is we see this adoption curve in AI right now is very barbelled." "There is on one end of the barbell, the AI labs are consuming gobs and gobs of compute. And there are a few runaway successful generative AI applications like Claude Code and ChatGPT." "And on the other end of the barbell are enterprises who are getting real value from AI in cost avoidance and productivity. And these are things like automating customer service or business process automation or fraud." "In the middle of the barbell is all of the current enterprise production workloads, some of which are using inference in a pervasive way, but most of which aren't. And that is going to change very significantly over time." "And that will be, in my opinion, that will be the largest absolute segment, the existing production workloads in the enterprise and new businesses and workloads that startups build too."

$AMZN CEO Andy Jassy describes AI adoption as very barbelled, with most existing enterprise workloads still not using inference though he expects that to change significantly over time. "I think it's actually kind of useful to look at at least our view of what we see in the demand and adoption curve right now, which is we see this adoption curve in AI right now is very barbelled." "There is on one end of the barbell, the AI labs are consuming gobs and gobs of compute. And there are a few runaway successful generative AI applications like Claude Code and ChatGPT." "And on the other end of the barbell are enterprises who are getting real value from AI in cost avoidance and productivity. And these are things like automating customer service or business process automation or fraud." "In the middle of the barbell is all of the current enterprise production workloads, some of which are using inference in a pervasive way, but most of which aren't. And that is going to change very significantly over time." "And that will be, in my opinion, that will be the largest absolute segment, the existing production workloads in the enterprise and new businesses and workloads that startups build too."

Fireside Alpha

29,242 görüntüleme • 6 gün önce

New short course: Evaluating AI Agents! Evals are important for driving AI system improvements, and in this course you'll learn to systematically assess and improve an AI agent’s performance. This is built in partnership with Arize AI and taught by John Gilhuly, Head of Developer Relations, and , Director of Product. I've often found evals to be a critical tool in the agent development process - they can be the difference between picking the right thing to work on vs. wasting weeks of effort. Whether you’re building a shopping assistant, coding agent, or research assistant, having a structured evaluation process helps you refine its performance systematically, rather than relying on random trial and error. This course shows you how to structure your evals to assess the performance of each component of an agent and its end-to-end performance. For each component, you select the appropriate evaluators, test examples, and performance metrics. This helps you identify areas for improvement both during development and in production. (If you're familiar with error analysis in supervised learning, think of this as adapting those ideas to agentic workflows.) In this course, you'll build an AI agent, and add observability to visualize and debug its steps. You’ll learn about code-based evals, in which you write code explicitly to test a certain step, as well as LLM-as-a-Judge evals, in which you prompt an LLM to efficiently come up with ways to evaluate more open-ended outputs. In detail, you’ll: - Understand key differences between evaluating LLM-based systems and traditional software testing. - Add observability to an agent by collecting traces of the steps taken by the agent and visualizing them - Choose the appropriate evaluator - code-based, LLM-as-a-Judge, human-annotation based - for each component. - Compute a convergence score to evaluate if your agent can respond to a query in an efficient number of steps. - Run structured experiments to improve the agent’s performance by exploring changes to the prompt, LLM model, or the agent’s logic. - Understand how to deploy these evaluation techniques to monitor the agent’s performance in production. By the end of this course, you’ll know how to trace AI agents, systematically evaluate them, and improve their performance. Please sign up here:

New short course: Evaluating AI Agents! Evals are important for driving AI system improvements, and in this course you'll learn to systematically assess and improve an AI agent’s performance. This is built in partnership with Arize AI and taught by John Gilhuly, Head of Developer Relations, and , Director of Product. I've often found evals to be a critical tool in the agent development process - they can be the difference between picking the right thing to work on vs. wasting weeks of effort. Whether you’re building a shopping assistant, coding agent, or research assistant, having a structured evaluation process helps you refine its performance systematically, rather than relying on random trial and error. This course shows you how to structure your evals to assess the performance of each component of an agent and its end-to-end performance. For each component, you select the appropriate evaluators, test examples, and performance metrics. This helps you identify areas for improvement both during development and in production. (If you're familiar with error analysis in supervised learning, think of this as adapting those ideas to agentic workflows.) In this course, you'll build an AI agent, and add observability to visualize and debug its steps. You’ll learn about code-based evals, in which you write code explicitly to test a certain step, as well as LLM-as-a-Judge evals, in which you prompt an LLM to efficiently come up with ways to evaluate more open-ended outputs. In detail, you’ll: - Understand key differences between evaluating LLM-based systems and traditional software testing. - Add observability to an agent by collecting traces of the steps taken by the agent and visualizing them - Choose the appropriate evaluator - code-based, LLM-as-a-Judge, human-annotation based - for each component. - Compute a convergence score to evaluate if your agent can respond to a query in an efficient number of steps. - Run structured experiments to improve the agent’s performance by exploring changes to the prompt, LLM model, or the agent’s logic. - Understand how to deploy these evaluation techniques to monitor the agent’s performance in production. By the end of this course, you’ll know how to trace AI agents, systematically evaluate them, and improve their performance. Please sign up here:

Andrew Ng

126,478 görüntüleme • 1 yıl önce

This is so real what Tyler said because people forget the person they look up to or idolize are human just like them and forget to consume the art and start to dig to deep in their business it becomes a violation. Even if from time to time they will show some of their personality or lifestyle, we should understand to only take what they give and not take it as a ticket to reach in too deep.

This is so real what Tyler said because people forget the person they look up to or idolize are human just like them and forget to consume the art and start to dig to deep in their business it becomes a violation. Even if from time to time they will show some of their personality or lifestyle, we should understand to only take what they give and not take it as a ticket to reach in too deep.

garçon

237,615 görüntüleme • 2 yıl önce

This is huge for production applications! Literally every developer has to deal with server errors they can't reproduce, and it doesn't matter how many logs they have. You don't have to be "every developer" anymore. Look into Hud's Runtime Code Sensor. Hud is one of my collaborators, and their code sensor is a game-changer for production applications. Watch the video I'm attaching here. For those of you who'd prefer a TL;DR; Hud captures how code behaves in production and puts those insights in context right in your IDE. This is really cool! You can see on top of every function data about how it works in production: • How long does it take to run? • How many times has the function failed? • Are the number of failures going up or down? All of this information is accessible to Cursor, Copilot, or WindSurf via an MCP server. That means that you can use the IDE to act on these issues! The way Hud works: • They provide a lightweight SDK • It takes one line to configure • All insights are right in your code • No dashboard necessary The best part: Their free plan includes ALL features they offer, so you can actually try it out properly and see how it fits your workflow. No paywall, no trial. Here is the link: Logging everything is great, but getting production insights right next to the code is definitely next-level!

This is huge for production applications! Literally every developer has to deal with server errors they can't reproduce, and it doesn't matter how many logs they have. You don't have to be "every developer" anymore. Look into Hud's Runtime Code Sensor. Hud is one of my collaborators, and their code sensor is a game-changer for production applications. Watch the video I'm attaching here. For those of you who'd prefer a TL;DR; Hud captures how code behaves in production and puts those insights in context right in your IDE. This is really cool! You can see on top of every function data about how it works in production: • How long does it take to run? • How many times has the function failed? • Are the number of failures going up or down? All of this information is accessible to Cursor, Copilot, or WindSurf via an MCP server. That means that you can use the IDE to act on these issues! The way Hud works: • They provide a lightweight SDK • It takes one line to configure • All insights are right in your code • No dashboard necessary The best part: Their free plan includes ALL features they offer, so you can actually try it out properly and see how it fits your workflow. No paywall, no trial. Here is the link: Logging everything is great, but getting production insights right next to the code is definitely next-level!

Santiago

28,515 görüntüleme • 1 yıl önce

NEW: Zohran Mamdani teared up as he recalled his “aunt” allegedly being too scared to wear a hijab on the subway in New York City after 9/11 — but there’s a catch. Records show she was working in Tanzania from 2000–2003, and publicly available photos don’t show her wearing a hijab. Now, Sean Hannity 🇺🇸 has one question for Mamdani: “Where are the tears for the NYPD? We lost 2,977 Americans on that day.”

NEW: Zohran Mamdani teared up as he recalled his “aunt” allegedly being too scared to wear a hijab on the subway in New York City after 9/11 — but there’s a catch. Records show she was working in Tanzania from 2000–2003, and publicly available photos don’t show her wearing a hijab. Now, Sean Hannity 🇺🇸 has one question for Mamdani: “Where are the tears for the NYPD? We lost 2,977 Americans on that day.”

Fox News

136,363 görüntüleme • 9 ay önce

The next version of SEO isn’t about ranking number one. It’s about what AI says about you when someone asks. Because Google, ChatGPT, Gemini, Claude, they’re not just showing links anymore. They’re forming opinions. They’re comparing brands. They’re deciding who sounds credible, who sounds risky, who gets recommended, and who gets ignored. That means your biggest search problem in 2027 might not be that you don’t show up. It might be that you show up wrong. Maybe AI describes your competitor as the trusted expert… and describes you as generic. Maybe it mentions your brand… but the sentiment is neutral while your competitor is glowing. Maybe it pulls from old reviews, outdated articles, weak bios, messy positioning, or random third-party pages you didn’t even know were shaping your reputation. That’s the part most businesses are missing. AI search visibility is not just a traffic game. It’s a reputation game. And that’s why we added the Sentiment tab inside Ubersuggest’s AI Search Visibility tool. It doesn’t just track whether your brand appears across ChatGPT, Gemini, Claude, and Google AI. It shows how those engines describe you. Positive, neutral, or negative. And it charts that sentiment over time against your competitors. So now you can answer the question that actually matters: Not just, “Am I showing up in AI answers?” But, “What is AI actually saying about me?”

Neil Patel

16,819 görüntüleme • 14 gün önce

I'm often asked for the best public example of AI evals done right for a real, production product. I finally have an answer. Teresa Torres shares how she shipped an AI interview coach, and used evals to rapidly squash bugs and improve the product. Teresa shows how she: 1. did error analysis FIRST to find real issues (instead of using generic metrics) 😍 2. used Jupyter notebooks to analyze errors 3. built custom annotation tools + custom widgets in notebooks 4. built a LLM-judge and assertions to test for specific errors 5. iterated through this feedback loop until it worked. 6. kept things simple the whole time It's also probably the best commercial for Jupyter notebooks you can imagine. 🥰 Chapter summary below. Link to YT in next thread 00:00:00 - Intro 00:01:45 - The Product: Building an AI Interview Coach 00:06:34 - The Problem: How Do I Know if My AI Coach is Any Good? 00:10:15 - Using Airtable for Traces and Annotation 00:12:15 - Discovering Jupyter Notebooks and Designing the First Evals 00:15:15 - Example Evals: LLM-as-Judge vs. Code-Based Assertions 00:21:00 - Learning Python with ChatGPT to Analyze Eval Results 00:31:00 - VS Code, Custom Tools, and an Eval Investigation Notebook 00:39:45 - Building a Custom Annotation Tool with Claude 00:41:00 - From Personal Project to Production App 00:46:02 - How Should PMs and Engineers Collaborate on AI Products? 00:55:45 - Q&A: Capturing Feedback and Annotations from End Users 00:58:11 - Q&A: Is a Technical Background Necessary to Build AI? 01:02:28 - Q&A: What's Next for Teresa? 01:03:13 - Q&A: Unpacking the Micro-Decisions of Building an AI App

I'm often asked for the best public example of AI evals done right for a real, production product. I finally have an answer. Teresa Torres shares how she shipped an AI interview coach, and used evals to rapidly squash bugs and improve the product. Teresa shows how she: 1. did error analysis FIRST to find real issues (instead of using generic metrics) 😍 2. used Jupyter notebooks to analyze errors 3. built custom annotation tools + custom widgets in notebooks 4. built a LLM-judge and assertions to test for specific errors 5. iterated through this feedback loop until it worked. 6. kept things simple the whole time It's also probably the best commercial for Jupyter notebooks you can imagine. 🥰 Chapter summary below. Link to YT in next thread 00:00:00 - Intro 00:01:45 - The Product: Building an AI Interview Coach 00:06:34 - The Problem: How Do I Know if My AI Coach is Any Good? 00:10:15 - Using Airtable for Traces and Annotation 00:12:15 - Discovering Jupyter Notebooks and Designing the First Evals 00:15:15 - Example Evals: LLM-as-Judge vs. Code-Based Assertions 00:21:00 - Learning Python with ChatGPT to Analyze Eval Results 00:31:00 - VS Code, Custom Tools, and an Eval Investigation Notebook 00:39:45 - Building a Custom Annotation Tool with Claude 00:41:00 - From Personal Project to Production App 00:46:02 - How Should PMs and Engineers Collaborate on AI Products? 00:55:45 - Q&A: Capturing Feedback and Annotations from End Users 00:58:11 - Q&A: Is a Technical Background Necessary to Build AI? 01:02:28 - Q&A: What's Next for Teresa? 01:03:13 - Q&A: Unpacking the Micro-Decisions of Building an AI App

Hamel Husain

51,376 görüntüleme • 11 ay önce

College athletes don’t get to “skip class” just because they wear a jersey. The NCAA literally requires a 2.0 GPA and at least 6 classes per semester. But with online and hybrid options now, most players are grinding school and football at the same time. Travel and practice make in-person tough, but the work still gets done… and the numbers prove it: graduation rate is up 20% over the last two decades. So when you show up on campus next year, don’t be shocked if the QB is in your lecture… and don’t be shocked if they’re working harder than everyone else. #CollegeFootball #StudentAthlete #NCAA #Football

College athletes don’t get to “skip class” just because they wear a jersey. The NCAA literally requires a 2.0 GPA and at least 6 classes per semester. But with online and hybrid options now, most players are grinding school and football at the same time. Travel and practice make in-person tough, but the work still gets done… and the numbers prove it: graduation rate is up 20% over the last two decades. So when you show up on campus next year, don’t be shocked if the QB is in your lecture… and don’t be shocked if they’re working harder than everyone else. #CollegeFootball #StudentAthlete #NCAA #Football

Jason Cabinda

83,502 görüntüleme • 3 ay önce

seventeen members promised that if they don’t have any work/schedules, they’ll go send off the member who’s going to enlist in the military! 🥹 🍒we’re going to keep doing the send-off thing, right? 🐰why why? 🍒i feel like it’s a bit difficult now.. 🦦then will i have to go alone? (😭😭) 🍊hey S you have to be there 🐸of course, we’ll go if we can. but if we’re overseas. then, it can’t be helped 🦦no, make time and show up! 🍊show up! 🦦try to make time to be there, the8 🍒i’ll go if i don’t have work! 🐸right! if i don’t have something to do 🦌that can’t be helped, of course 🐰of course you can’t show up if you have work. but if you don’t have work you have to show up 🍊the dates are set before work. the’re usually set before 🐻‍❄️understood

seventeen members promised that if they don’t have any work/schedules, they’ll go send off the member who’s going to enlist in the military! 🥹 🍒we’re going to keep doing the send-off thing, right? 🐰why why? 🍒i feel like it’s a bit difficult now.. 🦦then will i have to go alone? (😭😭) 🍊hey S you have to be there 🐸of course, we’ll go if we can. but if we’re overseas. then, it can’t be helped 🦦no, make time and show up! 🍊show up! 🦦try to make time to be there, the8 🍒i’ll go if i don’t have work! 🐸right! if i don’t have something to do 🦌that can’t be helped, of course 🐰of course you can’t show up if you have work. but if you don’t have work you have to show up 🍊the dates are set before work. the’re usually set before 🐻‍❄️understood

하니행뵹

68,363 görüntüleme • 5 ay önce

When Paterson residents lost water, we showed up with it. That’s what real leadership looks like—being hands-on, present, and fighting for New Jerseyans when they need it most. Nobody in New Jersey should accept this as normal. Democrats have been in charge for over 20 years and can’t even guarantee running water! We can do better. As Governor, I’ll always show up and make our state a better place to live and do business. It’s time.

When Paterson residents lost water, we showed up with it. That’s what real leadership looks like—being hands-on, present, and fighting for New Jerseyans when they need it most. Nobody in New Jersey should accept this as normal. Democrats have been in charge for over 20 years and can’t even guarantee running water! We can do better. As Governor, I’ll always show up and make our state a better place to live and do business. It’s time.

Jack Ciattarelli

62,142 görüntüleme • 11 ay önce