Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

DSPy is great at classification tasks, but it's hard to make a formal eval metric for 'fuzzy' creative tasks like writing a blog post or telling a joke. Solution: train a judge with DSPy to agree with you 90%+, then use that judge as the eval metric for the...

hammer_mt's profile picture

Mike Taylor

9,523 subscribers

18,654 Aufrufe • vor 9 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here:
3:27

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here:

Andrew Ng

181,457 Aufrufe • vor 1 Jahr

Glad that you guys both agree with me. When someone can't identify things that seem obvious, it's make you go hmmmm... seems like a good metric to use when "judging" objectively. 🤷‍♂️ Proper Doinks
0:58

Glad that you guys both agree with me. When someone can't identify things that seem obvious, it's make you go hmmmm... seems like a good metric to use when "judging" objectively. 🤷‍♂️ Proper Doinks

Archive Seed Bank

12,562 Aufrufe • vor 4 Monaten

🔥 Pokémon Red is becoming a go-to benchmark for testing advanced AIs such as Gemini. But is Pokémon Red really a good eval? We study this problem and identify three issues: 1️⃣ Navigation tasks are too hard. 2️⃣ Combat control is too simple. 3️⃣ Raising a strong Pokémon team is slow and expensive as an eval. We find most of the problems are not fundamental to games themselves, but how they have been used. We believe game-as-an-eval remains a compelling and underutilized evaluation strategy. We introduce Lmgame Bench to standardize game-as-an-eval. More details and findings in our blogpost:
0:41

🔥 Pokémon Red is becoming a go-to benchmark for testing advanced AIs such as Gemini. But is Pokémon Red really a good eval? We study this problem and identify three issues: 1️⃣ Navigation tasks are too hard. 2️⃣ Combat control is too simple. 3️⃣ Raising a strong Pokémon team is slow and expensive as an eval. We find most of the problems are not fundamental to games themselves, but how they have been used. We believe game-as-an-eval remains a compelling and underutilized evaluation strategy. We introduce Lmgame Bench to standardize game-as-an-eval. More details and findings in our blogpost:

Hao AI Lab

69,004 Aufrufe • vor 1 Jahr

You're so good at drunk dancing that you accidentally confess This officer needed proof for the judge that her suspect was intoxicated, so she got creative with the sobriety test. First, a catwalk. Then she asked him to dance. And this man DELIVERED like he was auditioning for a reality show.
0:52

You're so good at drunk dancing that you accidentally confess This officer needed proof for the judge that her suspect was intoxicated, so she got creative with the sobriety test. First, a catwalk. Then she asked him to dance. And this man DELIVERED like he was auditioning for a reality show.

Brown World

46,419 Aufrufe • vor 5 Monaten

LLMs are great for human in the loop applications, but fail at deterministic developer tasks. Interfaze (YC P26) is a new AI model that outperforms general LLMs on high accuracy tasks like: OCR, Object Detection, Web scraping, Speech-to-text, Classification and more. Congrats on the launch, Yoeven and Harsha!
1:21

LLMs are great for human in the loop applications, but fail at deterministic developer tasks. Interfaze (YC P26) is a new AI model that outperforms general LLMs on high accuracy tasks like: OCR, Object Detection, Web scraping, Speech-to-text, Classification and more. Congrats on the launch, Yoeven and Harsha!

Y Combinator

69,326 Aufrufe • vor 2 Monaten

I always preach for independent wrestlers to set up merch tables at shows… But what should they sell? EVERYTHING! Be creative. Make merch that you think your fans would want. Make different merch that you sell at various prices. The merch table is a great way to make more money. But more importantly, it’s a great way to make a connection with fans.
0:12

I always preach for independent wrestlers to set up merch tables at shows… But what should they sell? EVERYTHING! Be creative. Make merch that you think your fans would want. Make different merch that you sell at various prices. The merch table is a great way to make more money. But more importantly, it’s a great way to make a connection with fans.

Matt Cardona

15,667 Aufrufe • vor 1 Jahr

OH. MY. GOSH!!! The Charlotte NC judge who gave a $5k bond to a woman for assaulting her boyfriend who then ST*BBED HIM TO DE*TH 1 MONTH LATER... ...IS A DEMOCRAT JUDGE WITH PRIOR CHARGES HIMSELF FOR FALSIFYING LEGAL DOCUMENTS!!!! He served only 90 days of unsupervised probation and KEPT HIS JOB AS A JUDGE!!! This. Is. Insanity.
0:57

OH. MY. GOSH!!! The Charlotte NC judge who gave a $5k bond to a woman for assaulting her boyfriend who then ST*BBED HIM TO DE*TH 1 MONTH LATER... ...IS A DEMOCRAT JUDGE WITH PRIOR CHARGES HIMSELF FOR FALSIFYING LEGAL DOCUMENTS!!!! He served only 90 days of unsupervised probation and KEPT HIS JOB AS A JUDGE!!! This. Is. Insanity.

Matt Van Swol

123,600 Aufrufe • vor 3 Monaten

Ever Heard of a Metric in Differential Geometry? How About a Geodesic? These two ideas are central to how Mathematics and Physics describe Spacetime. A metric is how a surface measures distance and angle. A geodesic is the path that counts as "straight" after that measuring rule is chosen. Once the metric changes from point to point, straightness is no longer something you judge by eye. On a curved surface, the shortest paths are geodesics. They are chosen by the metric. Take a surface written as r(x,y) = (x, y, f(x,y)) Its metric is gᵢⱼ = ∂ᵢr · ∂ⱼr A geodesic q(t) = (x(t), y(t)) obeys q̈ᵏ + Γᵏᵢⱼ q̇ᶦq̇ʲ = 0 where Γᵏᵢⱼ are the Christoffel symbols of the surface metric. Launch many almost-parallel paths. At the start, they look like a clean beam until curvature takes over. Some paths focus, others separate and some slide around ridges. The surface is essentially the rulebook for motion. The shortest path is local. The geometry remembers everything. #DifferentialGeometry #Geodesics #RiemannianGeometry #MathematicalPhysics #Curvature #TensorCalculus #AppliedMathematics #Geometry #ScienceAnimation
0:33

Ever Heard of a Metric in Differential Geometry? How About a Geodesic? These two ideas are central to how Mathematics and Physics describe Spacetime. A metric is how a surface measures distance and angle. A geodesic is the path that counts as "straight" after that measuring rule is chosen. Once the metric changes from point to point, straightness is no longer something you judge by eye. On a curved surface, the shortest paths are geodesics. They are chosen by the metric. Take a surface written as r(x,y) = (x, y, f(x,y)) Its metric is gᵢⱼ = ∂ᵢr · ∂ⱼr A geodesic q(t) = (x(t), y(t)) obeys q̈ᵏ + Γᵏᵢⱼ q̇ᶦq̇ʲ = 0 where Γᵏᵢⱼ are the Christoffel symbols of the surface metric. Launch many almost-parallel paths. At the start, they look like a clean beam until curvature takes over. Some paths focus, others separate and some slide around ridges. The surface is essentially the rulebook for motion. The shortest path is local. The geometry remembers everything. #DifferentialGeometry #Geodesics #RiemannianGeometry #MathematicalPhysics #Curvature #TensorCalculus #AppliedMathematics #Geometry #ScienceAnimation

Mathelirium

22,170 Aufrufe • vor 24 Tagen

This makes the future way closer than you think. Director AI automates any tasks you can do in a browser. It interacts with web pages just like a person would. If you could use a browser to complete a task, you can have this AI do it for you. And it's free! Watch the video:
6:33

This makes the future way closer than you think. Director AI automates any tasks you can do in a browser. It interacts with web pages just like a person would. If you could use a browser to complete a task, you can have this AI do it for you. And it's free! Watch the video:

Santiago

36,273 Aufrufe • vor 1 Jahr

Teachers can use Canva with great ease, To make poetry lessons a delightful breeze. With templates for poems and creative art, Students' creativity can shine from the start. 🎤💥
0:21

Teachers can use Canva with great ease, To make poetry lessons a delightful breeze. With templates for poems and creative art, Students' creativity can shine from the start. 🎤💥

Canva Education

19,336 Aufrufe • vor 3 Jahren

.Dave Rubin: "How do you sort of quantify or analyze the behavior around [the pressures to make content only for clicks?]" Gad Saad: "I think the problem with Candace and a lot of those folks, is that the metric that they're trying to optimize is, as you've guys have said, clickbait."
2:17

.Dave Rubin: "How do you sort of quantify or analyze the behavior around [the pressures to make content only for clicks?]" Gad Saad: "I think the problem with Candace and a lot of those folks, is that the metric that they're trying to optimize is, as you've guys have said, clickbait."

The Rubin Report

23,589 Aufrufe • vor 7 Monaten

On a previous #TeawithGaryVee I was asked "What is more important to you as a leader? Challenging your employees with new tasks to grow or making sure they to give them tasks that they excel at?" if you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on
1:01

On a previous #TeawithGaryVee I was asked "What is more important to you as a leader? Challenging your employees with new tasks to grow or making sure they to give them tasks that they excel at?" if you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

Gary Vaynerchuk

20,386 Aufrufe • vor 9 Tagen

🚨 ELON MUSK: "It's not like, I agree with everything the administration does. I agree with much of what the administration does, but we have differences of opinion, you know, the things that I don't entirely agree with. But it's difficult for me to bring that up in an interview, because then it creates a bone of contention. So then I'm in a little stuck, in a bind where I'm like, Well, I don't want to, you know, speak up against the administration, but I don't want to, you also don't want to take responsibility for everything, the administration is doing."
0:38

🚨 ELON MUSK: "It's not like, I agree with everything the administration does. I agree with much of what the administration does, but we have differences of opinion, you know, the things that I don't entirely agree with. But it's difficult for me to bring that up in an interview, because then it creates a bone of contention. So then I'm in a little stuck, in a bind where I'm like, Well, I don't want to, you know, speak up against the administration, but I don't want to, you also don't want to take responsibility for everything, the administration is doing."

DogeDesigner

98,616 Aufrufe • vor 1 Jahr

Aaron Judge was hit coming off the field with a throw from Anthony Volpe. It is normal for a ball to be thrown to Judge as he comes off the field
0:27

Aaron Judge was hit coming off the field with a throw from Anthony Volpe. It is normal for a ball to be thrown to Judge as he comes off the field

Talkin' Yanks

693,166 Aufrufe • vor 1 Jahr

Ferrari wants Melanie to be a judge at the auditions for #YoungAndReckless season 3 👀! Do you agree?
0:20

Ferrari wants Melanie to be a judge at the auditions for #YoungAndReckless season 3 👀! Do you agree?

Baddies + More!

47,896 Aufrufe • vor 1 Jahr

Elon Musk: “You’d think that if you’re going to be appointed as a judge, you have to have proven that you have an excellent knowledge of the law and that you will make your decisions according to the law. That’s what we assume should be a requirement. You’ve got to know what the law is, and then you need to make decisions in accordance with the law. Not vibes. You can’t be just vibing as a judge.”
0:58

Elon Musk: “You’d think that if you’re going to be appointed as a judge, you have to have proven that you have an excellent knowledge of the law and that you will make your decisions according to the law. That’s what we assume should be a requirement. You’ve got to know what the law is, and then you need to make decisions in accordance with the law. Not vibes. You can’t be just vibing as a judge.”

Mars University

13,096,864 Aufrufe • vor 3 Monaten

🚀 Introducing Queue: A Smarter Way to Build with Replit Agent Today, we’re excited to launch Queue, a powerful new capability designed to enhance your experience with Replit Agent. With Queue, you can submit multiple tasks while your Agent is still working—no more waiting around or breaking your flow. As each task completes, the Agent automatically moves to the next. You can: ✅ Add multiple tasks for the Agent to process in order. ✅ Edit, reorder, or reprioritize messages on the fly. ✅ Attach files and adjust settings—including toggling extended thinking or selecting a different model—per task.
0:27

🚀 Introducing Queue: A Smarter Way to Build with Replit Agent Today, we’re excited to launch Queue, a powerful new capability designed to enhance your experience with Replit Agent. With Queue, you can submit multiple tasks while your Agent is still working—no more waiting around or breaking your flow. As each task completes, the Agent automatically moves to the next. You can: ✅ Add multiple tasks for the Agent to process in order. ✅ Edit, reorder, or reprioritize messages on the fly. ✅ Attach files and adjust settings—including toggling extended thinking or selecting a different model—per task.

Replit ⠕

75,576 Aufrufe • vor 1 Jahr

As a Canadian I really appreciate the fact that the Robotaxi automatically switches to Metric for speed and temperature for my rides. It’s a small touch, but I love it.
0:35

As a Canadian I really appreciate the fact that the Robotaxi automatically switches to Metric for speed and temperature for my rides. It’s a small touch, but I love it.

Devin Olsen

64,921 Aufrufe • vor 8 Monaten