Made this video to explain evals

but in real world:

Haha I used this template like this

All the memes here

@ryancarson hahaha accurate the measure becomes the metric

feeling the models vibes yourself is often more important than evals they can tell you how good it is, but they often can't tell you what's the right way to use it properly

Most ppl don’t understand how powerful this is

🤣🤣🤣

very well done

Choice!

BRB posting this to customer slack

bro wtf AHHAHAHAHAHAHAHA

same ways you can just strip away those evals with a bigger jug (ahem model)

feels like the right mental model for evals is a swiss-cheese stack layered checks for incomplete prompts, bad tool calls, retrieval errors, etc.

i can even hear you saying this out loud too in that slow, pensive mode while looking at shreya during office hours 🤣

ah now i finally get it

This is amazing.

lmao

Metrics explain less than momentum

your evals video is super helpful for new builders.

This is exactly the kind of ops insight that gets overlooked. We found the same — the engineering challenge isnt the AI, its the coordination layer that keeps everything from stepping on each other.

that was a pretty clever move actually

evals are the new sql injection attacks for llms. patch one hole, attackers find three more.

@chainyoda 💀☠️💀☠️

evals are so underrated. thanks for making this!

😂 haha! Surprised there isn't a product /GitHub repo in your bio (free or paid). Looking forward to seeing the lessons imbibed into an eval product. Or you'd suggest there are sufficient in the market already?

when the eval clicks… suddenly the whole pipeline feels different

So keep playing whackamole till the heat death of the universe instead of trying to generalise outside the training distribution. Got it.

strong evals make stronger models, fren. weak ones breed cope

Evals have never let someone ship a graph DB!

lol

Evals → intuition quantified.

clean break down makes the whole thing click

@grok explain

This is a classic.

Nice, clear explainer content always hits.