Introducing AttuneBench! We built this benchmark on a simple premise: for self-improving AI to reach its full usefulness to humanity, it needs high EQ. We decomposed EQ into distinct skills and evaluated 11 frontier models across 50 real-life topics, from relationships and marriage to school and job stress, using... 50,000+ first-person annotations.show more

2/ Leaderboard results This is the first benchmark grounded in real multi-turn human-model conversations. We found that different frontier models excel at different parts of EQ Anthropic’s Opus 4.7 is best at picking which response a person preferred, while Google’s Gemini 3.1 Pro is best at tracking how someone feels. GPT-5.5 is the only model strong at both turn-by-turn prediction and whole-conversation comprehension. The larger lesson is that emotional intelligence is jagged: a model can be strong at one kind of attunement and weak at another.

3/ Emotional intelligence is many independent skills It’s a bet that even the most human-seeming capabilities can be made measurable if we define them through the behaviors people can actually observe: whether a model notices distress, tracks changing preferences, adapts to context, and responds in a way the person experiences as helpful. Across 11 models, emotion recognition, behavioral judgment, preference prediction, and response quality were largely independent. You can explore conversations in our explorer:

4/ Other key insights - The perspective gap is persistent (All 11 models were better at predicting what the model did than what the participant wanted, with gaps of 3.0 to 7.6 percentage points.) - Multi-turn conversations expose drift (9 of 11 models became less accurate at reading behavior in the last third of a conversation than in the first) - Preference is the deeper signal (Emotion labeling is useful, but the harder problem is predicting what kind of response a specific person needs in context) - Models struggle most where affective accuracy may matter most

5/ AttuneBench v1.0 is open and free to run. We'll keep releasing new versions as models and methods evolve. Paper: Blog: Code: Leaderboard: This work was a collaboration between Thoughtful and @pareto_ai ’s Research team, led by @MarkWhiting and @phoebeyao. Special thanks to the participants who made this dataset possible.

Interesting! Congrats on launch.

this is very cool

Beautiful and important project. And to save future fans a few clicks, here's the link:

Great to see such a crucial benchmark! Congrats 👏