Loading video...
Video Failed to Load
Introducing EgoHome: a new open benchmark for dense, sub-second egocentric understanding. Human-annotated household-chore video with 4 annotation types on every clip: hand boxes, subtasks, hand actions, hand captions. It features over 750 segments under 500ms. Data, metrics, and judge prompts are all open. 🧵
45,561 views • 6 days ago •via X (Twitter)
8 Comments

EgoHome is the first egocentric benchmark to feature 4 types of annotations per video: • Hand boxes – 417,912 at 30 fps • Subtask timelines – 1,924 segments • Hand actions – 5,032 segments using Ego4D closed vocab • Hand captions – 5,277 free-form segments

EgoHome includes 3,371 segments under 1 s and 773 under 500 ms, with the shortest at 100 ms. WGO-Bench has 15 segments under 0.5 s. Hands move faster than current benchmarks can measure, and this density lets us break every result down by segment length, especially on fast actions.

It introduces a new metric: permissible subsegment Jaccard (PSJ). Rather than encouraging models to overfit to one particular choice of temporal granularity, PSJ allows the ground truth and predicted segments to match to each other in a many-to-many mapping. This permits predicted tasks to be subtasks of the ground truth (or vice versa)

We compare Perceptron Egocentric, at its default 4 fps and an 8 fps option, with GPT-6 Astra, Gemini 3.8 Flash and Elorian’s Glimpse, each at 4 fps and 1 fps, and with Pantheon’s Argus and NVIDIA’s FLORA. The Perceptron Egocentric harness matches or beats every other system on subtask labeling.

Everything is open-source: run your own model against EgoHome and tell us where it lands. Dataset: Code: Post:

That’s so cool!

Dense hand action labels under 500 ms are where most pipelines still fail consistency checks. We see temporal drift spike once clips drop below ~300 ms. How did you keep inter-annotator agreement stable at those lengths?

wow, didn't think GBT6 S-Tri has so much skill on this line of work. the world needs dense egocentric annotation
