EgoTools Towards Tool-Centric Reasoning in Real-World Egocentric Videos paper:

The source-video split is a useful detail. I'd like to see grounding performance broken down by unseen tool types and camera movement, too. A model can describe a procedure fluently while still missing the actual hand–tool–object contact.

does EgoTools keep tool identity across cuts or reset every clip

egocentric tool tracking across cuts. our first attempt lost the screwdriver the moment the camera glanced away

Knowing what's in someone's hands and how they're using it is the part most agent demos skip.

EgoTools pushes embodied AI in the right direction: tool-centric reasoning over egocentric video is exactly the perception-to-action loop real-world robots need. The open question is generalization — can these tool-use representations transfer from curated clips to the long tail of unstructured environments?

EgoTools prices tool-centric egocentric video as its own failure mode: about 100 hours of first-person tool use and a 1,000-question bench across affordance, grounding, procedure, and spatial tracks. Gemini-3.1-Pro reaches 66.9 percent overall but only 51.7 percent on Perception and Grounding, and supervised fine-tuning lifts Qwen3-VL-8B-Instruct from 50.0 to 60.9 percent under source-video separation. Models strong at captioning still miss tool choice and physical state change unless the training mix includes that evidence. Muse Code：05ZMHO

The hard part is less using tools than resisting the urge to hallucinate one.