Today, we announce Ego-Exo4D, the largest and most diverse multi-view dataset, showing human experts around the world performing a core set of skilled activities, w/ unprecedented multi-modality, novel new video-language resources, and rich annotations

Ego-Exo4D consists of more than 5,000 skills demonstrations, captured from first and third-person perspectives, featuring @meta_aria devices as the egocentric sensor and 4+ exocentric GoPros. This equals 1,400 hours of video. Open access starts on 12/11

Collaboration with @meta_aria allowed us to generate unique multimodal resources, including multichannel audio, calibrated gaze vectors, 3D point clouds, camera poses, IMU, among others.

Ego-Exo4D introduce three new video-language resources – first-person narrations from the camera-wearers; third-person descriptions of all atomic actions; and third-person spoken commentary by coaches and teachers, offering tips and insights time-stamped to each skill enactment

Ego4D prioritizes real-world authenticity, diversity and partnership. Data was collected with more than 800 people, ranging from skilled dancers to master chefs. Collection occurred in 13 cities around the world w/ the Ego4D consortium, a partnership between 15 research orgs

Ego4D and Ego-Exo4D is a partnership between @IULuddy, @cmu_africa , @UMNComputerSci, @CMU_Robotics, @ICatGT, @iiit_hyderabad, @KaustVision, @NUSingapore , @GRASPlab, @unict_it , @BristolUni, @UTokyo_News_en, @uandes, @unccs, @SFU, @meta_aria and @AIatMeta

@cmu_africa @UMNComputerSci @CMU_Robotics @ICatGT @iiit_hyderabad @KaustVision @NUSingapore @GRASPlab @unict_it @BristolUni @UTokyo_News_en @uandes @unccs @SFU @meta_aria @AIatMeta Learn more! Project website: Paper:

Nice work! Another interesting new dataset ### EgoBridge ### that also contains ego and exo videos,

Great work! The dataset should be definitely released as soon as possible🤯

Incoming