Loading video...

Video Failed to Load

Go Home

Argus: an open-source robotics data annotation and quality pipeline. With new frontier VLMs like GPT-6 Astra, we can generate rich, high-quality annotations for robotics data for both training and dataset analysis. Argus delivers detailed, timestamp-level annotations while catching issues like mislabeled instructions, sped-up recordings, swapped camera streams, and unflagged...

89,243 views • 1 month ago •via X (Twitter)

33 Comments

calix's profile picture
calix1 month ago

Argus takes in any episode in its raw format and outputs the following: - A densely annotated timeline with all robot or gripper actions, state changes, task progression percentages, and more - A record for every operator mistake, along with notes on the timing of the failure, the nature of the failure, the operator’s recovery response, and the ideal fix - Any metadata mistakes - A severity rating for each issue based on the predicted impact on training - Any subgoals and the completion outcome - The goal frame if it is achieved - Overall grade on completion along with a short rationale review

calix's profile picture
calix1 month ago

Argus can also be used on inference and tele-op trajectories captured in production. Here’s an example of a fully annotated trajectory from one of our production facilities.

calix's profile picture
calix1 month ago

Argus is a model-agnostic pipeline and can run on any sufficiently capable VLM. In our test comparisons, Astra and GPT-6.1 Sol produced the densest annotations, which we’ve found is highly correlated with overall accuracy.

calix's profile picture
calix1 month ago

Of the 3,546 episodes we labelled across nine datasets, 27% have at least one issue of medium severity or above, meaning part or all of the episode teaches the model something wrong. We’re releasing the full audit and more than 150K annotations at

calix's profile picture
calix1 month ago

Read the full analysis at and run Argus on your data at For collaborations or questions, contact us at [email protected]. Credit to @ericli_ for undertaking this project.

Henry Lien's profile picture
Henry Lien1 month ago

@PantheonInc Eric 🚀

koko 𝑥𝑠's profile picture
koko 𝑥𝑠1 month ago

@PantheonInc 🚀

andrew kim's profile picture
andrew kim1 month ago

@PantheonInc Incredible

Varun Nair's profile picture
Varun Nair1 month ago

@PantheonInc Pushing the frontier! VLMs are improving so fast, habit. Great harnesses and clarity for training ready data is key to extract signals from raw data and train the right policies!

Mohamed Zidan Cassim's profile picture
Mohamed Zidan Cassim1 month ago

@PantheonInc chef Li @ericli_

carlo agostinelli's profile picture
carlo agostinelli1 month ago

@PantheonInc 🚢

Sambhav Gupta's profile picture
Sambhav Gupta1 month ago

@PantheonInc Eric li rolling the robotics boulder up the hill

Xiao-ke (XK)'s profile picture
Xiao-ke (XK)1 month ago

@PantheonInc @ericli_ 🚀🚀🚀

sean jeong's profile picture
sean jeong1 month ago

@PantheonInc 🔥

Rich's profile picture
Rich1 month ago

@PantheonInc Mr Li

Ishan's profile picture
Ishan1 month ago

@PantheonInc So cool

Yang Fan Yun's profile picture
Yang Fan Yun1 month ago

@PantheonInc very very cool stuff @calixo888 @nettedjay 😎

Tian Fang's profile picture
Tian Fang1 month ago

@PantheonInc Very cool.

Nikhil Suresh's profile picture
Nikhil Suresh1 month ago

@PantheonInc 🚢🚢🚢

Felix Huang's profile picture
Felix Huang1 month ago

@PantheonInc Fire contributions for the community

Hudzah's profile picture
Hudzah1 month ago

@PantheonInc beautiful

ChrisPy's profile picture
ChrisPy1 month ago

@PantheonInc 🧑‍🍳

Sufyan Osamah's profile picture
Sufyan Osamah1 month ago

@PantheonInc Swapped camera streams and sped up recordings silently ruin a training run. Good that this is open source.

majjigi's profile picture
majjigi1 month ago

@PantheonInc damn, this is amazing.

Lucas's profile picture
Lucas1 month ago

@PantheonInc Congrats Calix!

Arin's profile picture
Arin1 month ago

@PantheonInc 27% of episodes having training-affecting issues is a wild number for datasets people assumed were clean

joseph's profile picture
joseph1 month ago

@PantheonInc cooking

Federico Martelli's profile picture
Federico Martelli1 month ago

@PantheonInc catching swapped camera streams and mislabeled instructions before training saves more than another round of fine tuning, factory data has the same mess

Alexander's profile picture
Alexander1 month ago

@PantheonInc Super cool ! Have you guys looked at how much variation there is in SR of VLA style models when there is variance in the task description?

^_^'s profile picture
^_^1 month ago

@PantheonInc That's amazing!

Jack's profile picture
Jack1 month ago

Generated annotations need a sampling audit or you inherit the model's blind spots at full scale. The failure mode is not random noise, it is consistent error, which looks exactly like clean data until something downstream learns the same mistake. Worth holding back a human labelled slice purely to measure drift against.

Anna /✦'s profile picture
Anna /✦1 month ago

@PantheonInc sharing with my friend in robotics!

Kifayath Ulla's profile picture
Kifayath Ulla1 month ago

@PantheonInc WOAHH

Related Videos

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 views • 7 months ago