A Rerun Viewer for the DROID Dataset! DROID: A... Large-Scale In-The-Wild Robot Manipulation Dataset is a robot manipulation dataset by Alexander Khazatsky et al. with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks.show more

Spatial AI ( is building large real-world datasets to... teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.show more

Eastworlds aims to build the largest dataset of high-quality... humanoid teleop data outside of China. While we believe human egocentric data will play a big role in pretraining embodied AI, robot teleop data remains the closest thing to expert demonstration.show more

How can we more effectively leverage robot data from... different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:show more

Unlike LLMs, we can’t scrape the internet for robot... data Announcing Project Go-Big: we’re building the world's largest humanoid pretraining dataset This is accelerated by our partnership with Brookfield, who owns over 100,000 residential unitsshow more

Turns out you can train humanoid hands without any... robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.show more

today we're launching a dataset of computer-use recordings to... build the next generation of computer-use agents. 300+ tasks across all kinds of software. open-source. link in the comments. Markovshow more

Heads up! Mosaic dropped a pretty wild dataset of... 1.26 million 360° images of Prague 🤯 If you're a researcher, creator or developer into 3D/AI/Geo, I think you're gonna wanna play with this Here's the scoop on this 15 TERAPIXEL dataset & the crazy things you can do with it 🧵show more

The newest version of our Almanac preprint is out,... and just in time for our demo at the Stanford AIMI Symposium 2023! Almanac is a retrieval-augmented LLM that provides up-to-date and verifiable answers to medical queries. Link: We benchmark our approach on a novel dataset of clinical scenarios (n = 130) evaluated by a panel of 5 board-certified & resident physicians, and demonstrate significant increases in factuality (mean of 18% at p-value < 0.05) across all specialties. More interestingly, because the retrieved data acts as a single source of truth, we find retrieval-based LLMs to be more robust to prompt injection and manipulation! Future work will involve expanding the scope of our dataset to more specialties and multimodal settings. #Medtwitter #MedEdshow more

Qubic #OpenScience #Neuraxon Life 2.5 (Research) version code released... & and a new Hugging Face massive dataset, almost 500Gb of raw TimeSeries Game data, 10 Sessions with 270+ rounds each, in total 100K+ Nxers in 2791 games with 3.76 Million Neurons & 17M Synapses to explore in a huge 1 Billion + events. By Jose Sánchez David Vivancos - e/acc for Qubic Science: Open Data Source Code: Dataset: Paper : Stay Tunned for more releases this week!show more

Peeling a potato is trivial -- until you try... to make a robot do it with a knife. This is actually one of the hardest problems in manipulation: contact-rich, force-sensitive, and success is subjective. We taught a robot arm to peel with &gt;90% success :Dshow more

China’s AI robot is going crazy EX-ROBOT is one... of the most popular robot companies in China you can see them in restaurants, hotels and homes and… talk with them to complete tasks for you 10 wild examples:show more

Today may be the ImageNet moment for robotics. RT-X:... the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:show more

Tactile interaction in the wild can unlock fine-grained manipulation!... 🌿🤖✋ We built a portable handheld tactile gripper that enables large-scale visuo-tactile data collection in real-world settings. By pretraining on this data, we bridge vision and touch—allowing robots to: ✅ Perform robust in-hand reorientation ✅ Control contact and force with precision 🔗 Project page: (1/6)show more

Ascento Guard is a two-wheeled, jumping robot designed for... security tasks on large properties. It is powered by AI, allowing it to patrol areas, detect threats, and respond autonomously. The robot was developed by the Ascento team at ETH Zurich.show more

Ukraine’s ground robot is now fighting like an infantryman.... Droid TW 12.7 held a frontline position for six weeks, writes Militarnyi. Operators sent it to a key crossroads each morning and pulled it back at night. Russians tried to break through — the robot stopped them. 1/show more

This is the power of YOLO, trained on a... laptop for ~1 hour, with a Kaggle dataset. Oh, and just ~100 lines of Python. I can make a startup on this and it took me literally a couple of hours.show more

Spent the past few weeks building a fully automated... pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.show more

Figure is aiming to develop the world’s largest and... most diverse real-world humanoid pretraining dataset. For this purpose, they’re partnering with Brookfield, a global asset manager overseeing $1 trillion in assets, including 100,000 residential units, 500M square feet of commercial office space, and 160M square feet of logistics space. The data collected from this collaboration will be used to train Figure’s Helix AI model, enabling humanoids to perform tasks autonomously in real-world environments designed for humans. In addition to data collection, the partnership will explore support for next-generation GPU data centers, real estate for robotic training environments, and commercial use cases across Brookfield’s global footprint.show more

NVIDIA announces the first open humanoid robot reference design... built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:show more