Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Introducing ACT-2 Preview The first robotics model to unify broad generalization with high reliability. A single fine-tuning example can teach Memo a new behavior that generalizes. Zero shot, real unseen homes, 99% success rate.

1,042,998 görüntüleme • 1 ay önce •via X (Twitter)

61 Yorum

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

We completed the most rigorous generalization test to date. Across 785 trials in 31 unseen environments, ACT-2 achieved 99.1% success in laundry folding. ACT-2 also achieved human-level fold quality: receiving an average rating of 4.72/5, with 98.3% earning four or five stars.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Tons of emergent behavior along the way: - Picking up clothes off the ground - Handling baby wear to 8XL shirts - Robustness against adversarial disturbances and lighting

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

We call this a Solve. Progress in robotics is difficult to measure because demos vary by setup. Demo ≠ Solved. A Solve declares two boundaries: scope and adaptation cost. Without both, 99% has no context.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data. For the first time, one fine-tuning example can teach a new behavior that generalizes. Below: 4 folding strategies, learned from one example each and tested on held-out setups.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Quantitatively, we measure the generalization gap as the difference between in-domain and out-of-domain performance. As we scale up pretraining, the gap falls sharply. This makes in-house performance a reliable predictor of performance in the wild.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

This property allows us to hill-climb performance in our office, and trust those gains to hold in unseen homes Our fleet of Memos runs in parallel to rapidly advance reliability, quality, and speed. Left: fleet-scale improvement in-house Right: Memo working across unseen homes

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Laundry is our first Solve of many. Our recipe is so general that scaling data and compute gives us predictable improvements. Unlocking one Solve accelerates the next Solve. The same ACT-2 model is learning to vacuum, organize toys, zip clothing, and turn pants inside out.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

This fall, ACT-2’s first Solve enters homes through our Beta Program, the final step towards fully autonomous home robot deployment. Full technical report:

Sholto Douglas profil fotoğrafı
Sholto Douglas1 ay önce

So cool, can't wait to have one at my place

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Very soon 😉

Roei Herzig profil fotoğrafı
Roei Herzig1 ay önce

Is it possible to get access to the pretrained model? We recently developed a mechanistic interpretability concept for fast-adaptation by tuning only the relevant parts of the network: If the pretrained model is strong, we can do magic. Give us access, and we can unlock continual learning together ;)

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

This is very cool, checking it out now!

Camilla Guo profil fotoğrafı
Camilla Guo1 ay önce

sending this to my mom

Dhaval Shroff profil fotoğrafı
Dhaval Shroff1 ay önce

Very cool to see this kind of generalization @tonyzzhao

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Just ask for generalization! @ericjang11

Tim Zaman profil fotoğrafı
Tim Zaman1 ay önce

But are you still shipping this year

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Have you applied!

Yuanhao Qu profil fotoğrafı
Yuanhao Qu1 ay önce

@timzaman Congrats @tonyzzhao on the launch!!! Can’t wait to have one at home.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

@timzaman Thank you Yuanhao! Apply here!

Kate Park profil fotoğrafı
Kate Park1 ay önce

Congrats! data engine end to end ftw 💪

Remi Cadene profil fotoğrafı
Remi Cadene1 ay önce

Extremely cool ;)

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Remi!! Congrats on the humanoid release. So cool.

Jason Ma profil fotoğrafı
Jason Ma1 ay önce

congrats tony! awesome to see generalization + reliability

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Jason! 🙏

Caitlin Kalinowski profil fotoğrafı
Caitlin Kalinowski1 ay önce

The wanting intensifies! Does this mean I will be able to teach it to fold things my way? 😍

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Exactly! I'm honestly quite surprised that it just works with SFT..!

Perry Jia profil fotoğrafı
Perry Jia1 ay önce

Proud of the team for this achievement

Yu Xiang profil fotoğrafı
Yu Xiang1 ay önce

@RemiCadene Very Impressive. Congratulations! There is a snapshot where the head camera is blocked, and the robot can still keep going. How to explain this behavior?

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

@RemiCadene ACT-2 takes all 5 cameras (one head, two on each hand) as input. And sometimes dropout is all you need!

Ryan Julian profil fotoğrafı
Ryan Julian1 ay önce

Congratulations @sundayrobotics team! Incredible work

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

@sundayrobotics Thank you Ryan!

Machine Space profil fotoğrafı
Machine Space1 ay önce

@Scobleizer I can’t believe nobody has asked for the holy grail of folding: The King Size Fitted Sheet.

Aaref Hilaly profil fotoğrafı
Aaref Hilaly1 ay önce

big breakthrough - congrats!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Aaref for being part of this journey ❤️

Rahul profil fotoğrafı
Rahul1 ay önce

This all seemed so far away till last year

Jiafei Duan profil fotoğrafı
Jiafei Duan1 ay önce

Really cool work! It will be cooler if the weights could be open source, seems to be a common question here.

Lindon Gao profil fotoğrafı
Lindon Gao1 ay önce

Congrats @tonyzzhao & team!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Lindon!!

Jake Roggenbuck profil fotoğrafı
Jake Roggenbuck1 ay önce

Generalizing reliability!!

Owen Brake profil fotoğrafı
Owen Brake1 ay önce

amazing, can you share the absolute number of episodes in the pre-training dataset?

clankr profil fotoğrafı
clankr1 ay önce

These results are pretty impressive. So this is what the Sunday team meant when they said most robotics companies were collecting data the wrong way.

Ted Xiao profil fotoğrafı
Ted Xiao1 ay önce

@RemiCadene Congrats, very exciting!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

@RemiCadene Thank you Ted! 🙏

Max Mclaughlin profil fotoğrafı
Max Mclaughlin1 ay önce

Impressive, but let's see it do all laundry, not just folding. Folding doesn't really change depending on where you are, full laundry definitely does

Sampriti Bhattacharyya profil fotoğrafı
Sampriti Bhattacharyya1 ay önce

This is cool, definitely one of the hard problems

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

It was very difficult. We weren’t sure if it’s possible beginning of the year.

Eren Chen profil fotoğrafı
Eren Chen1 ay önce

We all know what Green cap in Chinese culture means…

Ofir Ozeri profil fotoğrafı
Ofir Ozeri1 ay önce

@philfung What about the emergent behavior you said you saw? That’s amazing but you left us curious!

Haoru Xue profil fotoğrafı
Haoru Xue1 ay önce

Congrats! Looking solid!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Haoru! Precise posting after vague posting 😉

Ivan Kirigin profil fotoğrafı
Ivan Kirigin1 ay önce

How do you think about a "solve" when the cost of errors scales? Like 99% in self driving is unlaunchable.

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Precisely. The performance threshold could differ across applications. What a Solve highlights is that we should not omit Scope and Adaptation Budget when reporting the success rate. The Solve framework does not carry any opinion about the performance threshold itself.

Shuang Li profil fotoğrafı
Shuang Li1 ay önce

Very cool congrats!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Shuang!

Dhruv Batra profil fotoğrafı
Dhruv Batra1 ay önce

Very cool videos! I appreciate the detailed descriptions of the evals in the blog. Is there a model card somewhere?

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

Thank you Dhruv. We don't have it right now but might release it in the future with the full release 🫡

Dhruv Batra profil fotoğrafı
Dhruv Batra1 ay önce

Look forward to reading it if/when you do. Also signed up for the waitlist — kudos on the launch!

Sourish Jasti profil fotoğrafı
Sourish Jasti1 ay önce

so cool, congrats

Vikash Kumar profil fotoğrafı
Vikash Kumar1 ay önce

@wenlong_huang @tonyzzhao - it’s incredible how far ACT is going! Let there be no boundaries. Kudos to entire @sundayrobotics team

Zeeshan Patel profil fotoğrafı
Zeeshan Patel1 ay önce

@arthurallshire Congrats Tony, really cool results!

Tony Zhao profil fotoğrafı
Tony Zhao1 ay önce

@arthurallshire Thank you Zeeshan! Hope all is well and congrats on the new journey!

Benzer Videolar