Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Very cool to see how robots are now mastering these tasks, but as I read Benjie's blog ( I’m even more impressed by our human ingenuity in translating human dexterity into robot-specific behaviors (by the demonstrators): - wrapping sock on gripper to invert it - mashing plastic bag on... show more

Wenlong Huang

4,974 subscribers

20,552 просмотров • 8 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

The city of Venice is a marvel, but it’s under threat by the very sea that gave rise to it. I spent the day learning about how human ingenuity and technology are helping it defend itself from rising waters.

The city of Venice is a marvel, but it’s under threat by the very sea that gave rise to it. I spent the day learning about how human ingenuity and technology are helping it defend itself from rising waters.

Brad Smith

10,157 просмотров • 2 лет назад

The All-In Podcast discussed Figure's AI breakthrough this week I believe this AI breakthrough was essential to enabling a billion robots on the planet - allowing humans to communicate with them and have them carry out actions Helix, our internal AI model, powers the Figure 02 robot. The robot can now generalize to objects it has never seen before and can semantically plan responses to spoken commands, such as "put the groceries away where you think they should go" It’s important to note: the robot’s actuators are not the limiting factor. Right now, both dexterity and actuator speed are constrained by the software. In fact, the actuators are capable of operating at more than 5x their current speed, but our software is holding them back. Over time, as Helix improves, the robot will ultimately surpass human speeds I like these guys. Looking forward to having Chamath Palihapitiya at Figure HQ to see these robots in action!

The All-In Podcast discussed Figure's AI breakthrough this week I believe this AI breakthrough was essential to enabling a billion robots on the planet - allowing humans to communicate with them and have them carry out actions Helix, our internal AI model, powers the Figure 02 robot. The robot can now generalize to objects it has never seen before and can semantically plan responses to spoken commands, such as "put the groceries away where you think they should go" It’s important to note: the robot’s actuators are not the limiting factor. Right now, both dexterity and actuator speed are constrained by the software. In fact, the actuators are capable of operating at more than 5x their current speed, but our software is holding them back. Over time, as Helix improves, the robot will ultimately surpass human speeds I like these guys. Looking forward to having Chamath Palihapitiya at Figure HQ to see these robots in action!

Brett Adcock

135,552 просмотров • 1 год назад

"As long as the game is still played by human beings, I think you have to take the human element into it and leverage what's important at the time." @The_Higster offers insight on when to set up in a way that helps catchers steal strikes.

"As long as the game is still played by human beings, I think you have to take the human element into it and leverage what's important at the time." @The_Higster offers insight on when to set up in a way that helps catchers steal strikes.

Foul Territory

70,900 просмотров • 8 месяцев назад

Elon Musk: Tesla will probably start selling Optimus robots to the public by the end of next year. “Humanoid robotics will advance very quickly, I think. We do have some of the Tesla Optimus robots doing simple tasks in the factory. By the end of this year I think they'll be doing more complex tasks, but still deployed in an industrial environment. And probably by the end of next year we'd be selling humanoid robots to the public. That's when we are confident that it's very high reliability, very high safety, and the range of functionality is also very high. You can basically ask it to do anything you'd like.” Source: Interview at WEF, January 22, 2026

Elon Musk: Tesla will probably start selling Optimus robots to the public by the end of next year. “Humanoid robotics will advance very quickly, I think. We do have some of the Tesla Optimus robots doing simple tasks in the factory. By the end of this year I think they'll be doing more complex tasks, but still deployed in an industrial environment. And probably by the end of next year we'd be selling humanoid robots to the public. That's when we are confident that it's very high reliability, very high safety, and the range of functionality is also very high. You can basically ask it to do anything you'd like.” Source: Interview at WEF, January 22, 2026

ELON CLIPS

19,937 просмотров • 6 месяцев назад

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Brain-controlled exoskeletons to train humanoid robots! 🧠 Fourier just presented human tele-operators using brain control interfaces and exoskeletal arms to train humanoid robots on home tasks. The brain control interface is the interesting part. Instead of using a controller or joystick to teleoperate, the operator's movements and intentions are captured more naturally through the exoskeleton and BCI. This means the demonstrations are more fluid, more human-like, and better suited for training robots to perform delicate home tasks. Multiple tele-operators are simultaneously generating training data across multiple robots. This is how you build the dataset needed for eventual full autonomy, without waiting years for it to arrive. This might be the bridge between "robots that work in controlled environments" and "robots that work in homes." Not full autonomy right away, but trusted human intelligence operating through a robot body, getting better with every task completed. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

21,042 просмотров • 8 месяцев назад

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do? I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API! Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based). We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame! Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors. As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: Paper: Code: Deep dive with me: 🧵

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do? I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API! Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based). We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame! Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors. As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: Paper: Code: Deep dive with me: 🧵

Jim Fan

2,677,678 просмотров • 2 лет назад

"Humanoid robotics will advance very quickly We do have some of the Tesla Optimus robots doing simple tasks in the factory. Probably later this year... by the end of this year, I think they'll be doing more complex tasks But still deployed in an industrial environment. And probably sometime next year... I'd say that by the end of next year, I think we'd be selling humanoid robots to the public That's when we are confident that it's very high reliability, very high safety, and the range of functionality is also very high. You can basically ask it to do anything" - Elon Musk

"Humanoid robotics will advance very quickly We do have some of the Tesla Optimus robots doing simple tasks in the factory. Probably later this year... by the end of this year, I think they'll be doing more complex tasks But still deployed in an industrial environment. And probably sometime next year... I'd say that by the end of next year, I think we'd be selling humanoid robots to the public That's when we are confident that it's very high reliability, very high safety, and the range of functionality is also very high. You can basically ask it to do anything" - Elon Musk

X Freeze

675,647 просмотров • 8 месяцев назад

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

AK

125,353 просмотров • 3 лет назад

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.

AK

151,614 просмотров • 2 лет назад

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Quadrupeds are fast. Agile. Great at locomotion. But can they manipulate? A new approach from Carnegie Mellon University, Google DeepMind, and Bosch is teaching quadrupedal robots to do more than walk, they’re learning to interact. It’s called Human2LocoMan: a system that uses human data to pretrain robot policies before finetuning on real hardware. The result? A four-legged robot that can walk, carry, organize, scoop, and sort with both single and dual-arm control. By pretraining on human motion, they cut the amount of robot data in half—while improving success rates by over 80% in unfamiliar environments. Their Modularized Cross-Embodiment Transformer (MXT) learns from both human and robot demonstrations, then generalizes those skills to physical tasks—no hardcoded behaviors required. It’s locomotion and manipulation. A quadruped that can walk and clean up after itself?

Lukas Ziegler

61,610 просмотров • 1 год назад

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 просмотров • 8 месяцев назад

AI is not replacing jobs. There is a fundamental difference between a collection of tasks & a job. Job = U { Tasks, Judgment, Relationships, Context, Ownership} Few jobs are truly at risk of replacement by AI - these involve more deterministic IO (Input Output) execution e.g. data entry, QA, CS etc. Most jobs are resilient because: (1) Human principals (business owners) always seek to assign responsibility & blame to a human agent. (2) Deals get done not because of outbound spam but because of years of nurturing a relationship (showing up at conferences, events, meet-ups) (3) The physical world is messy AF. Humans read non-verbal & subtle cues which are only possible via in-person interactions. The midwits are right - AI is taking over tasks. In fact, if AI hasn’t taken over tasks for you then you’re doing something very wrong. If tasks are automated & workflows are changing, jobs won’t be lost. In fact, to manage the messy transition, humans will be more in demand than ever before. The data from Foundation Labs itself points to this (watch to the end of this rant)

AI is not replacing jobs. There is a fundamental difference between a collection of tasks & a job. Job = U { Tasks, Judgment, Relationships, Context, Ownership} Few jobs are truly at risk of replacement by AI - these involve more deterministic IO (Input Output) execution e.g. data entry, QA, CS etc. Most jobs are resilient because: (1) Human principals (business owners) always seek to assign responsibility & blame to a human agent. (2) Deals get done not because of outbound spam but because of years of nurturing a relationship (showing up at conferences, events, meet-ups) (3) The physical world is messy AF. Humans read non-verbal & subtle cues which are only possible via in-person interactions. The midwits are right - AI is taking over tasks. In fact, if AI hasn’t taken over tasks for you then you’re doing something very wrong. If tasks are automated & workflows are changing, jobs won’t be lost. In fact, to manage the messy transition, humans will be more in demand than ever before. The data from Foundation Labs itself points to this (watch to the end of this rant)

Rahul Mathur

10,664 просмотров • 3 месяцев назад

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that force-related data wasn’t captured: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a portable visuo-tactile gripper by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate cross-modal representation learning and downstream policy learning on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,510 просмотров • 1 год назад

President Paul Kagame : "There is No way I can Allow Anybody to Lower My value to That Level" “Some of the friends we deal with in the West. They come with this mindset that Africa is indifferent to human rights, to democracy, to freedom so they come to do it for us. Or to tell you that you should be doing it on their demands. And we are saying, 'No.' We as human beings. No human being anywhere would want to live by dictates by someone else. My understanding is that we are all human beings.” -The true son of Africa Your comment on this 👆

President Paul Kagame : "There is No way I can Allow Anybody to Lower My value to That Level" “Some of the friends we deal with in the West. They come with this mindset that Africa is indifferent to human rights, to democracy, to freedom so they come to do it for us. Or to tell you that you should be doing it on their demands. And we are saying, 'No.' We as human beings. No human being anywhere would want to live by dictates by someone else. My understanding is that we are all human beings.” -The true son of Africa Your comment on this 👆

JULIUS MUGABO

365,582 просмотров • 9 месяцев назад

How well do egocentric human videos scale? Apparently, really well. Dyna Robotics has introduced Dyna-2, a world-action model (WAM) pre-trained on more than 1 million hours of real egocentric human video. It learns by jointly predicting the next video frames and the actions, and that video prediction (world modeling) is what carries learning from humans to robots. • More human video predictably improves action prediction. • Task accuracy improves across embodiments (bimanual arms and a humanoid) with zero robot data in pre-training. • Post-trained on just a few hours of robot data, mean task performance rose 20% to 53% as human pre-training scaled 1k to 1M hours. WAM vs VLA, matched apple-to-apple: the WAM hit 1.55x the VLA's success rate and won 65% of head-to-heads. Deployed zero-shot at unseen customer sites, it passed 87% vs the VLA's 46%.

How well do egocentric human videos scale? Apparently, really well. Dyna Robotics has introduced Dyna-2, a world-action model (WAM) pre-trained on more than 1 million hours of real egocentric human video. It learns by jointly predicting the next video frames and the actions, and that video prediction (world modeling) is what carries learning from humans to robots. • More human video predictably improves action prediction. • Task accuracy improves across embodiments (bimanual arms and a humanoid) with zero robot data in pre-training. • Post-trained on just a few hours of robot data, mean task performance rose 20% to 53% as human pre-training scaled 1k to 1M hours. WAM vs VLA, matched apple-to-apple: the WAM hit 1.55x the VLA's success rate and won 65% of head-to-heads. Deployed zero-shot at unseen customer sites, it passed 87% vs the VLA's 46%.

The Humanoid Hub

15,970 просмотров • 1 месяц назад