正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Physical Intelligence

44,714 subscribers

116,845 次观看 • 1 年前 •via X (Twitter)

科学技术教育健康养生

Anya Rossi• Live Now

Private livecam show

11 条评论

Physical Intelligence 的头像

Physical Intelligence1 年前

Hi Robot uses a high-level VLM policy to parse instructions, camera observations, and user feedback (e.g., interjections) to decide what to do. Full blog post:

Physical Intelligence 的头像

Physical Intelligence1 年前

Just like when you are trying to cook a new kind of recipe, and you “talk” through it: first I add the salt, then the onions… the robot “talks” to itself to walk through a complex task, and then our π0 VLA follows these steps.

Physical Intelligence 的头像

Physical Intelligence1 年前

A person can talk to the robot while it’s doing this, interjecting, providing feedback, and making additional requests, like in this shopping scenario.

Physical Intelligence 的头像

Physical Intelligence1 年前

Hi Robot allows users to define new tasks through prompts, modifying how the robot should perform a task – for example, in this example the robot changes its behavior from the standard “bussing” policy based on user commands.

Physical Intelligence 的头像

Physical Intelligence1 年前

To learn more about Hi Robot, check out our blog post: The full-length paper is here:

Yang 的头像

Yang1 年前

Want to learn how practical AI skills and automations for your business and work? Check out our 50+ step-by-step video tutorials 100% FREE 20+ hours of Ai and Automation goodness absolutely free 🥳

Brett Peirce 的头像

Brett Peirce1 年前

Ummm.... The aluminum pan was recycling... Lol

Venky 的头像

Venky1 年前

Pretty cool. Curious - How is this different than reasoning or chain of thought in LLMs?

Allison Barr Allen 的头像

Allison Barr Allen1 年前

Very cool

Ryan Ray 🦏 的头像

Ryan Ray 🦏1 年前

and ppl still think it's not coming for blue collar

Oli 的头像

Oli1 年前

damn absoloutely phenomenal work

相关视频

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Sergey Levine

30,388 次观看 • 2 年前

Figure says it has increased humanoid robot production from one robot per day to one robot per hour in just 120 days. The robots undergo intensive testing, including repeated squats and jogging movements. Figure also demonstrated the robot's ability to climb stairs and navigate complex terrain using vision-based control.

Figure says it has increased humanoid robot production from one robot per day to one robot per hour in just 120 days. The robots undergo intensive testing, including repeated squats and jogging movements. Figure also demonstrated the robot's ability to climb stairs and navigate complex terrain using vision-based control.

Space and Technology

543,393 次观看 • 1 个月前

1958, The Robot Guest. This Robbie The Robot influenced many ideas of how we see and react to robots even today. It is also one reason anthropomorphic bipedal humanoid robots are easier psychologically.

1958, The Robot Guest. This Robbie The Robot influenced many ideas of how we see and react to robots even today. It is also one reason anthropomorphic bipedal humanoid robots are easier psychologically.

Brian Roemmele

19,785 次观看 • 6 个月前

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,435 次观看 • 1 年前

A Taiwanese startup is rethinking what a SCARA robot can do. They developed a robot that outperforms typical SCARA models in speed, stability, and space efficiency. Unlike most conventional arms, it can also move smoothly through singularity positions, where many robots lose control or need complex workarounds. That matters more than it sounds. Singularity limits often prevent standard industrial robots from handling certain tasks. Solving it could unlock more advanced automation in tighter, more demanding spaces. Video Credit: Marc Theermann #robotics #robots

A Taiwanese startup is rethinking what a SCARA robot can do. They developed a robot that outperforms typical SCARA models in speed, stability, and space efficiency. Unlike most conventional arms, it can also move smoothly through singularity positions, where many robots lose control or need complex workarounds. That matters more than it sounds. Singularity limits often prevent standard industrial robots from handling certain tasks. Solving it could unlock more advanced automation in tighter, more demanding spaces. Video Credit: Marc Theermann #robotics #robots

Wevolver

30,610 次观看 • 2 个月前

China unveils humanoid robot with lifelike skin and blinking eyes built for daily life | Prabhat Ranjan Mishra, Interesting Engineering Large Language Models (LLMs) and Vision-Language Models (VLMs) help process and interpret complex data from human interactions. A Shanghai-based company has developed humanoid robots that appear as real as humans. The advanced bionic humanoid robot is integrated with self-supervised AI algorithms. Named Elf V1, the robot can perceive the world, communicate, learn, and interact intelligently with its surroundings. Developed by AheadForm Technology, the robot offers up to 30 degrees of freedom, powered by a precise control system and an advanced AI learning algorithm. Robot offers expressive facial features The robot offers expressive facial features, moving eyes, and synchronized speech. It can also convey emotions and understand human non-verbal cues, making interactions more natural and engaging. The robot has highly interactive capabilities and lifelike appearances. AheadForm expects that its robots could soon seamlessly integrate into daily life, providing assistance, companionship, and support across various industries. “We believe that by developing realistic and expressive robot heads, we can bridge the gap between humans and machines, fostering a new era of interactive and intelligent robotics,” said the company in a statement. Reports revealed that to avoid the “uncanny valley” effect and be able to interact with us, they are given lifelike skin and capabilities to read our emotions and respond appropriately using dynamic expression simulation and emotion generation tech. Bionic skin and high-precision control system The Elf V1 series of humanoids features 30 facial muscles animated by brushless micro-motors and managed by a high-precision control system. Paired with an ability to detect their users’ emotions with low latency and bionic skin, their facial expressions are nearly identical to those of humans, reported CGTN. The company claims it’s pioneering the development of realistic humanoid robots designed to revolutionize human-robot interaction. It’s enhancing sophisticated humanoid robot heads that can express emotions, perceive their environment, and interact seamlessly with humans. By combining cutting-edge AI and advanced robotics, AheadForm aims to bring life to machines and transform how humans engage with technology. AI models boost robots’ responsiveness Seamless integration of Large Language Models (LLMs) and Vision-Language Models (VLMs) into the humanoid robots can help them process and interpret complex data from human interactions, enabling the robot to learn and adapt in real-time, achieving human-level understanding and responsiveness. AheadForm uses Brushless Motors that deliver ultra-quiet operation and high responsiveness, specifically designed for precision facial movements in humanoid robots. With its compact size, lightweight design, and energy efficiency, this motor is the ideal choice for next-generation robots that require precise, subtle facial control to create a truly human-like experience. Previously, the company unveiled the Lan Series that features realistic humanoid robots with soft skin and 10 degrees of freedom, offering a lifelike appearance and intuitive movements. This series is designed for cost-efficiency, for applications prioritizing mobility and manipulation.

China unveils humanoid robot with lifelike skin and blinking eyes built for daily life | Prabhat Ranjan Mishra, Interesting Engineering Large Language Models (LLMs) and Vision-Language Models (VLMs) help process and interpret complex data from human interactions. A Shanghai-based company has developed humanoid robots that appear as real as humans. The advanced bionic humanoid robot is integrated with self-supervised AI algorithms. Named Elf V1, the robot can perceive the world, communicate, learn, and interact intelligently with its surroundings. Developed by AheadForm Technology, the robot offers up to 30 degrees of freedom, powered by a precise control system and an advanced AI learning algorithm. Robot offers expressive facial features The robot offers expressive facial features, moving eyes, and synchronized speech. It can also convey emotions and understand human non-verbal cues, making interactions more natural and engaging. The robot has highly interactive capabilities and lifelike appearances. AheadForm expects that its robots could soon seamlessly integrate into daily life, providing assistance, companionship, and support across various industries. “We believe that by developing realistic and expressive robot heads, we can bridge the gap between humans and machines, fostering a new era of interactive and intelligent robotics,” said the company in a statement. Reports revealed that to avoid the “uncanny valley” effect and be able to interact with us, they are given lifelike skin and capabilities to read our emotions and respond appropriately using dynamic expression simulation and emotion generation tech. Bionic skin and high-precision control system The Elf V1 series of humanoids features 30 facial muscles animated by brushless micro-motors and managed by a high-precision control system. Paired with an ability to detect their users’ emotions with low latency and bionic skin, their facial expressions are nearly identical to those of humans, reported CGTN. The company claims it’s pioneering the development of realistic humanoid robots designed to revolutionize human-robot interaction. It’s enhancing sophisticated humanoid robot heads that can express emotions, perceive their environment, and interact seamlessly with humans. By combining cutting-edge AI and advanced robotics, AheadForm aims to bring life to machines and transform how humans engage with technology. AI models boost robots’ responsiveness Seamless integration of Large Language Models (LLMs) and Vision-Language Models (VLMs) into the humanoid robots can help them process and interpret complex data from human interactions, enabling the robot to learn and adapt in real-time, achieving human-level understanding and responsiveness. AheadForm uses Brushless Motors that deliver ultra-quiet operation and high responsiveness, specifically designed for precision facial movements in humanoid robots. With its compact size, lightweight design, and energy efficiency, this motor is the ideal choice for next-generation robots that require precise, subtle facial control to create a truly human-like experience. Previously, the company unveiled the Lan Series that features realistic humanoid robots with soft skin and 10 degrees of freedom, offering a lifelike appearance and intuitive movements. This series is designed for cost-efficiency, for applications prioritizing mobility and manipulation.

Owen Gregorian

179,005 次观看 • 8 个月前

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Introducing Yell At Your Robot (YAY Robot!) 🗣️- a fun collaboration b/w Stanford University and UC Berkeley 🤖 We enable robots to improve on-the-fly from language corrections: robots rapidly adapt in real-time and continuously improve from human verbal feedback. YAY Robot enables long-horizon, dexterous manipulation tasks like preparing trail-mix, packing a ziploc bag, and cleaning dishes:

Lucy Shi

122,827 次观看 • 2 年前

🚨Is it possible to devise an intuitive approach for crowdsourcing trainable data for robots without requiring a physical robot🤖? Can we democratize robot learning for all?🧑‍🤝‍🧑 Check out our latest #CoRL2023 paper-> AR2-D2: Training a Robot Without a Robot

🚨Is it possible to devise an intuitive approach for crowdsourcing trainable data for robots without requiring a physical robot🤖? Can we democratize robot learning for all?🧑‍🤝‍🧑 Check out our latest #CoRL2023 paper-> AR2-D2: Training a Robot Without a Robot

Jiafei Duan

38,871 次观看 • 2 年前

Our robots now run autonomously >80% of the time. We still have human teleoperators in-the-loop, who take control when the robot gets stuck or something goes wrong. As autonomy improves, we'll grow our robot-to-operator efficiency until we have one person overseeing 10+ robots. Useful, AI-powered robots are here. And it's only going to get better.

Our robots now run autonomously >80% of the time. We still have human teleoperators in-the-loop, who take control when the robot gets stuck or something goes wrong. As autonomy improves, we'll grow our robot-to-operator efficiency until we have one person overseeing 10+ robots. Useful, AI-powered robots are here. And it's only going to get better.

Ultra

26,315 次观看 • 9 个月前

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

RoboPapers

27,567 次观看 • 5 个月前

California startup announces breakthrough in general-purpose robotics with π0.5 AI — a vision-language-action model. Everything the robot sees, it sees for the first time. The robot has not been trained to recognize its surroundings. We are getting there! Only a few more years until we finally have household robots. A dream is coming true!

California startup announces breakthrough in general-purpose robotics with π0.5 AI — a vision-language-action model. Everything the robot sees, it sees for the first time. The robot has not been trained to recognize its surroundings. We are getting there! Only a few more years until we finally have household robots. A dream is coming true!

Chubby♨️

218,411 次观看 • 1 年前

A pentagon robot! 🖐🏽 Compared to typical SCARA models, it delivers higher speed, improved stability, better space efficiency, and can operate smoothly in tight areas, even through singularity positions. These robots are a good fit for handling complex tasks in challenging environments where conventional robots often fall short. Have you ever seen a robot like this? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

A pentagon robot! 🖐🏽 Compared to typical SCARA models, it delivers higher speed, improved stability, better space efficiency, and can operate smoothly in tight areas, even through singularity positions. These robots are a good fit for handling complex tasks in challenging environments where conventional robots often fall short. Have you ever seen a robot like this? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

42,565 次观看 • 3 个月前

A robot snake for rescue missions! 🛟 The RoBoa, a soft-bodied robot developed by students at ETH Zürich is designed to navigate disaster zones too dangerous for conventional robots. This "vine-like" search-and-rescue robot can snake through debris, locate trapped survivors, and even deliver supplies to inaccessible areas. Its sensor-packed head sends a live camera feed to operators who can control its movements remotely. The latest prototype can extend to 100 meters and adjust its diameter to suit mission needs.

A robot snake for rescue missions! 🛟 The RoBoa, a soft-bodied robot developed by students at ETH Zürich is designed to navigate disaster zones too dangerous for conventional robots. This "vine-like" search-and-rescue robot can snake through debris, locate trapped survivors, and even deliver supplies to inaccessible areas. Its sensor-packed head sends a live camera feed to operators who can control its movements remotely. The latest prototype can extend to 100 meters and adjust its diameter to suit mission needs.

Lukas Ziegler

24,788 次观看 • 1 年前

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Abhishek Gupta

11,994 次观看 • 1 年前

Announcing $60M for Matic Robots. We didn't ask "What's the most impressive robot we can demo?" We asked "What's the most useful robot we can ship? What comes after Roomba?" Customers answered with their wallets: It's Matic.

Announcing $60M for Matic Robots. We didn't ask "What's the most impressive robot we can demo?" We asked "What's the most useful robot we can ship? What comes after Roomba?" Customers answered with their wallets: It's Matic.

Mehul

1,708,473 次观看 • 5 个月前

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Researchers have developed Humanoid-GPT, a new AI system that helps humanoid robots perform complex movements and tasks. Tested on the Unitree G1 robot, it enables real-time whole-body control and can perform new actions without special training. During demonstrations, the robot successfully carried out a variety of movements and tasks, showing strong potential for real-world applications.

Space and Technology

22,052 次观看 • 17 天前

We can teach LLMs to write better robot code through natural language feedback. But can LLMs remember what they were taught and improve their teachability over time? Introducing our latest work, Learning to Learn Faster from Human Feedback with Language Model Predictive Control

We can teach LLMs to write better robot code through natural language feedback. But can LLMs remember what they were taught and improve their teachability over time? Introducing our latest work, Learning to Learn Faster from Human Feedback with Language Model Predictive Control

Jacky Liang

86,652 次观看 • 2 年前