正在加载视频...

视频加载失败

返回首页

Our CTO boyuan is demoing our new localization algorithm, which surpasses leading industry solutions. Localization enables a robot to determine its position in an environment, which is essential for navigation. While most systems require robots to start from a predefined location, ours doesn’t. It lets robots boot up anywhere...

openmind_agi's profile picture

OpenMind

151,789 subscribers

109,243 次观看 • 8 个月前 •via X (Twitter)

教育科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Demonstrating our latest localization system in action. At bootup, the robot has no prior position estimate, but within seconds it autonomously localizes itself using a fusion of three algorithms. Unlike our previous version, this updated system incorporates vision, allowing the robot to adapt to real-world changes (e.g., moved furniture) by recognizing previously seen environments. In this demo, we repeatedly reset navigation and reposition the robot to random locations, showing robust, repeatable localization. The robot then executes a full patrol, following a planned path (visualized in RViz) with real-time path tracking. Because our software is hardware-agnostic, it brings the same reliable performance to any robot it runs on.
1:52

Demonstrating our latest localization system in action. At bootup, the robot has no prior position estimate, but within seconds it autonomously localizes itself using a fusion of three algorithms. Unlike our previous version, this updated system incorporates vision, allowing the robot to adapt to real-world changes (e.g., moved furniture) by recognizing previously seen environments. In this demo, we repeatedly reset navigation and reposition the robot to random locations, showing robust, repeatable localization. The robot then executes a full patrol, following a planned path (visualized in RViz) with real-time path tracking. Because our software is hardware-agnostic, it brings the same reliable performance to any robot it runs on.

OpenMind

23,954 次观看 • 3 个月前

Training a Whole-Body Control Foundation Model -- new work from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->
1:56

Training a Whole-Body Control Foundation Model -- new work from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->

Chris Paxton

45,068 次观看 • 11 个月前

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.
2:49

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.

Sawyer Merritt

69,297 次观看 • 11 个月前

One of the why questions we’re always asking is how can we help operations employees access inventory more efficiently? We’re taking a big leap forward by introducing DeepFleet, an exciting new AI model that makes our one million+ robot fleet work smarter. Think of DeepFleet as an intelligent traffic management system for our fulfillment centers. Just like smart traffic systems reduce congestion in cities, DeepFleet coordinates our robots’ movements to find optimal paths and reduce bottlenecks. By leveraging our deep expertise in robotics and combining it with AWS tools like SageMaker, we’ve built an AI model that learns from our extensive data on inventory movement to make real-time decisions that improve robot travel time by 10% (which at our scale is pretty significant). What makes this particularly exciting is how AI and robotics create value across the board. For customers, it means faster delivery times and lower costs. For employees, it opens up new opportunities – our robots handle physically demanding tasks and heavy lifting, making our fulfillment centers safer, while our people take on more technical roles. This combination of robotics and generative AI is just getting started, and am looking forward to what we’ll invent next.
0:56

One of the why questions we’re always asking is how can we help operations employees access inventory more efficiently? We’re taking a big leap forward by introducing DeepFleet, an exciting new AI model that makes our one million+ robot fleet work smarter. Think of DeepFleet as an intelligent traffic management system for our fulfillment centers. Just like smart traffic systems reduce congestion in cities, DeepFleet coordinates our robots’ movements to find optimal paths and reduce bottlenecks. By leveraging our deep expertise in robotics and combining it with AWS tools like SageMaker, we’ve built an AI model that learns from our extensive data on inventory movement to make real-time decisions that improve robot travel time by 10% (which at our scale is pretty significant). What makes this particularly exciting is how AI and robotics create value across the board. For customers, it means faster delivery times and lower costs. For employees, it opens up new opportunities – our robots handle physically demanding tasks and heavy lifting, making our fulfillment centers safer, while our people take on more technical roles. This combination of robotics and generative AI is just getting started, and am looking forward to what we’ll invent next.

Andy Jassy

42,327 次观看 • 1 年前

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 次观看 • 1 年前

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA
1:21

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Flexion

69,622 次观看 • 9 天前

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.
50:15

Robots need to be able to apply pressure and make contact with objects as needed in order to accomplish their tasks. From compliance to working safely around humans to whole-body manipulation of heavy objects, combining force and position control can dramatically expand the capabilities of robots. This is especially true for legged robots, which have so much ability to exert forces on the world around them. But how do we train robots which can do this? Baoxiong Jia tells us more in our discussion of his team’s recent, Best Paper Award winning work on learning a unified policy for position and force control, called UniFP. To learn more, watch Episode #49 of RoboPapers, hosted by Michael Cho - Rbt/Acc and Chris Paxton.

RoboPapers

44,803 次观看 • 7 个月前

The First Manufacturer to Mass-Produce 5,000 Humanoid Robots AgiBOT rolled off its 5,000th humanoid robot (a Lingxi X2) today, marking a milestone for both the company and the humanoid robot industry as a whole: a significant step towards large-scale deployment in the real world. AgiBOT co-founder and CTO Peng Zhihui stated in a live said that these 5,000 units include the Yuanzheng series (1742), the Lingxi series (1846), and the Genie series (1412). These humanoid robots will be deployed in various settings, including scientific research, entertainment, shopping malls, factories, and warehouses. This also signals that AgiBOT has the capacity for mass production and is ready to meet future surges in customer demand. Its production partner, LingyiTech, was the main force behind the assembly of AgiBOT's 5,000 humanoid robots this year, and they are planning an automated production line with an annual capacity of 500,000 units. Next year will be a milestone year for humanoid robots, reaching 10,000 units annually. Including Tesla, 1X, UBTECH…
0:42

The First Manufacturer to Mass-Produce 5,000 Humanoid Robots AgiBOT rolled off its 5,000th humanoid robot (a Lingxi X2) today, marking a milestone for both the company and the humanoid robot industry as a whole: a significant step towards large-scale deployment in the real world. AgiBOT co-founder and CTO Peng Zhihui stated in a live said that these 5,000 units include the Yuanzheng series (1742), the Lingxi series (1846), and the Genie series (1412). These humanoid robots will be deployed in various settings, including scientific research, entertainment, shopping malls, factories, and warehouses. This also signals that AgiBOT has the capacity for mass production and is ready to meet future surges in customer demand. Its production partner, LingyiTech, was the main force behind the assembly of AgiBOT's 5,000 humanoid robots this year, and they are planning an automated production line with an annual capacity of 500,000 units. Next year will be a milestone year for humanoid robots, reaching 10,000 units annually. Including Tesla, 1X, UBTECH…

CyberRobo

47,095 次观看 • 7 个月前

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.
1:01

A cute humanoid robot from NYC Fauna Robotics has released a humanoid robot called Sprout, which is approximately 1 meter tall, lightweight, quiet, and features no dangerous edges and soft skin. Its appearance and interactions are designed to be "approachable," like a social companion. It is currently offered as a general-purpose robotics platform for developers, providing basic movement, perception, navigation, and expression capabilities, allowing real-world use cases to emerge within the community. Many humanoid robots are born in factories and warehouses, but Fauna Robotics aims to bring humanoid robots back into human living spaces. Fauna believes that if humanoid robots are to become as ubiquitous as computers and mobile phones in the future, the starting point must be a safe, friendly form suitable for homes, schools, and offices. Sprout is their first step in this direction.

CyberRobo

18,549 次观看 • 6 个月前

My view is that buying and holding real estate is not an effective investment strategy in our current economic environment, for a few reasons. 1) Real estate is more interest rate sensitive than it is inflation sensitive, so given our current circumstances it is likely to go down in real terms 2) It is a fixed asset that is easy to tax, which limits its impacts on your ability to diversify 3) Real estate is nailed down, so investing in it makes it more difficult to move money from one place to another That’s my view, in a nutshell. I’m curious to hear if you agree. #GovernmentDebt #debt #principles
0:54

My view is that buying and holding real estate is not an effective investment strategy in our current economic environment, for a few reasons. 1) Real estate is more interest rate sensitive than it is inflation sensitive, so given our current circumstances it is likely to go down in real terms 2) It is a fixed asset that is easy to tax, which limits its impacts on your ability to diversify 3) Real estate is nailed down, so investing in it makes it more difficult to move money from one place to another That’s my view, in a nutshell. I’m curious to hear if you agree. #GovernmentDebt #debt #principles

Ray Dalio

1,032,906 次观看 • 11 个月前

Real-time inference is a big challenge for VLAs. We’ve been working on a way to amortize inference delays in π0.5. Our new Real-Time Chunking (RTC) method speeds up π0.5 by allowing the robot to “think” while it’s moving, which makes it quite a bit faster! 🧵👇
0:29

Real-time inference is a big challenge for VLAs. We’ve been working on a way to amortize inference delays in π0.5. Our new Real-Time Chunking (RTC) method speeds up π0.5 by allowing the robot to “think” while it’s moving, which makes it quite a bit faster! 🧵👇

Sergey Levine

73,688 次观看 • 1 年前

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,350 次观看 • 1 年前

We have to face it. Robots will ultimately have a face. But why? The human face transmits information in a far more rapid manner than any other system. Working face to face with robots are going to require this signal. This is the face from AheafFrom robotics. For robotics companies that do not have human faces, I say best to start now or encourage an after market hobby garage builder face market. This is the way of it.
1:21

We have to face it. Robots will ultimately have a face. But why? The human face transmits information in a far more rapid manner than any other system. Working face to face with robots are going to require this signal. This is the face from AheafFrom robotics. For robotics companies that do not have human faces, I say best to start now or encourage an after market hobby garage builder face market. This is the way of it.

Brian Roemmele

27,755 次观看 • 8 个月前

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.
44:56

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.

RoboPapers

89,011 次观看 • 9 个月前

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.
1:31

🚨 Apple joins the robotics race! Researchers from Carnegie Mellon University and Apple have developed a robot collision avoidance system. They called it ARMOR and they implemented it on the Fourier GR-1 robot. What is ARMOR? 👘 ARMOR is a system that helps humanoid robots "see" their surroundings better by using small, smart depth sensors placed on their arms. It solves the problem of blind spots and occlusions in traditional robot perception, making robots much safer and more efficient. It uses Time-of-Flight (ToF) sensors to detect obstacles, providing a near-complete view of the robot’s environment. 🔭 A smart software powered by transformer-based AI learns from human movements to help robots plan their actions dynamically. ARMOR reduces collisions by an impressive 63.7%, making robots significantly better at navigating crowded or complex spaces. The system is fast as well! 💨 ARMOR-Policy cuts computational time by 26× compared to traditional planning methods, ensuring robots can act quickly. P.S. Thoughts on this? The research paper is in the comments! ~~~ ♻️ RT to help 1 robot find a new workplace.

Lukas Ziegler

169,472 次观看 • 1 年前

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:01:08

Just collecting manipulation data isn’t enough for robots - they need to be able to move around in the world, which has a whole different set of challenges from pure manipulation. And bringing navigation and manipulation together in a single framework is even more challenging. Enter HERMES, from Zhecheng Yuan and Tianming Wei. This is a four-stage process in which human videos are used to set up an RL sim-to-real training pipeline in order to overcome differences between robot and human kinematics, and used together with a navigation foundation model to move around in a variety of environments. To learn more, join us as Zhecheng Yuan and Tianming Wei tell us about how they built their system to perform mobile dexterous manipulation from human videos in a variety of environments. Watch Episode #45 of RoboPapers today, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

15,085 次观看 • 8 个月前

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.
0:40

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Runway

13,464 次观看 • 5 个月前

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:
0:36

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 次观看 • 3 个月前

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.
1:56

Who can say no to a humanoid robot that can actually feel a hug? 🤖 JQ Industries just built a fabric-based electronic skin for Unitree G1 — covering the torso, hands, and feet. This is not about appearance. It’s about giving robots real tactile sensing and pressure feedback. According to the company, the system includes tactile gloves, upper-body skin, and foot sensors. That means the robot can feel contact through its hands, detect pressure across its body, and use foot feedback to adjust balance. This is the missing layer in most humanoids. Vision alone can’t handle physical interaction. Robots need touch to grip safely, react to humans, and stay stable in real-world environments. JQ says its fiber sensors can detect forces as low as 0.01N, handle up to 3500kPa, and bend below 0.1mm radius. The system can also integrate with Isaac Sim, making tactile data usable for training and deployment. Unitree G1 is already built for manipulation and control — this adds the sensing layer it lacked. JQ says it’s now working with multiple humanoid companies, with some systems already shipping at scale. This is what happens when robots stop just seeing the world — and start physically feeling it.

RoboHub🤖

10,952 次观看 • 3 个月前