Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

This looks like another step toward removing the data bottleneck in robotics. The bottleneck exists because recording demonstrations is easy, but labeling every grasp, hold, move, and release is slow and expensive. Perceptron Egocentric dropped a new robotics video annotation system. You give it raw robot-camera or first-person/egocentric video....

rohanpaul_ai's profile picture

Rohan Paul

152,742 subscribers

18,391 Aufrufe • vor 5 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Better data for robot hands! 🤲🏼 Embodied reasoning is the bottleneck now. It’s the data we are missing. Specifically, understanding what the robot's hands are doing throughout a task. Perceptron AI new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames. That matters because a lot happens between frames. Hand positions shift. Contacts change. The robot adjusts grip. You miss all that if you're sampling. 0.280 semantic end-to-end F1 score on WGO-Bench vs 0.158 from the best pipelines built on Gemini Robotics-ER 1.6. Training a robot to manipulate objects requires understanding every moment of hand-object interaction. If your training data has gaps: missing frames, missed contacts, unclear hand positions, the policy learns from incomplete information. You end up with robots that work in demos but fail in real tasks. Continuous hand tracking gives you the full picture. The policy learns from cleaner, more complete data. That compounds into better real-world performance. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:25

Better data for robot hands! 🤲🏼 Embodied reasoning is the bottleneck now. It’s the data we are missing. Specifically, understanding what the robot's hands are doing throughout a task. Perceptron AI new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames. That matters because a lot happens between frames. Hand positions shift. Contacts change. The robot adjusts grip. You miss all that if you're sampling. 0.280 semantic end-to-end F1 score on WGO-Bench vs 0.158 from the best pipelines built on Gemini Robotics-ER 1.6. Training a robot to manipulate objects requires understanding every moment of hand-object interaction. If your training data has gaps: missing frames, missed contacts, unclear hand positions, the policy learns from incomplete information. You end up with robots that work in demos but fail in real tasks. Continuous hand tracking gives you the full picture. The policy learns from cleaner, more complete data. That compounds into better real-world performance. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

17,926 Aufrufe • vor 5 Tagen

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji
1:24

Egocentric Videos for Robot Training 🧵 Researchers at NYU and UC Berkeley published research where they developed a system called EgoZero that trains robots using human demonstration videos recorded with smart glasses. 👓 The system converts first-person human actions into 3D point-based state-action representations. The policies executed on a gripper-equipped robot, achieved a 70% zero-shot success rate across seven manipulation tasks, with only 20 minutes of human data per task. EgoZero stands as one of the solid empirical proofs that egocentric smart-glass video collected from everyday human behavior can serve as powerful, scalable training data for real robot learning. Vincent Liu Ademi Adeniji

VaderResearch

23,055 Aufrufe • vor 10 Monaten

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:
0:29

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,804 Aufrufe • vor 5 Monaten

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 4 Monaten

Jensen Huang says agentic AI is not a chatbot. It is a "digital robot" -- and the next bottleneck is not the model. It is first-person data. "Agentic AI is just a digital robot. It understands, it reasons, it plans, and it acts and uses tools." Then he makes the bigger jump: "The world's video data is from a third person, not first person. And so agentic systems, robotic systems, physical AI -- the data is the hardest problem." That is the hidden constraint in robotics. LLMs got the internet. Physical AI needs demonstrations, synthetic worlds, reinforcement learning, and first-person experience. This is why Jensen keeps calling it a stack: robots are not waiting for one magical model. They need the data engine, the simulator, the reward system, the runtime, and the computer. The next wave of AI is not just software that talks. It is software that sees, plans, acts, and eventually moves through the world.
2:32

Jensen Huang says agentic AI is not a chatbot. It is a "digital robot" -- and the next bottleneck is not the model. It is first-person data. "Agentic AI is just a digital robot. It understands, it reasons, it plans, and it acts and uses tools." Then he makes the bigger jump: "The world's video data is from a third person, not first person. And so agentic systems, robotic systems, physical AI -- the data is the hardest problem." That is the hidden constraint in robotics. LLMs got the internet. Physical AI needs demonstrations, synthetic worlds, reinforcement learning, and first-person experience. This is why Jensen keeps calling it a stack: robots are not waiting for one magical model. They need the data engine, the simulator, the reward system, the runtime, and the computer. The next wave of AI is not just software that talks. It is software that sees, plans, acts, and eventually moves through the world.

Karl Mehta

14,140 Aufrufe • vor 19 Tagen

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”
2:16

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

TBPN

18,811 Aufrufe • vor 1 Jahr

Co-training has become a key part of the recipe for training large robotics models; it means that you mix some proportion of real robot data with other data sources, like simulation or egocentric human video data. This is especially important because robotics data tends to lack diversity which can be somewhat compensated for by the inclusion of these other modalities. And yet there has not been a sizable study on what constitute good practices for cotraining until now! We talk to Fanqi Lin and Jose Barreiros about their new work, a massive study which evaluated 89 policies over thousands of rollouts to tell us which forms of co-training were most useful for robotics. Watch episode 70 of RoboPapers, with Michael Cho and Chris Paxton, now!
1:24:52

Co-training has become a key part of the recipe for training large robotics models; it means that you mix some proportion of real robot data with other data sources, like simulation or egocentric human video data. This is especially important because robotics data tends to lack diversity which can be somewhat compensated for by the inclusion of these other modalities. And yet there has not been a sizable study on what constitute good practices for cotraining until now! We talk to Fanqi Lin and Jose Barreiros about their new work, a massive study which evaluated 89 policies over thousands of rollouts to tell us which forms of co-training were most useful for robotics. Watch episode 70 of RoboPapers, with Michael Cho and Chris Paxton, now!

RoboPapers

19,152 Aufrufe • vor 3 Monaten

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.
1:09

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.

東京ロボティクス株式会社

10,860 Aufrufe • vor 1 Jahr

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.
0:50

The problem with humanoid teleoperation is that it is expensive and difficult to scale Enter NVIDIA's EgoScale: - A VLA model pretrained on thousands hours of egocentric human videos. - Mid-trained via 50 hours of human + 4 hours of robot "play" data for human-robot alignment. - Fine-tuned with very few examples of task-specific robot teleoperation (100 or fewer per task). - Successfully transfers across 5-finger (Sharpa) and 3-finger (Unitree G1) robot hands. - Performance scales predictably as data increases.

The Humanoid Hub

44,441 Aufrufe • vor 4 Monaten

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T
1:25

NVIDIA just announced EgoScale 🤖🧠 NVIDIA Research has uncovered a log-linear scaling law for robot dexterity by pretraining VLA models on over 20,000 hours of egocentric human video This massive dataset is 20 times larger than previous efforts and proves that robot intelligence follows a predictable path: the more human data, the lower the loss The secret is a simple recipe combining large-scale human pretraining with a small amount of aligned human-robot mid-training to bridge the gap In testing, this method boosted the average success rate by 54% on a 22-DoF robotic hand compared to policies built without pretraining EgoScale also enables one-shot task adaptation and works across different hardware, suggesting that human motion is a universal motor prior for robots Website: Paper: Source: NVIDIA Research #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #NVIDIA #EgoScale #GR00T

RoboHub🤖

43,752 Aufrufe • vor 4 Monaten

Awesome! This is literally a superhuman hand. 🤖✋ ChangingTek Robotics just dropped a dexterous hand that actually morphs between left and right hand configurations on the fly. It uses cable-driven tech to pull off shapes and reverse bends that human hands can't even dream of doing. The power is real too, with a 50N grip force and 10N at the fingertips while staying precise within 0.1N. It is kind of like a rigid octopus, meaning it is tough enough to crank a wrench or screwdriver but smooth enough to grab a drink. Having this much range of motion in a single hand basically doubles the efficiency of any robotic arm in tight spots. Source: ChangingTek Robotics #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #ChangingTek #DexterousHand #Bionics
0:33

Awesome! This is literally a superhuman hand. 🤖✋ ChangingTek Robotics just dropped a dexterous hand that actually morphs between left and right hand configurations on the fly. It uses cable-driven tech to pull off shapes and reverse bends that human hands can't even dream of doing. The power is real too, with a 50N grip force and 10N at the fingertips while staying precise within 0.1N. It is kind of like a rigid octopus, meaning it is tough enough to crank a wrench or screwdriver but smooth enough to grab a drink. Having this much range of motion in a single hand basically doubles the efficiency of any robotic arm in tight spots. Source: ChangingTek Robotics #Robot #Humanoid #Robotics #AI #EmbodiedAI #PhysicalAI #ChangingTek #DexterousHand #Bionics

RoboHub🤖

13,855 Aufrufe • vor 4 Monaten

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.
44:56

Robots has a data problem, in that robotics data is rare. While human video is quite common, it’s not usually directly usable for robots for a variety of reasons, most significantly that it’s missing explicit, accurate robot actions. Instead, Jeremy Collins proposes that we predict keypoint trajectories — basically, how any given point in an object will move as a robot performs a task. This lets us use action-free human video to train robot skills. Learn more by watching Episode #37 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton now.

RoboPapers

89,011 Aufrufe • vor 8 Monaten

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,223 Aufrufe • vor 3 Monaten

Holy… 🤯 this is literally a superhuman hand. One hand can act as both a left and a right hand. Depending on the task or the object being grasped, it can autonomously switch between left-hand and right-hand configurations, bend in reverse, and achieve degrees of freedom beyond a human hand,maximizing the efficiency of a single hand. it comes from a company in Jiangsu called ChangingTek Robotics. You could say the hand is a bit like an octopus,but it isn’t soft. It’s rigid and capable of gripping tools like wrenches, screwdrivers, drinks, and basically almost any object.
0:56

Holy… 🤯 this is literally a superhuman hand. One hand can act as both a left and a right hand. Depending on the task or the object being grasped, it can autonomously switch between left-hand and right-hand configurations, bend in reverse, and achieve degrees of freedom beyond a human hand,maximizing the efficiency of a single hand. it comes from a company in Jiangsu called ChangingTek Robotics. You could say the hand is a bit like an octopus,but it isn’t soft. It’s rigid and capable of gripping tools like wrenches, screwdrivers, drinks, and basically almost any object.

CyberRobo

65,363 Aufrufe • vor 4 Monaten

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →
1:23

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

46,829 Aufrufe • vor 7 Monaten

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.
0:05

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 Aufrufe • vor 4 Monaten

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:
0:35

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,025 Aufrufe • vor 5 Monaten

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:
0:36

A policy that teaches robot hands to touch things the way humans do... not just grab and move, but feel and adjust in real time. Robot manipulation research often stops at picking up objects and placing them. CGP goes further: it handles tasks like opening jars, flipping objects in-hand, wiping dishes, and grasping fragile eggs, the kind of dexterous, contact-rich skills that require constant micro-adjustments based on what the fingers are actually feeling. The robot doesn't just see what it's doing; it predicts what contact should feel like at each step, then checks whether reality matches the prediction. If a finger is slipping, the policy knows before the object drops. Works on real robot hands (both 4-finger and 5-finger designs) with tactile sensors embedded in the fingertips Robust to visual distractions! The robot keeps flipping a box correctly even when the camera view is disrupted, because it's grounding decisions in touch, not just vision. Baseline policies without contact grounding fail in predictable ways: slipping mid-task, incomplete motions, loss of grasp, CGP avoids these This is a meaningful step toward robots that can handle the physical world with the kind of reliable, adaptive grip that humans take for granted. Relevant for manufacturing, logistics, assistive robotics, and anywhere fragile or irregular objects need to be handled carefully. Published at RSS 2026, developed with Meta Reality Labs Research. Thanks for sharing, Zhengtong Xu / Zhengtong Xu ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,769 Aufrufe • vor 1 Monat

First FSD intervention by robot*irl in this interesting edge case The car is so slow and cautious around pedestrians/cyclists that it will just wait and wait and wait Granted, it’s a fairly unusual traffic situation with a driver in front of us interacting with a traffic supervisor and a lot of bikes around the car Human Pilot would’ve gone through here with ease, but the robot under supervision of robot*irl just doesn’t wanna move an inch What could be the issue here? You think this is safe and desirable behavior by FSD or does it feel like an imposed restriction and unnatural anxiety of cyclists?
1:26

First FSD intervention by robot*irl in this interesting edge case The car is so slow and cautious around pedestrians/cyclists that it will just wait and wait and wait Granted, it’s a fairly unusual traffic situation with a driver in front of us interacting with a traffic supervisor and a lot of bikes around the car Human Pilot would’ve gone through here with ease, but the robot under supervision of robot*irl just doesn’t wanna move an inch What could be the issue here? You think this is safe and desirable behavior by FSD or does it feel like an imposed restriction and unnatural anxiety of cyclists?

Kees Roelandschap

40,685 Aufrufe • vor 2 Monaten