Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific...

pascal_bornet's profile picture

Pascal Bornet

122,408 subscribers

186,223 Aufrufe • vor 3 Monaten •via X (Twitter)

Nachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:
0:29

Tired of teleoperation? One human video → 1,000s of robot demos. (📍GitHub ) Scaling Robot Data Without Dynamics Simulation or Robot Hardware Real2Render2Real (R2R2R) is a new way to scale robot data without physics simulation or hardware. You take a phone scan + a single monocular human demo. It tracks the motion, renders photorealistic scenes, and generates diverse, robot-agnostic trajectories ready for training. > No teleop, no sim, no robot, just a phone and a video > Train VLA models and diffusion policies directly on the output > Supports multiple robot embodiments with kinematic consistency > 1000s of demos in 1/27 the time of real-world collection Thank you, Max Fu, for sharing!! Project: Paper: Code coming soon: It shows that with the right pipeline, you can scale robot learning data without touching a robot. One of the most interesting directions in scalable robotics today. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

42,804 Aufrufe • vor 5 Monaten

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER
0:30

What if robots could learn real-world tasks from your perspective… without ever touching a robot? This is a system that trains robot policies using nothing but human-first, egocentric video data from smart glasses. No robots, no teleop, no sensors, just humans doing real tasks in the real world. Why it matters ✅ Learns robot policies from 20 minutes of human video; zero robot demos ✅ Generalizes to new objects, views, and even robot morphologies ✅ Uses 3D points for interpretable, spatially grounded learning ✅ Deploys directly to real-world robots with strong zero-shot success Thank you, Vincent Liu, for sharing!!! Learn more here: 🔗 Paper: 🌐 Website: 📍 BOOKMARK FOR LATER

Ilir Aliu - eu/acc

10,509 Aufrufe • vor 1 Jahr

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:
0:35

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,025 Aufrufe • vor 6 Monaten

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,350 Aufrufe • vor 1 Jahr

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 Aufrufe • vor 3 Monaten

Better data for robot hands! 🤲🏼 Embodied reasoning is the bottleneck now. It’s the data we are missing. Specifically, understanding what the robot's hands are doing throughout a task. Perceptron AI new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames. That matters because a lot happens between frames. Hand positions shift. Contacts change. The robot adjusts grip. You miss all that if you're sampling. 0.280 semantic end-to-end F1 score on WGO-Bench vs 0.158 from the best pipelines built on Gemini Robotics-ER 1.6. Training a robot to manipulate objects requires understanding every moment of hand-object interaction. If your training data has gaps: missing frames, missed contacts, unclear hand positions, the policy learns from incomplete information. You end up with robots that work in demos but fail in real tasks. Continuous hand tracking gives you the full picture. The policy learns from cleaner, more complete data. That compounds into better real-world performance. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:25

Better data for robot hands! 🤲🏼 Embodied reasoning is the bottleneck now. It’s the data we are missing. Specifically, understanding what the robot's hands are doing throughout a task. Perceptron AI new Egocentric offering tracks both hands through an entire video instead of guessing from sampled frames. That matters because a lot happens between frames. Hand positions shift. Contacts change. The robot adjusts grip. You miss all that if you're sampling. 0.280 semantic end-to-end F1 score on WGO-Bench vs 0.158 from the best pipelines built on Gemini Robotics-ER 1.6. Training a robot to manipulate objects requires understanding every moment of hand-object interaction. If your training data has gaps: missing frames, missed contacts, unclear hand positions, the policy learns from incomplete information. You end up with robots that work in demos but fail in real tasks. Continuous hand tracking gives you the full picture. The policy learns from cleaner, more complete data. That compounds into better real-world performance. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

17,926 Aufrufe • vor 15 Tagen

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 5 Monaten

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

59,708 Aufrufe • vor 7 Monaten

Collecting data is a PAAAAAAAIN❗ If only there were a solution out there... (𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 😱) Jannik Grothusen, researchers and founder in stealth mode 👀 from TU Munich, has created a new tool that helps collect data for robots to learn how to perform tasks. It’s inspired by a device from Stanford, but it’s even better! Here’s why: ✅ It can collect and process data in real-time, meaning no delays. ✅ It’s super easy to control, so collecting data is much faster and simpler. ✅ It can be used with different robot parts because of its smart, modular design. Best of all, this amazing tool is 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲, so anyone can use it or improve it! Github: The system is inspired by Cheng Chi; Universal Manipulation Interface (UMI). Credit: Jannik Grothusen
0:37

Collecting data is a PAAAAAAAIN❗ If only there were a solution out there... (𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 😱) Jannik Grothusen, researchers and founder in stealth mode 👀 from TU Munich, has created a new tool that helps collect data for robots to learn how to perform tasks. It’s inspired by a device from Stanford, but it’s even better! Here’s why: ✅ It can collect and process data in real-time, meaning no delays. ✅ It’s super easy to control, so collecting data is much faster and simpler. ✅ It can be used with different robot parts because of its smart, modular design. Best of all, this amazing tool is 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲, so anyone can use it or improve it! Github: The system is inspired by Cheng Chi; Universal Manipulation Interface (UMI). Credit: Jannik Grothusen

Ilir Aliu

47,397 Aufrufe • vor 1 Jahr

Physical Intelligence (Physical Intelligence) is building a foundation model that can control any robot to do any task — what the team describes as the GPT moment for robotics. The company's cross-embodiment approach trains across many different robot platforms, and recent results show tasks being performed zero-shot that last year required hundreds of hours of data collection. In this episode of the Lightcone Podcast , co-founder Quan Vuong (Quan Vuong) sat down with Garry Tan, Jared Friedman, Diana, and Harj Taggar to talk about why robotics is finally ready for its scaling moment, how PI runs its models in the cloud rather than on-device, and the playbook for what Quan sees as a Cambrian explosion of vertical robotics companies. 00:00 — Robotics just got cheaper 00:41 — The GPT moment for robotics 02:24 — Why robots didn’t work before 05:30 — The breakthrough that changed everything 09:12 — The data problem 13:33 — Robots learning without data 15:05 — Robots folding laundry (for real) 22:18 — From engineering problem → ops problem 29:12 — The startup playbook 38:46 — Thousands of robotics startups are coming
49:26

Physical Intelligence (Physical Intelligence) is building a foundation model that can control any robot to do any task — what the team describes as the GPT moment for robotics. The company's cross-embodiment approach trains across many different robot platforms, and recent results show tasks being performed zero-shot that last year required hundreds of hours of data collection. In this episode of the Lightcone Podcast , co-founder Quan Vuong (Quan Vuong) sat down with Garry Tan, Jared Friedman, Diana, and Harj Taggar to talk about why robotics is finally ready for its scaling moment, how PI runs its models in the cloud rather than on-device, and the playbook for what Quan sees as a Cambrian explosion of vertical robotics companies. 00:00 — Robotics just got cheaper 00:41 — The GPT moment for robotics 02:24 — Why robots didn’t work before 05:30 — The breakthrough that changed everything 09:12 — The data problem 13:33 — Robots learning without data 15:05 — Robots folding laundry (for real) 22:18 — From engineering problem → ops problem 29:12 — The startup playbook 38:46 — Thousands of robotics startups are coming

Y Combinator

115,590 Aufrufe • vor 3 Monaten

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.
2:46

Training humanoid robots on teleoperation alone won’t scale, says Rhoda AI CEO Jagdeep Singh. Unlike self-driving cars - which basically have four actuators (left, right, speed up, and slow down) and operate in a single environment (the road) - humanoid robots are completely different: "You're dealing with the full dexterity of a human hand - 20 degrees of freedom per hand. Every object is different. Every type of task is different." The problem isn't just the quantity of data - the bigger issue is diversity in the data, and it's why many humanoid robot demos will struggle to adapt to the real world: "If all the data you have is data that you've intentionally collected, then you almost by definition haven't seen the corner cases. You haven't seen all those edge scenarios that cause failure." Jagdeep says teleoperation is useful for fine-tuning robot behavior, but for pretraining, it's completely inadequate.

TBPN

19,218 Aufrufe • vor 4 Monaten

This looks like another step toward removing the data bottleneck in robotics. The bottleneck exists because recording demonstrations is easy, but labeling every grasp, hold, move, and release is slow and expensive. Perceptron Egocentric dropped a new robotics video annotation system. You give it raw robot-camera or first-person/egocentric video. It gives back structured labels that look more like training data: where each small manipulation starts and ends, what the subtask is, what each hand is doing, where the hands are frame by frame, and which hand is left or right. So, Input: raw video of a person or robot doing a task. Output: a machine-readable breakdown of the physical actions inside that video. Basically, it is a machine that converts robot video into policy-training supervision. That means timestamps, subtask boundaries, per-hand actions, and left-right hand grounding. So this will turn large piles of robot or egocentric video into supervision for training robot policies.
0:58

This looks like another step toward removing the data bottleneck in robotics. The bottleneck exists because recording demonstrations is easy, but labeling every grasp, hold, move, and release is slow and expensive. Perceptron Egocentric dropped a new robotics video annotation system. You give it raw robot-camera or first-person/egocentric video. It gives back structured labels that look more like training data: where each small manipulation starts and ends, what the subtask is, what each hand is doing, where the hands are frame by frame, and which hand is left or right. So, Input: raw video of a person or robot doing a task. Output: a machine-readable breakdown of the physical actions inside that video. Basically, it is a machine that converts robot video into policy-training supervision. That means timestamps, subtask boundaries, per-hand actions, and left-right hand grounding. So this will turn large piles of robot or egocentric video into supervision for training robot policies.

Rohan Paul

18,391 Aufrufe • vor 15 Tagen

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:59

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,901 Aufrufe • vor 3 Monaten

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 Aufrufe • vor 1 Monat

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.
1:00

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Neuracore

40,620 Aufrufe • vor 8 Monaten

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →
1:23

Feeling what a robot feels is becoming real! 🫰 Fluid Reality just ran its first full end-to-end touch teleoperation system, and the demo shows why high-resolution haptics might be the missing piece in robot control. A 22 mm fingertip display with 32 independently actuated “bubbles” was mapped directly to tactile sensors on a robot hand. 🫧 Three fingers streamed real-time touch data robot → human, giving the operator actual contact feedback instead of waving their hands blind, like most teleop systems today. It’s still early, but this is a real step toward dexterous ops and high-quality data for training Physical AI systems. Companies involved: Samsung Electronics, Alt-Bionics, Inc. (the robot hand), MANUS™ (hand tracking), and Sensible Robotics (fingertip sensors) Touch isn’t a nice-to-have in robotics. It’s the difference between manipulation that works in the real world P.S. Guess how many touch receptors we humans, have in our hand! :) ~~ ♻ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

46,829 Aufrufe • vor 7 Monaten

A high-quality haptic control for robots as a weekend project? All for under $600. [📍 Save Github & arXiv for later] A team built DOGlove, an open-source haptic glove for robot teleoperation that you can assemble yourself for under $600. It is built for real control, not demos. • Tracks 21 degrees of freedom with a custom joint design • Provides cable-driven force feedback plus fingertip haptics • Can be used to collect data for imitation learning Affordable haptics could make teleoperation more reliable and improve the quality of data we use to train physical AI. Project link: Credit: DOGlove team, shared by Han Zhang —- Weekly robotics and AI insights. Subscribe free:
2:39

A high-quality haptic control for robots as a weekend project? All for under $600. [📍 Save Github & arXiv for later] A team built DOGlove, an open-source haptic glove for robot teleoperation that you can assemble yourself for under $600. It is built for real control, not demos. • Tracks 21 degrees of freedom with a custom joint design • Provides cable-driven force feedback plus fingertip haptics • Can be used to collect data for imitation learning Affordable haptics could make teleoperation more reliable and improve the quality of data we use to train physical AI. Project link: Credit: DOGlove team, shared by Han Zhang —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

18,290 Aufrufe • vor 6 Monaten

Robotics has a data scarcity problem - you simply can't scrape robot control data from webpages. Introducing GR00T-Mimic and GR00T-Gen: using both Graphics 1.0 & Graphics 2.0 to multiply your robot datasets by 1,000,000x. We trade compute for synthetic data, so we are not capped by the fundamental physical limit of 24 hrs/robot/day. Robotics is right in the thick of Moravec's paradox: things that are easy for humans turn out to be incredibly hard for machines. We are crushing the Moravec's paradox, one token at a time. > Graphics 1.0: Isaac simulators with manually written, GPU-accelerated physics and rendering equations. > Graphics 2.0: big neural nets (Cosmos) that repaint the pixels from sim textures to real, given an open-ended prompt. Robot data multiplier workflow: 1. GR00T-Teleop: use XR device like Apple Vision Pro to map human finger poses to humanoid hands. 2. GR00T-Mimic: given a human-collected task demonstration, we augment the actions in Isaac and filter out ones that fail the task. 3. GR00T-Gen: apply Graphics 1.0 and then Graphics 2.0 to produce tons of visual variations. Join us to build Physical AI together! Job application links in thread:
2:33

Robotics has a data scarcity problem - you simply can't scrape robot control data from webpages. Introducing GR00T-Mimic and GR00T-Gen: using both Graphics 1.0 & Graphics 2.0 to multiply your robot datasets by 1,000,000x. We trade compute for synthetic data, so we are not capped by the fundamental physical limit of 24 hrs/robot/day. Robotics is right in the thick of Moravec's paradox: things that are easy for humans turn out to be incredibly hard for machines. We are crushing the Moravec's paradox, one token at a time. > Graphics 1.0: Isaac simulators with manually written, GPU-accelerated physics and rendering equations. > Graphics 2.0: big neural nets (Cosmos) that repaint the pixels from sim textures to real, given an open-ended prompt. Robot data multiplier workflow: 1. GR00T-Teleop: use XR device like Apple Vision Pro to map human finger poses to humanoid hands. 2. GR00T-Mimic: given a human-collected task demonstration, we augment the actions in Isaac and filter out ones that fail the task. 3. GR00T-Gen: apply Graphics 1.0 and then Graphics 2.0 to produce tons of visual variations. Join us to build Physical AI together! Job application links in thread:

Jim Fan

163,670 Aufrufe • vor 1 Jahr

No wonder they say teleoperation is dead Peking University’s DAGroup released HumanNet, a massive 1 million hour dataset of human-centric videos that turns everyday internet footage into gold for embodied AI. It’s got first-person and third-person views, super detailed actions, object interactions, tool use, and long sequences of real behavior. Everything’s cleaned up and annotated with 3D poses, SLAM tracks, and robot-ready labels. The crazy part is,Training on just 1,000 hours of their first-person videos performs as well as (or even slightly better than) 100 hours of actual robot teleop data on downstream tasks. Basically, human videos can now stand in for expensive robot data. Scaling laws for robotics just got a whole lot more realistic. Human priors + smart curation = the future of world models.
0:52

No wonder they say teleoperation is dead Peking University’s DAGroup released HumanNet, a massive 1 million hour dataset of human-centric videos that turns everyday internet footage into gold for embodied AI. It’s got first-person and third-person views, super detailed actions, object interactions, tool use, and long sequences of real behavior. Everything’s cleaned up and annotated with 3D poses, SLAM tracks, and robot-ready labels. The crazy part is,Training on just 1,000 hours of their first-person videos performs as well as (or even slightly better than) 100 hours of actual robot teleop data on downstream tasks. Basically, human videos can now stand in for expensive robot data. Scaling laws for robotics just got a whole lot more realistic. Human priors + smart curation = the future of world models.

CyberRobo

27,048 Aufrufe • vor 2 Monaten

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:
0:46

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,199 Aufrufe • vor 2 Jahren