Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Introducing the QUANXTA Zero Series, X Square Robot’s next-generation UMI data collection solution for embodied AI. The series includes three products: QUANXTA Zero-G0: VR headset + backpack + dual grippers QUANXTA Zero-G1: headband rig + dual grippers QUANXTA Zero-E0: headband rig Built for scalable embodied AI data production: Multimodal... show more

X Square Robot

1,971 subscribers

355,551 просмотров • 3 месяцев назад •via X (Twitter)

Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 4

Фото профиля RoboHub🤖

RoboHub🤖3 месяцев назад

Cool, we already got a chance to try it out early 👏👏👏

Фото профиля Clara Lafever Jane

Clara Lafever Jane3 месяцев назад

👌 👍

Фото профиля Sebastian Buzdugan

Sebastian Buzdugan3 месяцев назад

where do you correct calibration drift, because umi labels fail there first

Фото профиля Peyman Servati

Peyman Servati2 месяцев назад

How much?

Похожие видео

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

This year is truly the dawn of the “robot data oil” rush--spawning all kinds of pick-and-shovel tools to dig it up. X Square Robot, which offers in-home humanoid cleaning services, has just launched a full UMI/Egocentric data collection system: QUANXTA Zero. Real robots delivering actual on-site services is one powerful data collection method. Meanwhile, bodiless (no physical robot) UMI/Ego-centric( human-centric)data pipelines are also surging in popularity. Well,everyone’s racing to distill the physical world and extract the data that truly matters for advancing robot intelligence.

CyberRobo

21,950 просмотров • 3 месяцев назад

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

A robot performing a chemistry experiment might not sound that surprising. What’s interesting about this demo is how the robot was trained. Chinese embodied AI startup X Square Robot's TwinDEX uses a wearable three-finger end-effector to collect demonstrations from a human, rather than requiring the robot to be teleoperated during data collection. The resulting policy handles a 20+ step chemistry workflow: opening reagent bottles, scooping material, operating a pipette, transferring liquids, handling a glass rod, pouring, and coordinating both hands throughout the process. These are very different from simple pick-and-place tasks. There are continuous contacts, tool use, changing grasps, precise positioning, force control, and long-horizon state transitions. The reported experiments used robot-free demonstrations only, with zero on-robot teleoperation data for training. The bigger idea here is that if the data-collection hardware and deployment hardware are designed to be sufficiently consistent, robot-free demonstrations can still retain the information needed for real dexterous manipulation.

Sharon Riley

18,036 просмотров • 1 месяц назад

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 просмотров • 6 месяцев назад

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,836 просмотров • 6 месяцев назад

Sharing a copy of my ICRA'26 keynote talk from three months ago. 3 key messages: 1. Building a human-like machine, a dream that has fascinated humankind for centuries, is rapidly becoming a reality. 2. Human data is, by far, the most scalable source of training data to fuel robot foundation models. 3. World models are a "data sponge" that can absorb massive amounts of multimodal data for policy generalization. Video: Slides: The talk brought together our work over the past year: GR00T, EgoScale, SONIC, WAM, and more. Here’s the same demo I showed on stage: Our (now old!) model learning to assemble the YCB airplane with zero teleop data. We've made incredible progress since then. What's coming next? 🚀

Sharing a copy of my ICRA'26 keynote talk from three months ago. 3 key messages: 1. Building a human-like machine, a dream that has fascinated humankind for centuries, is rapidly becoming a reality. 2. Human data is, by far, the most scalable source of training data to fuel robot foundation models. 3. World models are a "data sponge" that can absorb massive amounts of multimodal data for policy generalization. Video: Slides: The talk brought together our work over the past year: GR00T, EgoScale, SONIC, WAM, and more. Here’s the same demo I showed on stage: Our (now old!) model learning to assemble the YCB airplane with zero teleop data. We've made incredible progress since then. What's coming next? 🚀

Yuke Zhu

11,330 просмотров • 15 дней назад

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

Impressive on every dimension! Genesis AI has announced GENE-26.5, its first robotic foundation model, aimed at achieving human-level dexterous manipulation: - Single model, shared weights, handles egg cracking, lab pipetting, bimanual Rubik's Cube, smoothie making, and wire harnessing. Most tasks: under 1 hour of task-specific data, 1x real-world speed. - Genesis Hand 1.0: human-size, 20 active DoF, soft-contact skin. Paired glove gives a 1:1:1 mapping (glove, human hand, robot hand). 100x cheaper than teleop, 5x more data-efficient internally. - Data engine pulls from three sources: glove data, egocentric (head-cam) video, and large-scale internet video. The 1:1 hand-to-human match closes the embodiment gap, letting Genesis use video data more effectively than rivals. - Flow matching across vision, tactile, proprioception, and language inputs. Per Genesis, scaling data and compute directly improves zero-shot

The Humanoid Hub

96,013 просмотров • 5 месяцев назад

First there was Skild AI showing zero-shot transfer to unfamiliar homes, now Figure has a humanoid doing whole-body tasks across 30 homes it’s never seen before. Helix 2.5 was pretrained from scratch on Index, Figure’s human-behavior dataset, then fine-tuned for those three tasks elsewhere. Zero-shot meaning new homes and objects, not tasks it was never trained to do. No data collection or fine-tuning in those 30 homes. adding Index pretraining raised complete task success from 9% to 56%, with no partial credit. They also found that doubling the human pretraining data predictably improved robot-action prediction, enough to forecast their largest run’s loss to four decimal places. Obviously 56% isn’t perfect, but getting that much more generalization from human experience alone is pretty exciting.

First there was Skild AI showing zero-shot transfer to unfamiliar homes, now Figure has a humanoid doing whole-body tasks across 30 homes it’s never seen before. Helix 2.5 was pretrained from scratch on Index, Figure’s human-behavior dataset, then fine-tuned for those three tasks elsewhere. Zero-shot meaning new homes and objects, not tasks it was never trained to do. No data collection or fine-tuning in those 30 homes. adding Index pretraining raised complete task success from 9% to 56%, with no partial credit. They also found that doubling the human pretraining data predictably improved robot-action prediction, enough to forecast their largest run’s loss to four decimal places. Obviously 56% isn’t perfect, but getting that much more generalization from human experience alone is pretty exciting.

Chris

15,972 просмотров • 26 дней назад

🚀 We just raised $40 million to build infrastructure for Physical AI! 🦾 AI is rapidly transforming critical industries like manufacturing, logistics, transportation, agriculture, construction, aerospace, and defense. Teams that win in the physical world are those who can create a data flywheel, leveraging infrastructure to capture, ingest, analyze, and evaluate the vast quantities of data generated by real-world systems. Robotics data is multimodal, time-synchronized, and bandwidth‑constrained at the edge. Traditional data and observability platforms were only designed to store and query text and time-series data, not petabyte-scale 3D, video, audio, GNSS, and proprioceptive data. The ability to efficiently capture, ingest, search, visualize, and evaluate multimodal data is critical to Physical AI development. Foxglove is a modern data engine for Physical AI, enabling you to record logs or capture demonstrations at the edge, sync recordings to the cloud or on-premises storage, find critical events across petabytes of data, evaluate robot performance, and watch a 3D frame-by-frame replay using our advanced visualization tool. 👉 Today is still Day 1 for Physical AI, and we're hiring for dozens of roles to assemble the best team in the industry. If you've built ML platforms, data infrastructure, dataset curation, evaluation and validation, or visualization tools at a leading robotics or autonomous vehicle company, let's chat – drop me a note or tag a friend below and I'll follow up personally! Thank you to Alexandra Sukin and Jeremy Levine at Bessemer, Seth Winterroth 🤖 at Eclipse, David Beyer and Sunil Dhaliwal at Amplify Partners, and Icehouse Ventures for joining us on this mission. Also a special shoutout to our angels tobi lutke Alex Kendall Kyle Vogt Milan Kovac Hussein Mehanna Pieter Abbeel Brad Porter Boris Sofman Kevin Peterson Chris Walti Lindon Gao Daniel Kan Adam Draper ⏻ Fred Ehrsam and Karri Saarinen!

🚀 We just raised $40 million to build infrastructure for Physical AI! 🦾 AI is rapidly transforming critical industries like manufacturing, logistics, transportation, agriculture, construction, aerospace, and defense. Teams that win in the physical world are those who can create a data flywheel, leveraging infrastructure to capture, ingest, analyze, and evaluate the vast quantities of data generated by real-world systems. Robotics data is multimodal, time-synchronized, and bandwidth‑constrained at the edge. Traditional data and observability platforms were only designed to store and query text and time-series data, not petabyte-scale 3D, video, audio, GNSS, and proprioceptive data. The ability to efficiently capture, ingest, search, visualize, and evaluate multimodal data is critical to Physical AI development. Foxglove is a modern data engine for Physical AI, enabling you to record logs or capture demonstrations at the edge, sync recordings to the cloud or on-premises storage, find critical events across petabytes of data, evaluate robot performance, and watch a 3D frame-by-frame replay using our advanced visualization tool. 👉 Today is still Day 1 for Physical AI, and we're hiring for dozens of roles to assemble the best team in the industry. If you've built ML platforms, data infrastructure, dataset curation, evaluation and validation, or visualization tools at a leading robotics or autonomous vehicle company, let's chat – drop me a note or tag a friend below and I'll follow up personally! Thank you to Alexandra Sukin and Jeremy Levine at Bessemer, Seth Winterroth 🤖 at Eclipse, David Beyer and Sunil Dhaliwal at Amplify Partners, and Icehouse Ventures for joining us on this mission. Also a special shoutout to our angels tobi lutke Alex Kendall Kyle Vogt Milan Kovac Hussein Mehanna Pieter Abbeel Brad Porter Boris Sofman Kevin Peterson Chris Walti Lindon Gao Daniel Kan Adam Draper ⏻ Fred Ehrsam and Karri Saarinen!

Adrian Macneil

46,718 просмотров • 11 месяцев назад

X Square Robot just closed its Series C at a valuation above RMB 20 billion, about $2.8 billion 🤖 IDG came into this round. The bigger signal is the cap table. HongShan and Xiaomi were already in across earlier rounds, while Meituan, Alibaba, ByteDance, and Xiaomi have each led rounds at different stages. That puts X Square in a rare position for an embodied AI company: top-tier financial capital on one side, and four of China’s biggest tech platforms on the other. This is not just a money story. Meituan, Alibaba, ByteDance, and Xiaomi bring very different strategic assets: real-world scenarios, cloud infrastructure, consumer traffic, supply chains, and hardware ecosystems. The deployment side is already moving: robot home-cleaning services first, then a “Robots Into Homes” program with the first batch entering real households. The model stack is worth watching too. X Square has open-sourced WALL-OSS-0.5 for robot manipulation and WALL-WM for world modeling. WALL-OSS-0.5 showed strong real-robot performance without post-training, while WALL-WM uses event-level prediction to align language, vision, and action around meaningful physical-world events. They are also building a model-driven data pipeline for large-scale collection, cleaning, annotation, quality control, and augmentation. That matters because home robotics dies in the long tail: weird rooms, messy objects, bad lighting, and tasks that never look the same twice. Founded in 2023, X Square is building general-purpose embodied AI robots and foundation models for real-world environments, tying models, robot hardware, high-precision manipulation, data, and deployment into one system.

X Square Robot just closed its Series C at a valuation above RMB 20 billion, about $2.8 billion 🤖 IDG came into this round. The bigger signal is the cap table. HongShan and Xiaomi were already in across earlier rounds, while Meituan, Alibaba, ByteDance, and Xiaomi have each led rounds at different stages. That puts X Square in a rare position for an embodied AI company: top-tier financial capital on one side, and four of China’s biggest tech platforms on the other. This is not just a money story. Meituan, Alibaba, ByteDance, and Xiaomi bring very different strategic assets: real-world scenarios, cloud infrastructure, consumer traffic, supply chains, and hardware ecosystems. The deployment side is already moving: robot home-cleaning services first, then a “Robots Into Homes” program with the first batch entering real households. The model stack is worth watching too. X Square has open-sourced WALL-OSS-0.5 for robot manipulation and WALL-WM for world modeling. WALL-OSS-0.5 showed strong real-robot performance without post-training, while WALL-WM uses event-level prediction to align language, vision, and action around meaningful physical-world events. They are also building a model-driven data pipeline for large-scale collection, cleaning, annotation, quality control, and augmentation. That matters because home robotics dies in the long tail: weird rooms, messy objects, bad lighting, and tasks that never look the same twice. Founded in 2023, X Square is building general-purpose embodied AI robots and foundation models for real-world environments, tying models, robot hardware, high-precision manipulation, data, and deployment into one system.

RoboHub🤖

12,975 просмотров • 3 месяцев назад

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC

We believe we’re the first robotics company to demonstrate a robot peeling an apple with dual dexterous human-like hands. This breakthrough closes a key gap in robotics, achieving bimanual, contact-rich manipulation and moving far beyond the limits of simple grippers. 🧵↓ Today’s AI models (VLMs) are excellent at perception but struggle with action. Controlling high-degree-of-freedom hands for tasks like this is incredibly complex, and precise finger-level teleoperation is nearly impossible for humans. Our first step was a shared-autonomy system: rather than controlling every finger, the operator triggers pre-learned skills like a “rotate apple or tennis ball” primitive via a keyboard press or pedal. This makes scalable data collection and RL training possible. How does the AI manage this? We created "MoDE-VLA" (Mixture of Dexterous Experts). It fuses vision, language, force, and touch data by using a team of specialist "experts," making control in high-dimensional spaces stable and effective. The combination of these two innovations allows for seamless, contact-rich manipulation. The human provides high-level guidance, and the robot executes the complex in-hand coordination required. This work paves the way for robots that can safely handle delicate tasks in human environments. Want the full technical details? 📄 Read the full research paper: Visit us at NVIDIA GTC Booth #1838, Hall 3 to learn more! #Robotics #AI #DexterousManipulation #VLA #NVIDIAGTC Nancy Villicaña NVIDIA GTC

Sharpa

20,693 просмотров • 7 месяцев назад

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,901 просмотров • 5 месяцев назад

Learning from Human Demonstrations: Show the Robot How to Act! The pipeline is very similar to older experiments using Gemini & pi0 with LeRobot. Pi-zero runs locally, while Gemini Flash generates the affordances and the high-level task. (More details are in the thread.) The new component is learning from demonstrations via Gemini 2.5 Pro. I capture a video while demoing & take one of the last frames. Gemini 2.5 Pro then extracts the instructions & passes them to Gemini Flash to process the scene. The fun part is that there's no fancy insight that came from me; other than the days spent figuring out the right prompts. It's the bitter lesson hitting you in the face -> Enhanced Gemini capabilities make this possible. For example, Gemini Flash cannot do Russian doll stacking, but Gemini 2.5 Pro can do it consistently. The current limitation is low-level manipulation: - As you can see, I'm aligning the objects so they are easy to grasp using the same technique from the training data. I couldn't get Gemini Flash to consistently output an accurate grasping angle, and Gemini 1.5 Pro was too expensive and slow for real-time deployment. - Getting a symmetrical gripper should also help a lot. Adding rubber to the tips would probably also help prevent objects from slipping. Collecting & curating the data was the most time consuming & labor intensive part. Next, to improve low-level manipulation and make the system more real-time, I'm shifting to focus more on sims & synthetic data. This aligns better with my core competence. I'm open to tips and suggestions.

Learning from Human Demonstrations: Show the Robot How to Act! The pipeline is very similar to older experiments using Gemini & pi0 with LeRobot. Pi-zero runs locally, while Gemini Flash generates the affordances and the high-level task. (More details are in the thread.) The new component is learning from demonstrations via Gemini 2.5 Pro. I capture a video while demoing & take one of the last frames. Gemini 2.5 Pro then extracts the instructions & passes them to Gemini Flash to process the scene. The fun part is that there's no fancy insight that came from me; other than the days spent figuring out the right prompts. It's the bitter lesson hitting you in the face -> Enhanced Gemini capabilities make this possible. For example, Gemini Flash cannot do Russian doll stacking, but Gemini 2.5 Pro can do it consistently. The current limitation is low-level manipulation: - As you can see, I'm aligning the objects so they are easy to grasp using the same technique from the training data. I couldn't get Gemini Flash to consistently output an accurate grasping angle, and Gemini 1.5 Pro was too expensive and slow for real-time deployment. - Getting a symmetrical gripper should also help a lot. Adding rubber to the tips would probably also help prevent objects from slipping. Collecting & curating the data was the most time consuming & labor intensive part. Next, to improve low-level manipulation and make the system more real-time, I'm shifting to focus more on sims & synthetic data. This aligns better with my core competence. I'm open to tips and suggestions.

Shreyas Gite

22,555 просмотров • 1 год назад

X Square Robot Unveils New Embodied AI Model, Says Robots Will Arrive in Homes in 35 Days Backed by Alibaba, ByteDance, Xiaomi and Meituan, X Square Robot unveiled a next-generation embodied AI foundation model for home robots and said its first deployments in everyday households will begin within 35 days. X Square Robot on Tuesday unveiled WALL-B, a new embodied AI foundation model designed for deployment in real-world homes, marking what the company described as a major step toward bringing general-purpose robots into daily family life. At a launch event themed "Born to Bot, Bot to Family," the company also introduced its World Unified Model (WUM) architecture, a training framework that combines vision, language, action and physical prediction within a single system from the outset. X Square said the model is intended to help robots operate in the far more unpredictable setting of a home, where tasks, layouts and interactions vary from moment to moment. "Robots in factories and in homes are completely different. In factories, they repeat the same action 10,000 times without variation. In a home, however, they need to perform 10,000 different actions, each unique and non-repetitive. Therefore, the challenge of a truly intelligent robot lies not in repeating a single action, but in the ability to execute new, untrained movements within unstructured environments. Deploying robots in the home is one of the most significant technical hurdles of our time," said Qian Wang, founder and CEO of X Square Robot. WALL-B is the first real-world implementation of the World Unified Model architecture. Unlike modular systems that train perception, language and control separately, X Square Robot said World Unified Model optimizes those capabilities jointly from the very beginning. The company said that allows physical prediction — including force, friction and collision dynamics — to emerge as part of the model itself, rather than being layered on afterward. "We train all capabilities—vision, language, action, and prediction—within the same network from day one. Much like infants, who do not learn to see, move and speak in isolated, sequential stages, but instead see, move listen and act simultaneously while receiving feedback, we have integrated all these capabilities into a unified whole," said Wang Hao, CTO of X Square. X Square Robot said the development of WALL-B rests on two pillars. The first is a data strategy that prioritizes training on authentic, non-staged home environments to cover the “long-tail” distribution of real-world scenarios, such as misplaced objects and temporary occlusions. Unlike models primarily trained on synthetic data or laboratory datasets, this strategy exposes WALL-B to the natural clutter of lived-in spaces—misplaced items, unexpected obstacles, and spontaneous human activity—ensuring that the training data reflects real-world conditions rather than a simplified version. The second is a physics-aware predictive mechanism that anticipates physical outcomes before an action is taken, enabling the model to respond to contact dynamics instead of just reacting. The development of the self-developed WUM architecture on physical robotic platforms highlights the company’s accumlated experience in bridging sim-to-real gaps across varied operational contexts. Wang commented that the current AI model is still in an "intern" stage, subject to errors requiring remote assistance. For instance, it may mistakenly place slippers in the kitchen or pause while wiping a table to "think". However, the model operates nonstop 24 hours a day, becoming increasingly "intelligent" as each day of operation generates new data. In 35 days, on May 25, X Square Robot will officially bring its robots into everyday homes, underscoring the company’s long-term commitment to the home robotics sector.

X Square Robot Unveils New Embodied AI Model, Says Robots Will Arrive in Homes in 35 Days Backed by Alibaba, ByteDance, Xiaomi and Meituan, X Square Robot unveiled a next-generation embodied AI foundation model for home robots and said its first deployments in everyday households will begin within 35 days. X Square Robot on Tuesday unveiled WALL-B, a new embodied AI foundation model designed for deployment in real-world homes, marking what the company described as a major step toward bringing general-purpose robots into daily family life. At a launch event themed "Born to Bot, Bot to Family," the company also introduced its World Unified Model (WUM) architecture, a training framework that combines vision, language, action and physical prediction within a single system from the outset. X Square said the model is intended to help robots operate in the far more unpredictable setting of a home, where tasks, layouts and interactions vary from moment to moment. "Robots in factories and in homes are completely different. In factories, they repeat the same action 10,000 times without variation. In a home, however, they need to perform 10,000 different actions, each unique and non-repetitive. Therefore, the challenge of a truly intelligent robot lies not in repeating a single action, but in the ability to execute new, untrained movements within unstructured environments. Deploying robots in the home is one of the most significant technical hurdles of our time," said Qian Wang, founder and CEO of X Square Robot. WALL-B is the first real-world implementation of the World Unified Model architecture. Unlike modular systems that train perception, language and control separately, X Square Robot said World Unified Model optimizes those capabilities jointly from the very beginning. The company said that allows physical prediction — including force, friction and collision dynamics — to emerge as part of the model itself, rather than being layered on afterward. "We train all capabilities—vision, language, action, and prediction—within the same network from day one. Much like infants, who do not learn to see, move and speak in isolated, sequential stages, but instead see, move listen and act simultaneously while receiving feedback, we have integrated all these capabilities into a unified whole," said Wang Hao, CTO of X Square. X Square Robot said the development of WALL-B rests on two pillars. The first is a data strategy that prioritizes training on authentic, non-staged home environments to cover the “long-tail” distribution of real-world scenarios, such as misplaced objects and temporary occlusions. Unlike models primarily trained on synthetic data or laboratory datasets, this strategy exposes WALL-B to the natural clutter of lived-in spaces—misplaced items, unexpected obstacles, and spontaneous human activity—ensuring that the training data reflects real-world conditions rather than a simplified version. The second is a physics-aware predictive mechanism that anticipates physical outcomes before an action is taken, enabling the model to respond to contact dynamics instead of just reacting. The development of the self-developed WUM architecture on physical robotic platforms highlights the company’s accumlated experience in bridging sim-to-real gaps across varied operational contexts. Wang commented that the current AI model is still in an "intern" stage, subject to errors requiring remote assistance. For instance, it may mistakenly place slippers in the kitchen or pause while wiping a table to "think". However, the model operates nonstop 24 hours a day, becoming increasingly "intelligent" as each day of operation generates new data. In 35 days, on May 25, X Square Robot will officially bring its robots into everyday homes, underscoring the company’s long-term commitment to the home robotics sector.

X Square Robot

53,212 просмотров • 5 месяцев назад

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Rohan Paul

11,253 просмотров • 3 месяцев назад

Everyone in Embodied AI is talking about Vision-Language-Action (VLA) models. Almost no one is talking about the physical nightmare of collecting the data to train them. You can't scrape a kitchen table or a warehouse shelf from a web browser. To get to millions of hours of diverse, real-world manipulation data, you need hundreds of rigs. But you can't buy them. If you build them out of off-the-shelf developer kits, they weigh 15 pounds, overheat in an hour, require bulky cabling, and break the first time an operator wears them on a job site. At the Instawork Robotics Lab (IRL), we had to build our own. Meet the Instacore: a rugged, 4lb wearable egocentric data-capture engine designed specifically to survive a full shift on a standard power pack. We didn't build a flashy humanoid. We did hard, blue-collar systems engineering: 💾 THE COMPUTE — A custom MediaTek Genio carrier board that runs completely fanless, routing 5 camera streams directly to on-board storage. ⚡ THE I/O PIPELINE — We ditched USB for industrial GMSL. Thin, ultra-flexible coaxial lines route high-speed data down to the backpack and Power-over-Coax (PoC) back up, completely eliminating batteries on the wrist. ⏱️ UNIFIED SENSOR CLOCK — The MediaTek Genio SoC drives a shared master clock straight to the ISPs driving our 5 global shutter sensors, stamping metadata at the microsecond of capture to ensure zero-drift temporal alignment. 👁️ OPTIMIZED OPTICS — 95 DFOV lenses on flexible PCB ribbon modules keep the wrist cams flat to prevent snags. A 145-degree chest camera captures the macro workspace, while a 50mm baseline rectilinear stereo head pair preserves close-up 3D mapping. To build hundreds of these, we took over a warehouse in Mountain View in April, called it the Instalab, and brought in talented Pros with assembly backgrounds from Tesla, Apple, and Meta. To test the systems, we had our Pros wear active rigs while assembling more rigs. The video below shows the raw, time-synchronized Foxglove playback of that exact loop. Now, we’re shipping these units globally to scale data collection for real Pros on the job.

Everyone in Embodied AI is talking about Vision-Language-Action (VLA) models. Almost no one is talking about the physical nightmare of collecting the data to train them. You can't scrape a kitchen table or a warehouse shelf from a web browser. To get to millions of hours of diverse, real-world manipulation data, you need hundreds of rigs. But you can't buy them. If you build them out of off-the-shelf developer kits, they weigh 15 pounds, overheat in an hour, require bulky cabling, and break the first time an operator wears them on a job site. At the Instawork Robotics Lab (IRL), we had to build our own. Meet the Instacore: a rugged, 4lb wearable egocentric data-capture engine designed specifically to survive a full shift on a standard power pack. We didn't build a flashy humanoid. We did hard, blue-collar systems engineering: 💾 THE COMPUTE — A custom MediaTek Genio carrier board that runs completely fanless, routing 5 camera streams directly to on-board storage. ⚡ THE I/O PIPELINE — We ditched USB for industrial GMSL. Thin, ultra-flexible coaxial lines route high-speed data down to the backpack and Power-over-Coax (PoC) back up, completely eliminating batteries on the wrist. ⏱️ UNIFIED SENSOR CLOCK — The MediaTek Genio SoC drives a shared master clock straight to the ISPs driving our 5 global shutter sensors, stamping metadata at the microsecond of capture to ensure zero-drift temporal alignment. 👁️ OPTIMIZED OPTICS — 95 DFOV lenses on flexible PCB ribbon modules keep the wrist cams flat to prevent snags. A 145-degree chest camera captures the macro workspace, while a 50mm baseline rectilinear stereo head pair preserves close-up 3D mapping. To build hundreds of these, we took over a warehouse in Mountain View in April, called it the Instalab, and brought in talented Pros with assembly backgrounds from Tesla, Apple, and Meta. To test the systems, we had our Pros wear active rigs while assembling more rigs. The video below shows the raw, time-synchronized Foxglove playback of that exact loop. Now, we’re shipping these units globally to scale data collection for real Pros on the job.

Ryan Hickman

27,290 просмотров • 4 месяцев назад

A mysterious embodied AI demo has recently sparked a lot of discussion. In the video, two robots from different manufacturers with significantly different hardware architectures, the Unitree G1 and AgiBot Yuanzheng A3, are reportedly running on the same “brain.” In a complex indoor environment, they work continuously for around 10 minutes in a single uncut take, performing a series of long-horizon tasks including cleaning windows, organizing objects, resuming interrupted tasks, and cooperating with each other. What makes it even more interesting is that when one robot cannot reach a high shelf, it attempts to use a box to solve the problem. The two robots also appear capable of cooperating based on each other’s physical capabilities. If the claimed level of autonomy and the use of the same model across different embodiments are eventually verified, I think there are three things that really deserve attention: 1. Cross-embodiment generalization. If the same foundation model can operate two substantially different robotic platforms, it could mean that robotic “intelligence” is gradually becoming decoupled from a specific physical body. 2. Long-horizon closed-loop execution. Continuously performing complex tasks for 10 minutes, while being able to pause, switch tasks, and later resume previous ones, is much more meaningful than completing a single 10-second demo. 3. Collaboration and dynamic planning. The two robots appear able to adjust their behavior according to environmental changes and each other’s physical capabilities. These are some of the characteristics that truly general-purpose embodied intelligence will eventually need. That said, I would remain cautious for now. The video demonstrates extremely impressive behavior, but stronger claims such as “self-evolution,” “true understanding of the physical world,” or overturning the Scaling Law with only dozens of hours of training data still require much stronger evidence. Failure recovery and replanning during a task also do not automatically demonstrate that the model is learning by itself. So I wouldn’t call this the “ChatGPT moment” of embodied AI yet. But if the team later discloses the model architecture, training data scale, level of human intervention, and can repeatedly reproduce these capabilities in completely unfamiliar environments, this seemingly rough 10-minute video could become one of the most memorable embodied AI demos of 2026. For now, my biggest question is simple: Who is the mysterious team behind it?

A mysterious embodied AI demo has recently sparked a lot of discussion. In the video, two robots from different manufacturers with significantly different hardware architectures, the Unitree G1 and AgiBot Yuanzheng A3, are reportedly running on the same “brain.” In a complex indoor environment, they work continuously for around 10 minutes in a single uncut take, performing a series of long-horizon tasks including cleaning windows, organizing objects, resuming interrupted tasks, and cooperating with each other. What makes it even more interesting is that when one robot cannot reach a high shelf, it attempts to use a box to solve the problem. The two robots also appear capable of cooperating based on each other’s physical capabilities. If the claimed level of autonomy and the use of the same model across different embodiments are eventually verified, I think there are three things that really deserve attention: 1. Cross-embodiment generalization. If the same foundation model can operate two substantially different robotic platforms, it could mean that robotic “intelligence” is gradually becoming decoupled from a specific physical body. 2. Long-horizon closed-loop execution. Continuously performing complex tasks for 10 minutes, while being able to pause, switch tasks, and later resume previous ones, is much more meaningful than completing a single 10-second demo. 3. Collaboration and dynamic planning. The two robots appear able to adjust their behavior according to environmental changes and each other’s physical capabilities. These are some of the characteristics that truly general-purpose embodied intelligence will eventually need. That said, I would remain cautious for now. The video demonstrates extremely impressive behavior, but stronger claims such as “self-evolution,” “true understanding of the physical world,” or overturning the Scaling Law with only dozens of hours of training data still require much stronger evidence. Failure recovery and replanning during a task also do not automatically demonstrate that the model is learning by itself. So I wouldn’t call this the “ChatGPT moment” of embodied AI yet. But if the team later discloses the model architecture, training data scale, level of human intervention, and can repeatedly reproduce these capabilities in completely unfamiliar environments, this seemingly rough 10-minute video could become one of the most memorable embodied AI demos of 2026. For now, my biggest question is simple: Who is the mysterious team behind it?

Ice Universe

28,209 просмотров • 1 месяц назад