Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

How to master multi-robot cooperative mapping? We're excited to release MR.ScaleMaster, which masters the scales to scale - any feed-forward Visual SLAM (MASt3R-SLAM, π³, VGGT-SLAM 2.0, LoGeR, …) - across any robot (legged, wheeled, handheld, …) 🔗

GiseopK's profile picture

Giseop Kim

1,884 subscribers

28,881 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

SOMEONE JUST BUILT AN OPERATING SYSTEM FOR ROBOTS not a framework... not a library an actual OS dimos is an open-source platform that lets you build, run, and deploy software on any robot humanoids, quadrupeds, drones, wheeled bots you write python modules .... they talk to each other via typed streams...just like microservices, but for physical hardware the wild part..? you can give your AI editor (cursor, claude code) MCP access to send real physical commands "move forward 1 meter" → robot moves. it ships with a full navigation stack SLAM, collision avoidance, route planning. plus a dashboard to debug, visualize, and control your robot in real-time via waypoints, VR, or keyboard. supported hardware already includes unitree G1 humanoids, dji drones, and multiple robot arms. programming physical robots is finally as simple as programming software
3:35

SOMEONE JUST BUILT AN OPERATING SYSTEM FOR ROBOTS not a framework... not a library an actual OS dimos is an open-source platform that lets you build, run, and deploy software on any robot humanoids, quadrupeds, drones, wheeled bots you write python modules .... they talk to each other via typed streams...just like microservices, but for physical hardware the wild part..? you can give your AI editor (cursor, claude code) MCP access to send real physical commands "move forward 1 meter" → robot moves. it ships with a full navigation stack SLAM, collision avoidance, route planning. plus a dashboard to debug, visualize, and control your robot in real-time via waypoints, VR, or keyboard. supported hardware already includes unitree G1 humanoids, dji drones, and multiple robot arms. programming physical robots is finally as simple as programming software

Vaishnavi

19,745 Aufrufe • vor 4 Monaten

🚀 Excited to attend my first #NeurIPS2025 and present our lighting-aware SLAM method, NFL-BA! 🔗 🔦 Why this matters: Traditional Bundle Adjustment (BA) in SLAM assumes static lighting, but many real scenarios—endoscopy, search & rescue, subterranean robotics—use co-located light + camera, creating dynamic, near-field lighting that breaks these assumptions. ✨ What we introduce: Near-Field Lighting Bundle Adjustment Loss (NFL-BA) — a formulation that explicitly models near-field illumination inside the BA objective, allowing SLAM systems to jointly reason about geometry, appearance, and lighting. 📈 Results: Modeling lighting directly leads to ~38% improvement in mapping & tracking across dynamic-lighting sequences in both endoscopy and indoor scenes. 🧩 Plug-and-play: NFL-BA integrates seamlessly into existing neural rendering–based SLAM pipelines, using both implicit (NeRF-style) or explicit (3DGS) scene representations. 🌟 If you’re interested in SLAM, neural rendering, or illumination modeling, come check it out on Friday's poster session, 11-2pm, #4407.
0:38

🚀 Excited to attend my first #NeurIPS2025 and present our lighting-aware SLAM method, NFL-BA! 🔗 🔦 Why this matters: Traditional Bundle Adjustment (BA) in SLAM assumes static lighting, but many real scenarios—endoscopy, search & rescue, subterranean robotics—use co-located light + camera, creating dynamic, near-field lighting that breaks these assumptions. ✨ What we introduce: Near-Field Lighting Bundle Adjustment Loss (NFL-BA) — a formulation that explicitly models near-field illumination inside the BA objective, allowing SLAM systems to jointly reason about geometry, appearance, and lighting. 📈 Results: Modeling lighting directly leads to ~38% improvement in mapping & tracking across dynamic-lighting sequences in both endoscopy and indoor scenes. 🧩 Plug-and-play: NFL-BA integrates seamlessly into existing neural rendering–based SLAM pipelines, using both implicit (NeRF-style) or explicit (3DGS) scene representations. 🌟 If you’re interested in SLAM, neural rendering, or illumination modeling, come check it out on Friday's poster session, 11-2pm, #4407.

Roni Sengupta

21,162 Aufrufe • vor 8 Monaten

MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction TL;DR: The first RGB-only multi-agent 3D Gaussian Splatting SLAM for collaborative photorealistic scene reconstruction. Contributions: (1) We propose the first monocular RGB-only multi-agent 3D Gaussian Splatting SLAM system. It integrates Gaussian front-ends, compact submap summaries, inter-agent verification, Sim(3) submap pose graph, and occupancy-aware fusion into a unified framework, achieving accurate tracking and photorealistic reconstruction without depth sensors. (2) We propose a Pose-Graph Bundle Adjustment (PGBA)-consistent Sim(3) loop closure mechanism for multi-agent systems, which jointly resolves intra- and inter-agent scale drift through a submap-level Sim(3) pose graph coupling geometric and photometric residuals. Robustness is ensured by a spatial-extent gate that rejects degenerate loops and an adaptive edge invalidation scheme consistent with evolving PGBA corrections. (3) We propose an occupancy-aware fusion framework for coherent multi-agent Gaussian maps. It combines occupancy-grid deduplication, decoupled coordinator, and joint pose-Gaussian photometric refinement to eliminate duplicated Gaussians, residual misalignment, and photometric seams across agents. (4) We introduce ReplicaMultiagent Plus dataset. While existing multi-agent datasets are typically limited to 2-3 agents with short trajectories, our dataset scales to 4 agents with long-horizon trajectories. In addition, we provide ground-truth geometry and semantic annotations, supporting the evaluation of monocular, RGB-D, and semantic multi-agent SLAM for collaborative dense reconstruction.
0:29

MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction TL;DR: The first RGB-only multi-agent 3D Gaussian Splatting SLAM for collaborative photorealistic scene reconstruction. Contributions: (1) We propose the first monocular RGB-only multi-agent 3D Gaussian Splatting SLAM system. It integrates Gaussian front-ends, compact submap summaries, inter-agent verification, Sim(3) submap pose graph, and occupancy-aware fusion into a unified framework, achieving accurate tracking and photorealistic reconstruction without depth sensors. (2) We propose a Pose-Graph Bundle Adjustment (PGBA)-consistent Sim(3) loop closure mechanism for multi-agent systems, which jointly resolves intra- and inter-agent scale drift through a submap-level Sim(3) pose graph coupling geometric and photometric residuals. Robustness is ensured by a spatial-extent gate that rejects degenerate loops and an adaptive edge invalidation scheme consistent with evolving PGBA corrections. (3) We propose an occupancy-aware fusion framework for coherent multi-agent Gaussian maps. It combines occupancy-grid deduplication, decoupled coordinator, and joint pose-Gaussian photometric refinement to eliminate duplicated Gaussians, residual misalignment, and photometric seams across agents. (4) We introduce ReplicaMultiagent Plus dataset. While existing multi-agent datasets are typically limited to 2-3 agents with short trajectories, our dataset scales to 4 agents with long-horizon trajectories. In addition, we provide ground-truth geometry and semantic annotations, supporting the evaluation of monocular, RGB-D, and semantic multi-agent SLAM for collaborative dense reconstruction.

MrNeRF

19,499 Aufrufe • vor 3 Monaten

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.
2:49

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.

Sawyer Merritt

69,381 Aufrufe • vor 1 Jahr

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!
0:29

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Bilawal Sidhu

704,068 Aufrufe • vor 1 Jahr

Elon Musk just dropped a major Optimus update: "We're in the final stages of completion of Optimus 3, which is really going to be by far the most advanced robot in the world. Nothing's even close I haven't even seen any demos of robots that are as good as Optimus 3" Production starts this summer - very slow at first, following the classic S-curve ramp High-volume production expected around summer next year Optimus 4 design will be complete next year, with plans to release an improved robot design every single year Tesla is about to lap the entire robotics industry
0:53

Elon Musk just dropped a major Optimus update: "We're in the final stages of completion of Optimus 3, which is really going to be by far the most advanced robot in the world. Nothing's even close I haven't even seen any demos of robots that are as good as Optimus 3" Production starts this summer - very slow at first, following the classic S-curve ramp High-volume production expected around summer next year Optimus 4 design will be complete next year, with plans to release an improved robot design every single year Tesla is about to lap the entire robotics industry

X Freeze

70,358 Aufrufe • vor 5 Monaten

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork
0:20

We might be solving the wrong problem in robotics. That’s what this makes clear. UMI → Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything → hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation → ~35 demos/hour UMI → ~111 demos/hour And the data transfers across robots → UR5, Franka, others. The design is surprisingly practical: → GoPro fisheye lens (155° FOV) + mirrors for depth → SLAM + IMU for precise 6DoF tracking → latency matching for dynamic tasks → diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove → deployed in 500+ homes → ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso → with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments → to learning directly from humans at scale So here’s the real question: Will robotics be unlocked by better models… or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,551 Aufrufe • vor 4 Monaten

I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to Rerun and Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses prefix.dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!
0:32

I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to Rerun and Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses prefix.dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!

Pablo Vela

40,864 Aufrufe • vor 4 Monaten

Once we started to work with large global retailers, we needed a better way to scale this process. Ideally, the staff at the store could do this themselves — rather than us flying our team across the world — and then we could lower the cost and timelines. So we built a self-serve version of our survey app, with a tutorial mode designed for beginners. Over time, we collected millions of data points, and so we were able to develop an algorithm which would auto-correct mistakes. In other words, if the surveyor accidentally placed their ground-truth location in the wrong place on the map, we could use our algorithms to detect it, and correct it. So now we have WiFi, and with and our efforts on producing a high quality survey, we have the best WiFi positioning available. With WiFi on its own, it’s achieving 3 meter accuracy. This is a great foundation to build on. WiFi + Motion data To refine this down to 1-meter accuracy, we realised that we could combine WiFi with the same technology behind self-driving cars and robotics: a motion system called SLAM (Simultaneous Localization and Mapping). SLAM uses the accelerometer, gyroscope and camera system to understand precise device motion. Imagine a car driving through a tunnel, using the motion since its last GPS ping to keep location accurate until it comes out the other side. On a phone, this technology is very reliable, and measures device motion with high precision. But SLAM is measuring motion within its own coordinate space, it’s not aligned with the real world. SLAM tracks the user’s relative motion, like “moved forward 2 meters, then turned left”, but does “forward” mean “north”, or some other direction? It’s not calibrated, so it could mean any location, any direction. We can’t rely on the compass to help us out with this, because phone compasses are notoriously incorrect — everyone knows the frustration of being sent the wrong way down a street. So our job was to align this motion data with the triangulation data we were receiving from WiFi. We designed an algorithm that could simulate every possibility, filter the unlikely scenarios, and hone in your location, using WiFi as an anchor. So WiFi gives us the initial blue dot, SLAM gives us motion, and as the user starts walking and we receive more data, our algorithms can refine location accuracy down to a consistent 1-meter accuracy. We’ve tested these algorithms in many locations, on hundreds of hours of ground-truth data:
0:47

Once we started to work with large global retailers, we needed a better way to scale this process. Ideally, the staff at the store could do this themselves — rather than us flying our team across the world — and then we could lower the cost and timelines. So we built a self-serve version of our survey app, with a tutorial mode designed for beginners. Over time, we collected millions of data points, and so we were able to develop an algorithm which would auto-correct mistakes. In other words, if the surveyor accidentally placed their ground-truth location in the wrong place on the map, we could use our algorithms to detect it, and correct it. So now we have WiFi, and with and our efforts on producing a high quality survey, we have the best WiFi positioning available. With WiFi on its own, it’s achieving 3 meter accuracy. This is a great foundation to build on. WiFi + Motion data To refine this down to 1-meter accuracy, we realised that we could combine WiFi with the same technology behind self-driving cars and robotics: a motion system called SLAM (Simultaneous Localization and Mapping). SLAM uses the accelerometer, gyroscope and camera system to understand precise device motion. Imagine a car driving through a tunnel, using the motion since its last GPS ping to keep location accurate until it comes out the other side. On a phone, this technology is very reliable, and measures device motion with high precision. But SLAM is measuring motion within its own coordinate space, it’s not aligned with the real world. SLAM tracks the user’s relative motion, like “moved forward 2 meters, then turned left”, but does “forward” mean “north”, or some other direction? It’s not calibrated, so it could mean any location, any direction. We can’t rely on the compass to help us out with this, because phone compasses are notoriously incorrect — everyone knows the frustration of being sent the wrong way down a street. So our job was to align this motion data with the triangulation data we were receiving from WiFi. We designed an algorithm that could simulate every possibility, filter the unlikely scenarios, and hone in your location, using WiFi as an anchor. So WiFi gives us the initial blue dot, SLAM gives us motion, and as the user starts walking and we receive more data, our algorithms can refine location accuracy down to a consistent 1-meter accuracy. We’ve tested these algorithms in many locations, on hundreds of hours of ground-truth data:

Andrew Hart

90,946 Aufrufe • vor 1 Jahr

Jannik Sinner on facing Alexander Zverev in the Wimbledon final “Less exchanges. Less rhythm. This we know because of the surface being that fast. Being also quite warm it’s gonna be good for a big server. He’s definitely a big server. In any case, I try to do my best on Sunday. It feels good again being back in a grand slam final. It’s good for me. I’ll just try to do my best. Whatever happened in the past between me and him, it happened. In between, he won a slam in Paris, which gave him a lot of confidence. We can see the level he’s producing here. We all hope for a good match. Then we see how it goes.”
2:43

Jannik Sinner on facing Alexander Zverev in the Wimbledon final “Less exchanges. Less rhythm. This we know because of the surface being that fast. Being also quite warm it’s gonna be good for a big server. He’s definitely a big server. In any case, I try to do my best on Sunday. It feels good again being back in a grand slam final. It’s good for me. I’ll just try to do my best. Whatever happened in the past between me and him, it happened. In between, he won a slam in Paris, which gave him a lot of confidence. We can see the level he’s producing here. We all hope for a good match. Then we see how it goes.”

The Tennis Letter

287,994 Aufrufe • vor 1 Monat

BBC Radio 5 Live called me earlier to ask if I wanted to go on Nicky Campbell’s show to discuss what happened in Golders Green yesterday. I went on (after the producer confirmed that I wouldn’t be using any racist language, which is now standard procedure) and tried to get my point across about people being too quick to slam Zack Polanski instead of concentrating on the horrendous attack and how the last part of his quote kept being cut out on purpose as an attempt to ‘smear.’ I also mentioned the work of the Jewish community on Canvey Island to try and ‘build bridges,’ so to speak and how it’s an example that people need to follow. As you can probably gather, I’m not really that comfortable away from social media posting, but I gave it a go anyway …
5:06

BBC Radio 5 Live called me earlier to ask if I wanted to go on Nicky Campbell’s show to discuss what happened in Golders Green yesterday. I went on (after the producer confirmed that I wouldn’t be using any racist language, which is now standard procedure) and tried to get my point across about people being too quick to slam Zack Polanski instead of concentrating on the horrendous attack and how the last part of his quote kept being cut out on purpose as an attempt to ‘smear.’ I also mentioned the work of the Jewish community on Canvey Island to try and ‘build bridges,’ so to speak and how it’s an example that people need to follow. As you can probably gather, I’m not really that comfortable away from social media posting, but I gave it a go anyway …

Simon Harris

104,389 Aufrufe • vor 3 Monaten

🇨🇳 It has started. A new home service in China pairs human cleaners with autonomous AI robots to tackle household chores. Residents in Shenzhen can now book a service where a human professional and an autonomous robot arrive together to clean their home. Real houses present a chaotic mess of dropped toys and random furniture that confuse traditional machines. X Square Robot and a major service platform named 58[.]com decided to tackle this chaos by launching China's first robot cleaner service in March-26. Customers use an application to hire a cleaning crew that consists of 1 human worker and 1 robot. The human takes care of the tricky chores that require complex judgment. The robot handles the repetitive physical work like picking up trash and wiping down flat surfaces. This machine runs on a system called WALL-A, which acts as a single continuous AI brain rather than a list of pre-written rules. They built this AI foundation model to perceive its surroundings and make its own decisions without human guidance. It processes visual data and plans multi-step actions. And deploying these robots into actual homes now provides the massive amounts of extremely important training data to improve it continuously. Alibaba and ByteDance backed this project. IMO, if the foundational model behind it figures out how to navigate a messy living room without getting stuck, it can learn to operate in almost any other physical environment.
1:19

🇨🇳 It has started. A new home service in China pairs human cleaners with autonomous AI robots to tackle household chores. Residents in Shenzhen can now book a service where a human professional and an autonomous robot arrive together to clean their home. Real houses present a chaotic mess of dropped toys and random furniture that confuse traditional machines. X Square Robot and a major service platform named 58[.]com decided to tackle this chaos by launching China's first robot cleaner service in March-26. Customers use an application to hire a cleaning crew that consists of 1 human worker and 1 robot. The human takes care of the tricky chores that require complex judgment. The robot handles the repetitive physical work like picking up trash and wiping down flat surfaces. This machine runs on a system called WALL-A, which acts as a single continuous AI brain rather than a list of pre-written rules. They built this AI foundation model to perceive its surroundings and make its own decisions without human guidance. It processes visual data and plans multi-step actions. And deploying these robots into actual homes now provides the massive amounts of extremely important training data to improve it continuously. Alibaba and ByteDance backed this project. IMO, if the foundational model behind it figures out how to navigate a messy living room without getting stuck, it can learn to operate in almost any other physical environment.

Rohan Paul

52,996 Aufrufe • vor 5 Monaten

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.
0:27

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

Virtuals Protocol

216,144 Aufrufe • vor 1 Jahr

SNEAKO speaks on Clavicular potentially facing up to 8 years in prison if convicted on all counts and given the maximum penalties, while people allegedly tied to Epstein Island seem to avoid that same level of accountability. 😳 “Clavicular is getting prosecuted, Clavicular is facing charges and stuff and he’s the one put in jail… we’re not gonna see any of the same prosecution for them or even close to. It’s so absurd. How do you expect me to watch the Super Bowl and be a good goy and slam a PrizePicks bet on which team backed by people that are implicated on the island?”
1:08

SNEAKO speaks on Clavicular potentially facing up to 8 years in prison if convicted on all counts and given the maximum penalties, while people allegedly tied to Epstein Island seem to avoid that same level of accountability. 😳 “Clavicular is getting prosecuted, Clavicular is facing charges and stuff and he’s the one put in jail… we’re not gonna see any of the same prosecution for them or even close to. It’s so absurd. How do you expect me to watch the Super Bowl and be a good goy and slam a PrizePicks bet on which team backed by people that are implicated on the island?”

SNEAKO UPDATES

149,463 Aufrufe • vor 6 Monaten

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.
1:17

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Rohan Paul

11,253 Aufrufe • vor 1 Monat

Jensen Huang just validated Elon Musk’s entire ecosystem in a single breath. Not one product. All of it. Huang: “The work he’s doing in Grok, self-driving cars, and Optimus. These are all world-class. Every single one of them is revolutionary. Every single one of them is going to be a gigantic opportunity.” To the public, a chatbot, a car, and a robot look like three separate bets. They are one project. The total automation of human cognition and physical labor. A digital brain. A spatial nervous system. A physical body. Musk is building all three simultaneously. Huang is supplying the compute to fuse them. Huang: “We do a lot of business with Tesla and xAI. Elon is an extraordinary engineer, and I love working with him. We’ve built some amazing computers together, and we’re going to build many more.” This is not a vendor relationship. It is the most consequential technological alliance in history and most people think it is a business partnership. Then Huang said what should end every debate about Optimus. Huang: “This is the first robot that really has a chance to achieve the high volume and technology scale necessary to advance technology.” Huang: “Right around the corner. Likely to be the next multi-trillion dollar industry.” The humanoid robot race will not be won in a research lab. It will be won on the manufacturing floor. Every other robotics company on earth can build a robot. Tesla can flood the planet with them. Because Tesla already knows how to stamp metal, build batteries, and deploy autonomous inference at global scale. The rest of the industry has prototypes. Tesla has the most sophisticated manufacturing operation on earth. When the world’s leading chipmaker calls your robot the next multi-trillion dollar industry, the debate is over. One supplies the chips. One builds everything the chips make possible. When that infrastructure scales across Grok, FSD, and Optimus simultaneously, the question stops being whether this changes everything. It becomes how fast.
1:08

Jensen Huang just validated Elon Musk’s entire ecosystem in a single breath. Not one product. All of it. Huang: “The work he’s doing in Grok, self-driving cars, and Optimus. These are all world-class. Every single one of them is revolutionary. Every single one of them is going to be a gigantic opportunity.” To the public, a chatbot, a car, and a robot look like three separate bets. They are one project. The total automation of human cognition and physical labor. A digital brain. A spatial nervous system. A physical body. Musk is building all three simultaneously. Huang is supplying the compute to fuse them. Huang: “We do a lot of business with Tesla and xAI. Elon is an extraordinary engineer, and I love working with him. We’ve built some amazing computers together, and we’re going to build many more.” This is not a vendor relationship. It is the most consequential technological alliance in history and most people think it is a business partnership. Then Huang said what should end every debate about Optimus. Huang: “This is the first robot that really has a chance to achieve the high volume and technology scale necessary to advance technology.” Huang: “Right around the corner. Likely to be the next multi-trillion dollar industry.” The humanoid robot race will not be won in a research lab. It will be won on the manufacturing floor. Every other robotics company on earth can build a robot. Tesla can flood the planet with them. Because Tesla already knows how to stamp metal, build batteries, and deploy autonomous inference at global scale. The rest of the industry has prototypes. Tesla has the most sophisticated manufacturing operation on earth. When the world’s leading chipmaker calls your robot the next multi-trillion dollar industry, the debate is over. One supplies the chips. One builds everything the chips make possible. When that infrastructure scales across Grok, FSD, and Optimus simultaneously, the question stops being whether this changes everything. It becomes how fast.

Dustin

66,374 Aufrufe • vor 5 Monaten

We have SLAM on the Robocap! 🎉 Visualized with Rerun Using NVIDIA AI Developer cuVSLAM for GPU-accelerated multicamera tracking. I basically wrote zero code myself and fully used Claude Code for this. It worked because I had so many existing examples to point to that it just wrote everything the way I would have. A few technical wins: 1. Used rattler-build from prefix.dev to package the compiled cuVSLAM CUDA binaries, which made it SUPER easy to use across repos. This also means it works on the DGX Spark (ARM64) out of the box. 2. Zero-setup experience: git clone && pixi run track-robocap auto-downloads a 100MB dataset from HuggingFace and tracks frames. 3. Real-time 3D visualization with trajectories, landmarks, pose graphs, and video playback in Rerun. Still visual-only (not visual-inertial yet), and loop closure needs some debugging. Next steps are getting this into a Gradio interface, then into daggr, and extending it to work with other datasets from exoego-forge. The last piece I'm excited about: Rerun's RRD files now support layers for incremental data. Planning to build pipelines that go from raw sensor data → slam -> human pose → depth estimation → etc. Repo here:
0:21

We have SLAM on the Robocap! 🎉 Visualized with Rerun Using NVIDIA AI Developer cuVSLAM for GPU-accelerated multicamera tracking. I basically wrote zero code myself and fully used Claude Code for this. It worked because I had so many existing examples to point to that it just wrote everything the way I would have. A few technical wins: 1. Used rattler-build from prefix.dev to package the compiled cuVSLAM CUDA binaries, which made it SUPER easy to use across repos. This also means it works on the DGX Spark (ARM64) out of the box. 2. Zero-setup experience: git clone && pixi run track-robocap auto-downloads a 100MB dataset from HuggingFace and tracks frames. 3. Real-time 3D visualization with trajectories, landmarks, pose graphs, and video playback in Rerun. Still visual-only (not visual-inertial yet), and loop closure needs some debugging. Next steps are getting this into a Gradio interface, then into daggr, and extending it to work with other datasets from exoego-forge. The last piece I'm excited about: Rerun's RRD files now support layers for incremental data. Planning to build pipelines that go from raw sensor data → slam -> human pose → depth estimation → etc. Repo here:

Pablo Vela

50,825 Aufrufe • vor 5 Monaten

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 Aufrufe • vor 4 Monaten