正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough... for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:show more

Ilir Aliu

54,720 subscribers

53,992 次观看 • 6 个月前 •via X (Twitter)

新闻政治科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,797 次观看 • 2 个月前

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

70,090 次观看 • 10 个月前

A simple MPC controller for path tracking implemented in python. Few resources actually teach it without assuming a PhD first. This open-source project does exactly that: An iterative MPC tracking controller (built with CVXPY and MuJoCo) that shows how to make a robot (or autonomous vehicle) follow a path and avoid obstacles in real time. → Autonomous logistics, robot navigation, vehicle control; there's a path-following algorithm behind all of it. → Most implementations are black boxes. This one shows you what's actually happening. → It uses convex optimization (no non-linear solver), and still handles real vehicle physics through iterative linearization. What makes it practical is that you can run it with the mushr car model in MuJoCo, or headless without any physics simulation if you just want to understand the logic. If you're making the jump from classical control to real-time optimisation, this is a very good starting point. 📌 Repository: ——- Weekly robotics and AI insights. Subscribe free:

A simple MPC controller for path tracking implemented in python. Few resources actually teach it without assuming a PhD first. This open-source project does exactly that: An iterative MPC tracking controller (built with CVXPY and MuJoCo) that shows how to make a robot (or autonomous vehicle) follow a path and avoid obstacles in real time. → Autonomous logistics, robot navigation, vehicle control; there's a path-following algorithm behind all of it. → Most implementations are black boxes. This one shows you what's actually happening. → It uses convex optimization (no non-linear solver), and still handles real vehicle physics through iterative linearization. What makes it practical is that you can run it with the mushr car model in MuJoCo, or headless without any physics simulation if you just want to understand the logic. If you're making the jump from classical control to real-time optimisation, this is a very good starting point. 📌 Repository: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

16,064 次观看 • 21 天前

Most robots still need markers, checkerboards, or long calibration rituals just to know where their arms are. Now it works from raw images in seconds. roboreg is a markerless multi arm localization toolkit that plugs into ROS 2 and RViz. No special hardware. No custom setup. You toggle between robot descriptions and the system figures out the rest. The idea is simple: ✅ Hand eye calibration from plain RGB or RGB D images ✅ Only three robot poses needed for millimeter accuracy ✅ Works with any ROS 2 compatible robot and camera ✅ Fully open source under Apache 2.0 It is powered by Hydra, a new marker free ICP variant that converges far more reliably than classical baselines and runs in under a second. If you want to try it: roboreg: ROS 2 roboreg: Hydra paper: pip install roboreg More details and discussion on Open Robotics Discourse:

Most robots still need markers, checkerboards, or long calibration rituals just to know where their arms are. Now it works from raw images in seconds. roboreg is a markerless multi arm localization toolkit that plugs into ROS 2 and RViz. No special hardware. No custom setup. You toggle between robot descriptions and the system figures out the rest. The idea is simple: ✅ Hand eye calibration from plain RGB or RGB D images ✅ Only three robot poses needed for millimeter accuracy ✅ Works with any ROS 2 compatible robot and camera ✅ Fully open source under Apache 2.0 It is powered by Hydra, a new marker free ICP variant that converges far more reliably than classical baselines and runs in under a second. If you want to try it: roboreg: ROS 2 roboreg: Hydra paper: pip install roboreg More details and discussion on Open Robotics Discourse:

Ilir Aliu

18,442 次观看 • 10 个月前

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 次观看 • 9 个月前

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

This work makes a humanoid robot do simple parkour moves by looking with a depth camera and choosing the right move on the fly. The big deal is that it turns lots of small human moves into long, real-time robot behavior, without hand-coding every transition or retraining for each new course. A humanoid robot is usually good at steady walking, but it often fails when it has to do fast moves like jumping up, vaulting, or rolling, and then keep going to the next obstacle. The hard part is that you cannot easily collect training data for every possible obstacle shape, distance, and mistake, so robots end up learning a few moves that only work in a narrow setup. This work starts from short clips of real human parkour moves, like stepping over, vaulting, climbing, and rolling. It uses motion matching, which is basically a smart “pick the next clip that fits best right now” search, to stitch those short clips into a long, smooth plan that looks like a human doing a whole course. Then it trains a controller with reinforcement learning (RL), which means the robot learns by trial and error to copy that plan while staying balanced and not falling. After training separate expert controllers for different moves, it compresses them into 1 controller that uses only onboard depth sensing and a simple “go this fast in this direction” command. In real tests on a Unitree G1 humanoid, it can clear multiple obstacles in a row, adapt when obstacles get moved, and climb a wall up to 1.25m.

Rohan Paul

37,121 次观看 • 7 个月前

AI in robotics gets all the attention right now, but sometimes the most interesting work is very practical. Viet built a small vision system that counts potatoes on a conveyor belt. No giant dataset. No huge model. Just a clear problem and a smart setup. He used Ultralytics’ ObjectCounter, trained a tiny YOLO11 nano model, and because there was no potato dataset, he annotated a single frame with SAM 2 and trained from that. One frame. Still works across the whole video. It is a good reminder that useful AI in industry often looks like this. Focused. Lightweight. Solves a real task. If you work in manufacturing or robotics, these small systems are usually the fastest wins. They save time, reduce errors, and do not need massive infrastructure. Nice work, Viet. His projects: —- Weekly robotics and AI insights. Subscribe free:

AI in robotics gets all the attention right now, but sometimes the most interesting work is very practical. Viet built a small vision system that counts potatoes on a conveyor belt. No giant dataset. No huge model. Just a clear problem and a smart setup. He used Ultralytics’ ObjectCounter, trained a tiny YOLO11 nano model, and because there was no potato dataset, he annotated a single frame with SAM 2 and trained from that. One frame. Still works across the whole video. It is a good reminder that useful AI in industry often looks like this. Focused. Lightweight. Solves a real task. If you work in manufacturing or robotics, these small systems are usually the fastest wins. They save time, reduce errors, and do not need massive infrastructure. Nice work, Viet. His projects: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

1,677,279 次观看 • 10 个月前

🔥 Phoenix is officially live on Solaris AI Flow. You can now trade Phoenix perps inside a Solaris AI workflow. No code. Drop a node onto the canvas, pick an operation, and wire it to anything: AI signals, price feeds, schedules, alerts. The full order-book DEX from Ellipsis Labs, now programmable. Built so you can trade with confidence: ✦ Paper mode is on by default. Every order is checked and simulated against the live order book, real depth and real slippage, but nothing is signed or broadcast. ✦ Paper behaves exactly like Live. If an order would be rejected on-chain, it is rejected in simulation too. No false fills, no surprises. ✦ Going Live is one switch, and it asks for confirmation before any real funds move. Build it. Test it. Trade it. A full perps strategy, proven on paper before a single dollar moves. 30 operations in one node: ✦ Read live markets, order book depth, candles, and funding rates ✦ Track your positions, collateral, and PnL, realized and unrealized ✦ Place limit, market, and stop-loss orders, plus conditional triggers ✦ Cancel orders, manage margin, and move collateral, all from the workflow No scripts. No backend. No terminal to babysit. Just a workflow that trades. You can try it for free. Demo + Link down below 👇

🔥 Phoenix is officially live on Solaris AI Flow. You can now trade Phoenix perps inside a Solaris AI workflow. No code. Drop a node onto the canvas, pick an operation, and wire it to anything: AI signals, price feeds, schedules, alerts. The full order-book DEX from Ellipsis Labs, now programmable. Built so you can trade with confidence: ✦ Paper mode is on by default. Every order is checked and simulated against the live order book, real depth and real slippage, but nothing is signed or broadcast. ✦ Paper behaves exactly like Live. If an order would be rejected on-chain, it is rejected in simulation too. No false fills, no surprises. ✦ Going Live is one switch, and it asks for confirmation before any real funds move. Build it. Test it. Trade it. A full perps strategy, proven on paper before a single dollar moves. 30 operations in one node: ✦ Read live markets, order book depth, candles, and funding rates ✦ Track your positions, collateral, and PnL, realized and unrealized ✦ Place limit, market, and stop-loss orders, plus conditional triggers ✦ Cancel orders, manage margin, and move collateral, all from the workflow No scripts. No backend. No terminal to babysit. Just a workflow that trades. You can try it for free. Demo + Link down below 👇

Solaris AI

10,075 次观看 • 4 个月前

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 次观看 • 2 年前

JUST IN: Reimagine Robotics has just emerged from stealth! 🥷🏻 Its approach to robot training is one of the most human-centric I've seen. The founder is Jonathan Scholz, the person who built and led Google DeepMind's Applied Robotics team in London for seven years. This is not a first-time founder taking a swing at robotics. He has spent a decade at the frontier of the field. The philosophy is powerful. He calls it "monkey-see, monkey-do." 🐒 A worker shows the robot what to do. Watches it attempt the task. Corrects it on the spot. The robot learns. No specialist programmers. No months of integration. And it's already working in the real world: → A made-to-order plastics business trained robots to tend 3D printers overnight, removing print beds, operating latches, pressing controls → A hard drive disassembly facility built a three-robot cell combining robots and people to recover critical materials → Time to prototype and test a new robot behaviour reduced from one day to 10 MINUTES That last number is the one that changes everything. When testing a new behaviour takes 10 minutes instead of a day, the entire pace of deployment transforms. Scholz's framing of the human-robot relationship is worth reading carefully: "A robot that learns on the job depends on people. The worker identifies the bottleneck, shows the robot how to help, and corrects it until it is useful." It's August, and we keep getting robotics bangers week in week. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

JUST IN: Reimagine Robotics has just emerged from stealth! 🥷🏻 Its approach to robot training is one of the most human-centric I've seen. The founder is Jonathan Scholz, the person who built and led Google DeepMind's Applied Robotics team in London for seven years. This is not a first-time founder taking a swing at robotics. He has spent a decade at the frontier of the field. The philosophy is powerful. He calls it "monkey-see, monkey-do." 🐒 A worker shows the robot what to do. Watches it attempt the task. Corrects it on the spot. The robot learns. No specialist programmers. No months of integration. And it's already working in the real world: → A made-to-order plastics business trained robots to tend 3D printers overnight, removing print beds, operating latches, pressing controls → A hard drive disassembly facility built a three-robot cell combining robots and people to recover critical materials → Time to prototype and test a new robot behaviour reduced from one day to 10 MINUTES That last number is the one that changes everything. When testing a new behaviour takes 10 minutes instead of a day, the entire pace of deployment transforms. Scholz's framing of the human-robot relationship is worth reading carefully: "A robot that learns on the job depends on people. The worker identifies the bottleneck, shows the robot how to help, and corrects it until it is useful." It's August, and we keep getting robotics bangers week in week. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

23,389 次观看 • 1 个月前

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,950 次观看 • 2 个月前

Hey #NeuraxonMini is literally out! , we manage to "transplant" a Neuraxon 2 bioinspired #AI brain to a physical robot the #SpheroMini moving from our last Scientific Paper (link bellow) by David Vivancos - e/acc & Jose Sánchez for Qubic #OpenScience hybridized with #Aigarth to the real World. First you need a Sphero Education Mini robot about 50$ Then you can try the first cool demos at Hugging Face: 1.- Neuraxon2MiniControl to drive the sphero robot 2.- Neuraxon2MiniWrite to write letters or words with physical moves of the sphero robot using Neuraxon Video Tutorials on youtube later today. Why this matters? Remember we are not building "dead" LLMs we are building #AliveAIs and for that we need to explore how it behaves in reality, from how it learns to how it fails, and what better way that in the emerging field of #robotics , time will tell if your next #HumanoidRobot have a #Neuraxon brain... Read the Paper: Explore the Neuraxon code here: Are you ready for #TrueAI ?

Hey #NeuraxonMini is literally out! , we manage to "transplant" a Neuraxon 2 bioinspired #AI brain to a physical robot the #SpheroMini moving from our last Scientific Paper (link bellow) by David Vivancos - e/acc & Jose Sánchez for Qubic #OpenScience hybridized with #Aigarth to the real World. First you need a Sphero Education Mini robot about 50$ Then you can try the first cool demos at Hugging Face: 1.- Neuraxon2MiniControl to drive the sphero robot 2.- Neuraxon2MiniWrite to write letters or words with physical moves of the sphero robot using Neuraxon Video Tutorials on youtube later today. Why this matters? Remember we are not building "dead" LLMs we are building #AliveAIs and for that we need to explore how it behaves in reality, from how it learns to how it fails, and what better way that in the emerging field of #robotics , time will tell if your next #HumanoidRobot have a #Neuraxon brain... Read the Paper: Explore the Neuraxon code here: Are you ready for #TrueAI ?

David Vivancos - e/acc

29,293 次观看 • 7 个月前

Furniture assembly is the task everyone name-drops and nobody actually attempts at real scale. Every demo I have seen is a scaled down IKEA leg or a single arm on a toy chair. This paper does it properly, real scale, bimanual, up to 7 subtasks and 1,550 control steps per episode, and it is validated on a real Kinova Gen3, not just in sim. That real-robot number is the one that matters: only a 16 percent drop on the hardest task going from simulation to hardware. That is a small enough gap to take seriously, and it did not happen by accident. They built a VR teleoperation rig specifically for coordinated dual-arm collection, because generic single-arm teleop setups do not capture the coordination real assembly needs, and the model predicts a continuous progress signal alongside the action chunk rather than a discrete subtask label, letting it auto-transition and catch drift before it compounds into total failure. The simulation ablation is what got them there, 48 to 80 percent over baselines, with another 21 points from their perception and control design study alone, but that is groundwork, not the headline. Watch the video, there is a clip of the robot misgrasping the seat panel, reopening the gripper, and regrasping on its own. That is not scripted recovery behaviour, it emerged from training, and it emerged on hardware. Excellent work from the team from Mitsubishi Electric Research Laboratories, with Oxford and UNC Chapel Hill Clinical Laboratory Science. Video and project page in comments. #Robotics #Manipulation #VLA

Furniture assembly is the task everyone name-drops and nobody actually attempts at real scale. Every demo I have seen is a scaled down IKEA leg or a single arm on a toy chair. This paper does it properly, real scale, bimanual, up to 7 subtasks and 1,550 control steps per episode, and it is validated on a real Kinova Gen3, not just in sim. That real-robot number is the one that matters: only a 16 percent drop on the hardest task going from simulation to hardware. That is a small enough gap to take seriously, and it did not happen by accident. They built a VR teleoperation rig specifically for coordinated dual-arm collection, because generic single-arm teleop setups do not capture the coordination real assembly needs, and the model predicts a continuous progress signal alongside the action chunk rather than a discrete subtask label, letting it auto-transition and catch drift before it compounds into total failure. The simulation ablation is what got them there, 48 to 80 percent over baselines, with another 21 points from their perception and control design study alone, but that is groundwork, not the headline. Watch the video, there is a clip of the robot misgrasping the seat panel, reopening the gripper, and regrasping on its own. That is not scripted recovery behaviour, it emerged from training, and it emerged on hardware. Excellent work from the team from Mitsubishi Electric Research Laboratories, with Oxford and UNC Chapel Hill Clinical Laboratory Science. Video and project page in comments. #Robotics #Manipulation #VLA

Stephen James

14,952 次观看 • 2 个月前

two weeks in Shenzhen and it’s an been eye-opening experience im here for a month with MIT SCALE and the speed of hardware production is easily 10× faster than sf. from product concept to production, everything is streamlined within a single building. Nearly every component you could need is available immediately thru Taobao(e bay of China) or HQB and there are 24/7 makerspaces equipped to build anything from micrometer-scale PCBs to full assembly-line robots. What’s striking is that Shenzhen already has much of the hardware and robotics that startups in sf are still trying to build except these systems have been deployed and operating for years. The manufacturing capability, supply chain depth, and technical execution here are world-class. That said, one gap keeps surfacing: brand design, storytelling, and cohesive user experience. Software polish, UX consistency, and attention to detail often feel secondary. Take this with a grain of salt, but it increasingly feels like the company that pairs Shenzhen-level hardware velocity with strong design sensibility and UX-first thinking will dominate the market.

Miyu Horiuchi

149,719 次观看 • 8 个月前

Open-Source Multi-Sensor Data Platform for Neural 3D Reconstruction and Physical AI [📍github] It handles cameras, LiDAR, radar, poses, calibrations & labels in one clean format. No more messy custom parsers. • Super efficient (non-redundant storage) • New .itar single-file format with lightning-fast random access • Streams straight from S3/GCS/Azure – perfect for huge datasets • Built-in converters for Waymo, ScanNet++ & more • Already powers NVIDIA NuRec, 3DGRUT & gsplat Saw this at Janick Martinez Esturo, thanks for sharing! Easy to try: • pip install nvidia-ncore • GitHub: • Docs & project page: NCore slashes data wrangling time, cuts storage waste, and makes large-scale neural 3D training faster and simpler than ever. A real standard for physical AI. ——- Weekly robotics and AI insights. Subscribe free:

Open-Source Multi-Sensor Data Platform for Neural 3D Reconstruction and Physical AI [📍github] It handles cameras, LiDAR, radar, poses, calibrations & labels in one clean format. No more messy custom parsers. • Super efficient (non-redundant storage) • New .itar single-file format with lightning-fast random access • Streams straight from S3/GCS/Azure – perfect for huge datasets • Built-in converters for Waymo, ScanNet++ & more • Already powers NVIDIA NuRec, 3DGRUT & gsplat Saw this at Janick Martinez Esturo, thanks for sharing! Easy to try: • pip install nvidia-ncore • GitHub: • Docs & project page: NCore slashes data wrangling time, cuts storage waste, and makes large-scale neural 3D training faster and simpler than ever. A real standard for physical AI. ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

33,265 次观看 • 5 个月前

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.

Michael Black

22,353 次观看 • 9 个月前

Most humanoid projects talk about real work. Very few last an hour on a real line. This week I saw a case that matters for anyone building robots, perception, or physical AI. Kinisi deployed its first mobile manipulation system into a live recycling facility. Not a demo. Not a staged test. A real production line with real output pressure. Why this matters if you want robotics to deliver real value on your floor: • Handles mixed glass with random poses and no fixed fixtures. • Runs real grasp selection under noise, vibration and production variability. • Maintains throughput while avoiding breakage on a delicate material. • Shows mobile manipulation doing actual shift work instead of controlled lab runs. Kinisi published a video that shows what the robot sees and how sensor data turns into action. This is the part most teams struggle to explain to customers, so the educational angle is useful for anyone working on adoption. On top of this, the team signed a pilot with a global automotive manufacturer to explore humanoid use cases in production. The direction is clear. Wheeled mobility (not legs!) plus strong perception seems to be shaping a large part of industrial humanoids right now. I know Brennand from earlier conversations and from our podcast session, and I am always glad to see European teams push the category forward. Wishing the Kinisi team continued success. —- Weekly robotics and AI insights. Subscribe free:

Most humanoid projects talk about real work. Very few last an hour on a real line. This week I saw a case that matters for anyone building robots, perception, or physical AI. Kinisi deployed its first mobile manipulation system into a live recycling facility. Not a demo. Not a staged test. A real production line with real output pressure. Why this matters if you want robotics to deliver real value on your floor: • Handles mixed glass with random poses and no fixed fixtures. • Runs real grasp selection under noise, vibration and production variability. • Maintains throughput while avoiding breakage on a delicate material. • Shows mobile manipulation doing actual shift work instead of controlled lab runs. Kinisi published a video that shows what the robot sees and how sensor data turns into action. This is the part most teams struggle to explain to customers, so the educational angle is useful for anyone working on adoption. On top of this, the team signed a pilot with a global automotive manufacturer to explore humanoid use cases in production. The direction is clear. Wheeled mobility (not legs!) plus strong perception seems to be shaping a large part of industrial humanoids right now. I know Brennand from earlier conversations and from our podcast session, and I am always glad to see European teams push the category forward. Wishing the Kinisi team continued success. —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

24,815 次观看 • 10 个月前