Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Multi-robot learning is getting a serious boost! 📚 Researchers have extended Isaac Lab to train heterogeneous multi-agent robotic policies at scale. The new framework supports high-resolution physics, GPU-accelerated simulation, and both homogeneous and heterogeneous agents working together on coordination tasks. They benchmarked different approaches (MAPPO: Multi-Agent Proximal Policy Optimization...

38,997 Aufrufe • vor 9 Monaten •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

AK

19,684 Aufrufe • vor 2 Jahren

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.

Robora

23,489 Aufrufe • vor 11 Monaten

🚨 BREAKING: NVIDIA just announced the Isaac GR00T Reference Humanoid Robot. The first fully open humanoid robot reference design built on Jetson Thor, and it's going straight to the world's top research institutions. This is Jensen Huang's bet on open physical AI infrastructure. The hardware stack is serious: → Unitree H2 Plus chassis, 6 feet tall, 150 pounds, 31 degrees of freedom → Sharpa Wave tactile five-finger hands, 22 degrees of freedom, bringing total to 75 across the full body → NVIDIA Jetson AGX Thor onboard compute, 2,070 FP4 teraflops of AI performance, 128GB unified memory → Multi-view sensing, stereo head camera, wrist cameras, IMU Alongside this announcement, Unitree also introduced the H2 Plus as a standalone product, a frontier humanoid combining Unitree's own body, Sharpa's five-finger hands and NVIDIA Robotics Jetson Thor compute into one fully integrated research platform. The full Isaac GR00T software stack ships with it, teleoperation for data capture, open foundation models, Isaac Sim for training, Isaac Lab for evaluation, and accelerated ROS middleware for deployment. The complete loop from data to real-world robot in one unified platform. ETH Zürich, Stanford Robotics Center, UC San Diego and Ai2 are already on board as launch research partners. NVIDIA Robotics did to AI what it's now doing to robotics, build the platform, open the ecosystem, let the world build on top of it. Whoever owns the infrastructure layer wins. NVIDIA knows this better than anyone. 👀 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

16,062 Aufrufe • vor 3 Monaten

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference

Sasha Malysheva

12,064 Aufrufe • vor 24 Tagen

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,950 Aufrufe • vor 1 Monat

As a newly appointed 𝗔𝘀𝘀𝗶𝘀𝘁𝗮𝗻𝘁 𝗣𝗿𝗼𝗳𝗲𝘀𝘀𝗼𝗿 at Imperial College London, I'm thrilled to announce the 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) at 𝗜𝗺𝗽𝗲𝗿𝗶𝗮𝗹 𝗖𝗼𝗹𝗹𝗲𝗴𝗲 𝗟𝗼𝗻𝗱𝗼𝗻. 𝗦𝗮𝗳𝗲 𝗪𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗟𝗮𝗯 (𝗦𝗪𝗜𝗥𝗟) ( is a new research lab focused on the intersection of safety and intelligence in next-generation robotics. We're hiring exceptional PhD students who are passionate about pushing the boundaries of robot learning. 𝗪𝗵𝗮𝘁 𝗺𝗮𝗸𝗲𝘀 𝗦𝗪𝗜𝗥𝗟 𝘂𝗻𝗶𝗾𝘂𝗲? We operate at the exciting convergence of: • Online & offline reinforcement learning • Imitation learning & human demonstrations • Sample-efficient learning methods • Whole-body and soft robotics systems We're 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝗽𝗿𝗼𝘀𝗽𝗲𝗰𝘁𝗶𝘃𝗲 𝗣𝗵𝗗 𝘀𝘁𝘂𝗱𝗲𝗻𝘁𝘀 interested in: • Developing safe exploration algorithms for robotic systems • Creating sample-efficient learning methods that minimize real-world trials • Building foundation models for robotics with safety guarantees • Advancing soft robotics and compliant human-robot interaction • Bridging theory and practice in embodied AI Why now? As robots become more capable and work closer with humans, we need systems that are both intelligent enough to handle complex tasks 𝗔𝗡𝗗 safe enough for real-world deployment. Traditional approaches treat safety and intelligence as competing priorities, we believe they're synergistic. If you're a motivated researcher who wants to develop the theoretical foundations and practical algorithms for tomorrow's safe, intelligent robots, I'd love to hear from you. Want to join? Apply via

Stephen James

16,673 Aufrufe • vor 11 Monaten

Something big is happening in robotics - and it’s hiding in plain sight. This post is not about dancing robots but in the data that powers them. Open robotics datasets have exploded this year, turning the field into a more scalable and collaborative ecosystem. In just two years, Hugging Face datasets grew from 11k to over 600k - and robotics is by far the fastest-growing segment. We went from 1k robotics datasets in 2024 to 27k in 2025! For comparison, text generation, the second-largest category, has only around 5k datasets in 2025. That gap is massive. Open datasets are important because robotics lives and dies by real-world robot data - video, actions, sensors, failures. By making this data easy to upload, reuse, and benchmark, researchers, startups, and large players are now releasing real-robot datasets that would have stayed locked inside labs just a few years ago. Major contributors include NVIDIA, LeRobot initiative, and a rapidly growing maker community. This surge is also enabled by cheaper video storage, better tooling, and an open-source AI culture now spilling into the physical world. And it really matters: open robotics data dramatically lowers entry barriers, accelerates learning-by-doing, and speeds up progress toward generalist and humanoid robots. Robotics won’t scale through hardware alone - but to a large extent through shared data. Viz below from AI World - link to the story and more viz/filters in comment.

Pierre-Alexandre Balland

186,094 Aufrufe • vor 8 Monaten

LangGraph. CrewAI. Agno. Which one to pick? The good news is that this will not matter soon! Finally, we have a full picture of how the industry is solving this with just three open protocols that work across ALL frameworks. It's not about picking the best framework. Instead, it's about understanding how protocols create interoperability. The Agent Protocol Landscape shows how three complementary protocols are creating a universal language for Agents: > AG-UI (Agent-User Interaction): - The bi-directional connection between agentic backends and frontends. - This is how agents become truly interactive inside your apps, not just as chatbots, but collaborative co-workers. > MCP (Model Context Protocol): - The standard for how agents connect to tools, data, and workflows. > A2A (Agent-to-Agent): - The protocol for multi-agent coordination. - How agents delegate tasks and share intent across systems. These aren't competing standards. They're layers of the same stack and have handshakes with each other. So instead of building point-to-point integrations, you build to protocols. Moreover, you can integrate LangGraph, CrewAI, or Agno into the same frontend, without rewriting your UI logic. These protocols let everything work together. For instance: - Your LangGraph agent pulls data via MCP. - It delegates analysis to a CrewAI agent via A2A. - Results stream to your React app via AG-UI. - Users see real-time collaboration in your interface. This way, you can focus on building agent capabilities instead of integration mechanics. The protocols handle interoperability automatically. CopilotKit unifies this entire stack into one framework so you can build "Cursor for X" style apps without implementing each protocol from scratch. It gives you all three protocols, generative UI support, and production-ready infrastructure in one framework. I have shared this playbook in the replies! It breaks down handshakes, misconceptions, and real examples and shows exactly how to start building.

Avi Chawla

30,932 Aufrufe • vor 9 Monaten

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷

Axis Robotics

27,858 Aufrufe • vor 8 Monaten

Imagine you go to a store and you want to buy candy. The shopkeeper knows you're a real kid because they can see you standing right there. Now imagine you send a robot to buy candy for you. The shopkeeper looks at the robot and thinks: wait, who sent this? Is this robot allowed to buy candy? What if someone else's robot pretends to be yours and steals your candy money? That's basically what's happening with AI right now. Companies like Visa let people buy things all over the world. But now, smart computer robots (AI agents) want to buy things too. Shop around, compare prices, even pay for stuff. Visa looked at this and said: nope, not yet. Because they have no way to check if the robot is real, who it belongs to, or if it's allowed to spend that money. The problem is that all the rules we have for checking identity - showing your ID, scanning your face, typing your password - only work for humans. Robots can't do any of that. Worse, bad robots can actually copy and fake human identities really well. So Evin McMullen evin, Billions Network co-founder and CEO, says we need a new kind of ID system. One where you can prove something is true without showing all your private stuff. Like proving you're tall enough for a ride without telling anyone your exact height. That's called zero-knowledge proof. And for the robots specifically, we need something called KYA - Know Your Agent. It's like giving every robot its own ID card that says: this is who I am, this is what I'm allowed to do, and this is the human responsible for me. Until we build that, the robot economy can't really get going. Here is Evin’s Thought Leader article at Silicon Valleys Journal

Billions Network

21,775 Aufrufe • vor 6 Monaten

🚨 BREAKING: Walden Robotics has just come out of stealth with $300 million in funding and a $1.1 billion valuation. Another unicorn in the robotics space. 🦄 Just 6 months after incubation. The company was spun out of Toyota's robotics research lab by co-founder Russ Tedrake, a former Toyota Research Institute executive and MIT professor who taught a course on robotic legs. The seed round was co-led by Deviation Capital and Toyota, with participation from: NVIDIA, Boeing, Samsung Ventures, CoreWeave Ventures and AE Ventures. The robot is already working. A pilot is live at a North American Toyota factory where a Walden humanoid is pulling eight-hour shifts alongside human workers, loading and unloading car parts, cleaning machinery, kitting for assembly. A shift. Every day. Walden builds its own hardware, software and AI models, designed to continuously learn and improve in real production environments. Tedrake's words on the opportunity are worth noting: "Everyone recognises the magnitude of the opportunity and the technology feels ready, but success is not assured. You have to think through the business case, the unit economics, and how to marry the best of manufacturing and logistics with disruptive AI technology." Rare honesty in a space full of hype. The race to own that market is accelerating every single week. 🤖 Great story by Bloomberg here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

75,904 Aufrufe • vor 1 Monat