正在加载视频...

视频加载失败

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

14,390 次观看 • 1 年前 •via X (Twitter)

13 条评论

Sebastian Risi 的头像
Sebastian Risi1 年前

Current ML methods are powerful but brittle. Most fail when conditions change — sim-to-real gaps, uneven terrain, or even minor morphological damage can cause catastrophic failures. Domain randomization is a way to increase robustness but can be expensive to train.

Sebastian Risi 的头像
Sebastian Risi1 年前

Inspired by biological brains, we use Hebbian plasticity where synaptic weights adapt in real-time based on neural activity. This allows dynamic adaptation, even in unseen environments and seems to not crucially depend on domain randomization. We tested this approach on two challenging, high-DOF robots: 🪲 An 18-DOF dung beetle-inspired robot 🦎 A 16-DOF gecko-like robot Both showed zero-shot sim-to-real transfer, adapting instantly without retraining.

Sebastian Risi 的头像
Sebastian Risi1 年前

What type of solution did the Hebbian network find? PCA of weight dynamics revealed limit cycle attractors, suggesting rhythmic motor control emerges from plasticity + sensory feedback. A limit cycle attractor can be observed in the weights of the ANN with Hebbian updates (left). In contrast, non-optimized Hebbian rules cannot achieve a limit cycle attractor and the weights were continually updated toward a single direction in the weight space (right).

Sebastian Risi 的头像
Sebastian Risi1 年前

We believe this work highlights the power of bio-inspired learning mechanisms in robotics. Our Hebbian networks are simple, lightweight, and robust and they generalize without requiring architectural complexity or training tricks. Paper: Project page with videos: A great collaboration with @BinggwongLeung , @worasuchad, @JoachimWinther. and @PManoonpong.

kyuxu 的头像
kyuxu1 年前

really cool. has hebbian learning been explored in combination with topology evolving networks, e.g. NEAT, or is the parameter search space on a fixed architecture already difficult to optimize?

Sebastian Risi 的头像
Sebastian Risi1 年前

It has actually! Here is a review article of that field: Although the article is now a few years old so maybe time for an update.

Abe 的头像
Abe1 年前

Zero shot is terrifying

Anesu Tembo 的头像
Anesu Tembo1 年前

Great work!!!

kira 的头像
kira1 年前

@drmichaellevin Grow up from your toys If you ready 🙊

Qiqi Duan (段琦琦) 的头像
Qiqi Duan (段琦琦)1 年前

Interesting!

matty 的头像
matty1 年前

Is there a reason why these bio inspired approaches don’t get applied to language? It’s always robots automata etc ..

تطوير الالعاب - Ludology 的头像
تطوير الالعاب - Ludology1 年前

Compared to SNN ?

VistaShares ETFs 的头像
VistaShares ETFs1 年前

From semiconductors to data centers, AIS targets the critical components behind AI's exponential growth. Capture potential returns from this transformative technology sector.

相关视频

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 次观看 • 1 年前