Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

14,390 Aufrufe • vor 1 Jahr •via X (Twitter)

13 Kommentare

Profilbild von Sebastian Risi
Sebastian Risivor 1 Jahr

Current ML methods are powerful but brittle. Most fail when conditions change — sim-to-real gaps, uneven terrain, or even minor morphological damage can cause catastrophic failures. Domain randomization is a way to increase robustness but can be expensive to train.

Profilbild von Sebastian Risi
Sebastian Risivor 1 Jahr

Inspired by biological brains, we use Hebbian plasticity where synaptic weights adapt in real-time based on neural activity. This allows dynamic adaptation, even in unseen environments and seems to not crucially depend on domain randomization. We tested this approach on two challenging, high-DOF robots: 🪲 An 18-DOF dung beetle-inspired robot 🦎 A 16-DOF gecko-like robot Both showed zero-shot sim-to-real transfer, adapting instantly without retraining.

Profilbild von Sebastian Risi
Sebastian Risivor 1 Jahr

What type of solution did the Hebbian network find? PCA of weight dynamics revealed limit cycle attractors, suggesting rhythmic motor control emerges from plasticity + sensory feedback. A limit cycle attractor can be observed in the weights of the ANN with Hebbian updates (left). In contrast, non-optimized Hebbian rules cannot achieve a limit cycle attractor and the weights were continually updated toward a single direction in the weight space (right).

Profilbild von Sebastian Risi
Sebastian Risivor 1 Jahr

We believe this work highlights the power of bio-inspired learning mechanisms in robotics. Our Hebbian networks are simple, lightweight, and robust and they generalize without requiring architectural complexity or training tricks. Paper: Project page with videos: A great collaboration with @BinggwongLeung , @worasuchad, @JoachimWinther. and @PManoonpong.

Profilbild von kyuxu
kyuxuvor 1 Jahr

really cool. has hebbian learning been explored in combination with topology evolving networks, e.g. NEAT, or is the parameter search space on a fixed architecture already difficult to optimize?

Profilbild von Sebastian Risi
Sebastian Risivor 1 Jahr

It has actually! Here is a review article of that field: Although the article is now a few years old so maybe time for an update.

Profilbild von Abe
Abevor 1 Jahr

Zero shot is terrifying

Profilbild von Anesu Tembo
Anesu Tembovor 1 Jahr

Great work!!!

Profilbild von kira
kiravor 1 Jahr

@drmichaellevin Grow up from your toys If you ready 🙊

Profilbild von Qiqi Duan (段琦琦)
Qiqi Duan (段琦琦)vor 1 Jahr

Interesting!

Profilbild von matty
mattyvor 1 Jahr

Is there a reason why these bio inspired approaches don’t get applied to language? It’s always robots automata etc ..

Profilbild von تطوير الالعاب - Ludology
تطوير الالعاب - Ludologyvor 1 Jahr

Compared to SNN ?

Profilbild von VistaShares ETFs
VistaShares ETFsvor 1 Jahr

From semiconductors to data centers, AIS targets the critical components behind AI's exponential growth. Capture potential returns from this transformative technology sector.

Ähnliche Videos

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 Aufrufe • vor 1 Jahr