Загрузка видео...

Не удалось загрузить видео

На главную

Excited to share our latest work: “Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robots” 🪲🦎 We show that Hebbian learning outperforms LSTM-based adaptation for real-world transfer. It even works without domain randomization! It can handle: ✅ Uneven terrain ✅ Morphological damage ✅ Sim-to-real gaps

14,390 просмотров • 1 год назад •via X (Twitter)

Комментарии: 13

Фото профиля Sebastian Risi
Sebastian Risi1 год назад

Current ML methods are powerful but brittle. Most fail when conditions change — sim-to-real gaps, uneven terrain, or even minor morphological damage can cause catastrophic failures. Domain randomization is a way to increase robustness but can be expensive to train.

Фото профиля Sebastian Risi
Sebastian Risi1 год назад

Inspired by biological brains, we use Hebbian plasticity where synaptic weights adapt in real-time based on neural activity. This allows dynamic adaptation, even in unseen environments and seems to not crucially depend on domain randomization. We tested this approach on two challenging, high-DOF robots: 🪲 An 18-DOF dung beetle-inspired robot 🦎 A 16-DOF gecko-like robot Both showed zero-shot sim-to-real transfer, adapting instantly without retraining.

Фото профиля Sebastian Risi
Sebastian Risi1 год назад

What type of solution did the Hebbian network find? PCA of weight dynamics revealed limit cycle attractors, suggesting rhythmic motor control emerges from plasticity + sensory feedback. A limit cycle attractor can be observed in the weights of the ANN with Hebbian updates (left). In contrast, non-optimized Hebbian rules cannot achieve a limit cycle attractor and the weights were continually updated toward a single direction in the weight space (right).

Фото профиля Sebastian Risi
Sebastian Risi1 год назад

We believe this work highlights the power of bio-inspired learning mechanisms in robotics. Our Hebbian networks are simple, lightweight, and robust and they generalize without requiring architectural complexity or training tricks. Paper: Project page with videos: A great collaboration with @BinggwongLeung , @worasuchad, @JoachimWinther. and @PManoonpong.

Фото профиля kyuxu
kyuxu1 год назад

really cool. has hebbian learning been explored in combination with topology evolving networks, e.g. NEAT, or is the parameter search space on a fixed architecture already difficult to optimize?

Фото профиля Sebastian Risi
Sebastian Risi1 год назад

It has actually! Here is a review article of that field: Although the article is now a few years old so maybe time for an update.

Фото профиля Abe
Abe1 год назад

Zero shot is terrifying

Фото профиля Anesu Tembo
Anesu Tembo1 год назад

Great work!!!

Фото профиля kira
kira1 год назад

@drmichaellevin Grow up from your toys If you ready 🙊

Фото профиля Qiqi Duan (段琦琦)
Qiqi Duan (段琦琦)1 год назад

Interesting!

Фото профиля matty
matty1 год назад

Is there a reason why these bio inspired approaches don’t get applied to language? It’s always robots automata etc ..

Фото профиля تطوير الالعاب - Ludology
تطوير الالعاب - Ludology1 год назад

Compared to SNN ?

Фото профиля VistaShares ETFs
VistaShares ETFs1 год назад

From semiconductors to data centers, AIS targets the critical components behind AI's exponential growth. Capture potential returns from this transformative technology sector.

Похожие видео

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 просмотров • 1 год назад