Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

"Physics of AI" blog update: Learning sparse attention has a sticky plateau, representation collapse, and branching dynamics

Ziming Liu

14,454 subscribers

81,841 Aufrufe • vor 8 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

One man, one dog and 2000 sheep: a time lapse that has so many physics and fluid dynamics references [source and full video:

One man, one dog and 2000 sheep: a time lapse that has so many physics and fluid dynamics references [source and full video:

Massimo

7,440,334 Aufrufe • vor 3 Jahren

This rocket engine wasn't designed by humans, but by AI. It was created using Noyron, a physics based engineering AI built by LEAP 71. Instead of learning from past designs, it uses real physics, thermodynamics, and manufacturing rules to generate engines directly from requirements.

This rocket engine wasn't designed by humans, but by AI. It was created using Noyron, a physics based engineering AI built by LEAP 71. Instead of learning from past designs, it uses real physics, thermodynamics, and manufacturing rules to generate engines directly from requirements.

vixhaℓ

588,028 Aufrufe • vor 8 Monaten

The Branching sickness End of Year project update is here. Head over to our discord to take a look under the hood of current Branching Sickness development and apply to join the team of volunteers dedicated to bringing this game to life.

The Branching sickness End of Year project update is here. Head over to our discord to take a look under the hood of current Branching Sickness development and apply to join the team of volunteers dedicated to bringing this game to life.

Branching Sickness

85,875 Aufrufe • vor 2 Jahren

Update 0.5 for the Early Access Program of Assetto Corsa Rally is almost here and it's a big one! It brings you Rally Greece, the first iteration of Multiplayer, an updated tyre model & physics update with FFB fixes and improvements, and much more! Get the full details and changelog in the blog post below! #AssettoCorsa #ACRally #Rally

Update 0.5 for the Early Access Program of Assetto Corsa Rally is almost here and it's a big one! It brings you Rally Greece, the first iteration of Multiplayer, an updated tyre model & physics update with FFB fixes and improvements, and much more! Get the full details and changelog in the blog post below! #AssettoCorsa #ACRally #Rally

Assetto Corsa

29,885 Aufrufe • vor 2 Monaten

Day 12 of learning AI bs from scratch > a look into recurrent neural networks (RNN's) > learned about Long Short-Term Memory > read couple chapters of > drawbacks of RNN's > Simple Neural networks → CNN → RNN will try and train one MLP, CNN and a RNN from scratch next and try and summarize its working it a blog. back to doing based learning after this, much more retention like that.

Day 12 of learning AI bs from scratch > a look into recurrent neural networks (RNN's) > learned about Long Short-Term Memory > read couple chapters of > drawbacks of RNN's > Simple Neural networks → CNN → RNN will try and train one MLP, CNN and a RNN from scratch next and try and summarize its working it a blog. back to doing based learning after this, much more retention like that.

Arham Amin

62,415 Aufrufe • vor 1 Monat

The theory of higher order topological dynamics, which combines multilevel interactions between discrete topology and nonlinear dynamics, has the potential to enhance our understanding of complex systems such as the functions of the nervous system, the development of next-generation machine learning and the creation of advanced nodal processing algorithms. An important and unexpected collective behavior of signal processing in multilevel nodal networks has been observed to lead to a synchronization and diffusion of the irrotational and the solenoidal components of the systems revealing a deep relation of these mechanisms with the complexity of discrete topology. The perspective of the preliminary study linked here offers insights into how topology morphs dynamics, how dynamics stem from topology and how topology evolves dynamically. 🔗

The theory of higher order topological dynamics, which combines multilevel interactions between discrete topology and nonlinear dynamics, has the potential to enhance our understanding of complex systems such as the functions of the nervous system, the development of next-generation machine learning and the creation of advanced nodal processing algorithms. An important and unexpected collective behavior of signal processing in multilevel nodal networks has been observed to lead to a synchronization and diffusion of the irrotational and the solenoidal components of the systems revealing a deep relation of these mechanisms with the complexity of discrete topology. The perspective of the preliminary study linked here offers insights into how topology morphs dynamics, how dynamics stem from topology and how topology evolves dynamically. 🔗

Maurizio Iβλἄ

40,749 Aufrufe • vor 1 Jahr

Attention, Helldivers. July 30 has been designated as a date of operational significance. Update your calendar accordingly. Further briefing to follow.

Attention, Helldivers. July 30 has been designated as a date of operational significance. Update your calendar accordingly. Further briefing to follow.

HELLDIVERS™ 2

2,682,018 Aufrufe • vor 2 Monaten

In nature, researchers from Google Quantum AI, TU München, & UoN Physics & Astronomy simulated a (2+1)D lattice gauge theory and visualized string dynamics, revealing the deconfined-to-confined excitation transition as the effective electric field increases. →

In nature, researchers from Google Quantum AI, TU München, & UoN Physics & Astronomy simulated a (2+1)D lattice gauge theory and visualized string dynamics, revealing the deconfined-to-confined excitation transition as the effective electric field increases. →

Google Quantum AI

20,878 Aufrufe • vor 1 Jahr

The laws of physics apply everywhere. ⚛️ Co-developed by NVIDIA, Google DeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the The Linux Foundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning. Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 #CoRL2025

The laws of physics apply everywhere. ⚛️ Co-developed by NVIDIA, Google DeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the The Linux Foundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning. Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 #CoRL2025

NVIDIA Robotics

83,214 Aufrufe • vor 11 Monaten

As a learning test, I ported part of my 2D Cloth Physics Simulation to the web using #Rust 🦀 🧪 Try the mini demo: Real-time simulation with Verlet integration, touch/mouse interaction, and web delivery. #gamedev #physics #simulation #verletintegration

As a learning test, I ported part of my 2D Cloth Physics Simulation to the web using #Rust 🦀 🧪 Try the mini demo: Real-time simulation with Verlet integration, touch/mouse interaction, and web delivery. #gamedev #physics #simulation #verletintegration

Claudio Z.

90,876 Aufrufe • vor 1 Jahr

How did initially independent AI agents form a swarm in a recent safety incident? Our physics of agents theory from March predicts rapid collective belief collapse when many agents with plastic personas exchange short messages. We now need Mechanistic Swarm Interpretability! 🧵

How did initially independent AI agents form a swarm in a recent safety incident? Our physics of agents theory from March predicts rapid collective belief collapse when many agents with plastic personas exchange short messages. We now need Mechanistic Swarm Interpretability! 🧵

Hidenori Tanaka

34,376 Aufrufe • vor 12 Tagen

This is literally magic 🤯 FMA-Net is a new AI method for video deblurring! It uses complex motion representation learning for spatio-temporally-variant restoration with kernels that are aware of motion trajectories. More info below ⬇️⬇️

This is literally magic 🤯 FMA-Net is a new AI method for video deblurring! It uses complex motion representation learning for spatio-temporally-variant restoration with kernels that are aware of motion trajectories. More info below ⬇️⬇️

Alex Carlier

144,291 Aufrufe • vor 2 Jahren

Test-time learning has a catch that scaling alone does not solve: the state a system writes to in order to learn is also the record a user must inspect to audit it. Optimize it only for learning, and auditability usually degrades. Blog: Part I treats the learning state and the audit record as the same object. This starts the ADAPT series, exploring the architecture section by section.

Test-time learning has a catch that scaling alone does not solve: the state a system writes to in order to learn is also the record a user must inspect to audit it. Optimize it only for learning, and auditability usually degrades. Blog: Part I treats the learning state and the audit record as the same object. This starts the ADAPT series, exploring the architecture section by section.

Rei

37,866 Aufrufe • vor 1 Monat

Jungkook, with an IQ of 128, is actually a whole physics genius, both theoretical and applied. He has not only told us several physics factoids but also uses physics principles to effortlessly do things that seem difficult.

Jungkook, with an IQ of 128, is actually a whole physics genius, both theoretical and applied. He has not only told us several physics factoids but also uses physics principles to effortlessly do things that seem difficult.

🕷️

50,428 Aufrufe • vor 21 Tagen

Attention students! Witchbrook's first dev blog is here! We'll be giving you a peek behind the curtain of Mossport's simulation, and exploring the magic behind the game's living world: 📰

Attention students! Witchbrook's first dev blog is here! We'll be giving you a peek behind the curtain of Mossport's simulation, and exploring the magic behind the game's living world: 📰

✨Witchbrook ✧ Wishlist now!

62,140 Aufrufe • vor 1 Jahr

AI in classrooms has the potential to break down barriers and provide personalised learning experiences 🚀📈 That’s why we’re trialling the use of AI tutoring tools in schools, creating safe UK benchmarks 💻

AI in classrooms has the potential to break down barriers and provide personalised learning experiences 🚀📈 That’s why we’re trialling the use of AI tutoring tools in schools, creating safe UK benchmarks 💻

Department for Science, Innovation and Technology

25,795 Aufrufe • vor 7 Monaten

🚨 BREAKING: Hisbah Official Issues Warning to Plateau Governor A Hisbah Islamic enforcement officer has reportedly warned Plateau State Governor Caleb Mutfwang, insisting that Plateau is a Sharia state and that Hisbah officials have the right to enforce compliance with Sharia laws. This is a serious matter that deserves immediate clarification from the Plateau State Government and relevant authorities. Can an Islamic religious enforcement body legally impose Sharia rules on residents of Plateau State, regardless of their faith? Nigeria is a multi-religious country, and the rights of every citizen Christian, Muslim or traditional worshipper must be protected under the Constitution. Religious freedom and the rule of law must remain above religious or political pressure. 🇳🇬

🚨 BREAKING: Hisbah Official Issues Warning to Plateau Governor A Hisbah Islamic enforcement officer has reportedly warned Plateau State Governor Caleb Mutfwang, insisting that Plateau is a Sharia state and that Hisbah officials have the right to enforce compliance with Sharia laws. This is a serious matter that deserves immediate clarification from the Plateau State Government and relevant authorities. Can an Islamic religious enforcement body legally impose Sharia rules on residents of Plateau State, regardless of their faith? Nigeria is a multi-religious country, and the rights of every citizen Christian, Muslim or traditional worshipper must be protected under the Constitution. Religious freedom and the rule of law must remain above religious or political pressure. 🇳🇬

Bia Pulse

24,500 Aufrufe • vor 22 Tagen

Building robots that can effectively operate alongside human workers is difficult. 🛠️ Advances in open-source physics, open foundation models, and frameworks are helping accelerate physical #AI deployment. ✔️ Newton Physics Engine, an open-source GPU-powered simulation built on OpenUSD, speeds up robot learning for advanced manipulation and mobility. ✔️ NVIDIA Cosmos Reason, an open reasoning vision language model, gives robots the ability to think like humans using prior knowledge, common sense and physics ✔️NVIDIA Isaac GR00T N1.6, an open robot foundation model, enables humanoids to understand ambiguous instructions Leading robotics developers including Agility Robotics, Lightwheel, Mentee Robotics, UniversalRobots, and Wandelbots are adopting simulation technologies and libraries to accelerate physical AI development and deployment. Omniverse Ambassador Dylan Tobin built an AI chatbot trained on Isaac Sim workflows, helping devs navigate Omniverse faster. Read the full blog 👉

Building robots that can effectively operate alongside human workers is difficult. 🛠️ Advances in open-source physics, open foundation models, and frameworks are helping accelerate physical #AI deployment. ✔️ Newton Physics Engine, an open-source GPU-powered simulation built on OpenUSD, speeds up robot learning for advanced manipulation and mobility. ✔️ NVIDIA Cosmos Reason, an open reasoning vision language model, gives robots the ability to think like humans using prior knowledge, common sense and physics ✔️NVIDIA Isaac GR00T N1.6, an open robot foundation model, enables humanoids to understand ambiguous instructions Leading robotics developers including Agility Robotics, Lightwheel, Mentee Robotics, UniversalRobots, and Wandelbots are adopting simulation technologies and libraries to accelerate physical AI development and deployment. Omniverse Ambassador Dylan Tobin built an AI chatbot trained on Isaac Sim workflows, helping devs navigate Omniverse faster. Read the full blog 👉

NVIDIA

48,500 Aufrufe • vor 11 Monaten

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:

Infini-AI-Lab

79,105 Aufrufe • vor 3 Monaten