Загрузка видео...

Не удалось загрузить видео

На главную

"Physics of AI" blog update: Learning sparse attention has a sticky plateau, representation collapse, and branching dynamics

ZimingLiu11's profile picture

Ziming Liu

14,333 subscribers

81,780 просмотров • 6 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

One man, one dog and 2000 sheep: a time lapse that has so many physics and fluid dynamics references [source and full video:
0:18

One man, one dog and 2000 sheep: a time lapse that has so many physics and fluid dynamics references [source and full video:

Massimo

7,440,047 просмотров • 3 лет назад

This rocket engine wasn't designed by humans, but by AI. It was created using Noyron, a physics based engineering AI built by LEAP 71. Instead of learning from past designs, it uses real physics, thermodynamics, and manufacturing rules to generate engines directly from requirements.
0:19

This rocket engine wasn't designed by humans, but by AI. It was created using Noyron, a physics based engineering AI built by LEAP 71. Instead of learning from past designs, it uses real physics, thermodynamics, and manufacturing rules to generate engines directly from requirements.

vixhaℓ

587,660 просмотров • 6 месяцев назад

The Branching sickness End of Year project update is here. Head over to our discord to take a look under the hood of current Branching Sickness development and apply to join the team of volunteers dedicated to bringing this game to life.
0:12

The Branching sickness End of Year project update is here. Head over to our discord to take a look under the hood of current Branching Sickness development and apply to join the team of volunteers dedicated to bringing this game to life.

Branching Sickness

85,709 просмотров • 2 лет назад

Update 0.5 for the Early Access Program of Assetto Corsa Rally is almost here and it's a big one! It brings you Rally Greece, the first iteration of Multiplayer, an updated tyre model & physics update with FFB fixes and improvements, and much more! Get the full details and changelog in the blog post below! #AssettoCorsa #ACRally #Rally
0:15

Update 0.5 for the Early Access Program of Assetto Corsa Rally is almost here and it's a big one! It brings you Rally Greece, the first iteration of Multiplayer, an updated tyre model & physics update with FFB fixes and improvements, and much more! Get the full details and changelog in the blog post below! #AssettoCorsa #ACRally #Rally

Assetto Corsa

29,424 просмотров • 24 дней назад

The theory of higher order topological dynamics, which combines multilevel interactions between discrete topology and nonlinear dynamics, has the potential to enhance our understanding of complex systems such as the functions of the nervous system, the development of next-generation machine learning and the creation of advanced nodal processing algorithms. An important and unexpected collective behavior of signal processing in multilevel nodal networks has been observed to lead to a synchronization and diffusion of the irrotational and the solenoidal components of the systems revealing a deep relation of these mechanisms with the complexity of discrete topology. The perspective of the preliminary study linked here offers insights into how topology morphs dynamics, how dynamics stem from topology and how topology evolves dynamically. 🔗
0:20

The theory of higher order topological dynamics, which combines multilevel interactions between discrete topology and nonlinear dynamics, has the potential to enhance our understanding of complex systems such as the functions of the nervous system, the development of next-generation machine learning and the creation of advanced nodal processing algorithms. An important and unexpected collective behavior of signal processing in multilevel nodal networks has been observed to lead to a synchronization and diffusion of the irrotational and the solenoidal components of the systems revealing a deep relation of these mechanisms with the complexity of discrete topology. The perspective of the preliminary study linked here offers insights into how topology morphs dynamics, how dynamics stem from topology and how topology evolves dynamically. 🔗

Maurizio Iβλἄ

40,749 просмотров • 1 год назад

In nature, researchers from Google Quantum AI, TU München, & UoN Physics & Astronomy simulated a (2+1)D lattice gauge theory and visualized string dynamics, revealing the deconfined-to-confined excitation transition as the effective electric field increases. →
0:13

In nature, researchers from Google Quantum AI, TU München, & UoN Physics & Astronomy simulated a (2+1)D lattice gauge theory and visualized string dynamics, revealing the deconfined-to-confined excitation transition as the effective electric field increases. →

Google Quantum AI

20,862 просмотров • 1 год назад

The laws of physics apply everywhere. ⚛️ Co-developed by NVIDIA, Google DeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the The Linux Foundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning. Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 #CoRL2025
0:11

The laws of physics apply everywhere. ⚛️ Co-developed by NVIDIA, Google DeepMind and Disney Research, Walt Disney Imagineering, the new Newton Beta, managed by the The Linux Foundation, now also runs with Isaac Lab and with MuJoCo Warp, which delivers warp speed for robot learning. Learn how to train a quadruped and simulate multiphysics in our tech blog. 🔗 #CoRL2025

NVIDIA Robotics

83,151 просмотров • 9 месяцев назад

As a learning test, I ported part of my 2D Cloth Physics Simulation to the web using #Rust 🦀 🧪 Try the mini demo: Real-time simulation with Verlet integration, touch/mouse interaction, and web delivery. #gamedev #physics #simulation #verletintegration
0:18

As a learning test, I ported part of my 2D Cloth Physics Simulation to the web using #Rust 🦀 🧪 Try the mini demo: Real-time simulation with Verlet integration, touch/mouse interaction, and web delivery. #gamedev #physics #simulation #verletintegration

Claudio Z.

90,790 просмотров • 1 год назад

This is literally magic 🤯 FMA-Net is a new AI method for video deblurring! It uses complex motion representation learning for spatio-temporally-variant restoration with kernels that are aware of motion trajectories. More info below ⬇️⬇️
0:19

This is literally magic 🤯 FMA-Net is a new AI method for video deblurring! It uses complex motion representation learning for spatio-temporally-variant restoration with kernels that are aware of motion trajectories. More info below ⬇️⬇️

Alex Carlier

144,268 просмотров • 2 лет назад

Attention students! Witchbrook's first dev blog is here! We'll be giving you a peek behind the curtain of Mossport's simulation, and exploring the magic behind the game's living world: 📰
0:12

Attention students! Witchbrook's first dev blog is here! We'll be giving you a peek behind the curtain of Mossport's simulation, and exploring the magic behind the game's living world: 📰

✨Witchbrook ✧ Wishlist now!

62,084 просмотров • 1 год назад

AI in classrooms has the potential to break down barriers and provide personalised learning experiences 🚀📈 That’s why we’re trialling the use of AI tutoring tools in schools, creating safe UK benchmarks 💻
0:12

AI in classrooms has the potential to break down barriers and provide personalised learning experiences 🚀📈 That’s why we’re trialling the use of AI tutoring tools in schools, creating safe UK benchmarks 💻

Department for Science, Innovation and Technology

25,727 просмотров • 5 месяцев назад

Building robots that can effectively operate alongside human workers is difficult. 🛠️ Advances in open-source physics, open foundation models, and frameworks are helping accelerate physical #AI deployment. ✔️ Newton Physics Engine, an open-source GPU-powered simulation built on OpenUSD, speeds up robot learning for advanced manipulation and mobility. ✔️ NVIDIA Cosmos Reason, an open reasoning vision language model, gives robots the ability to think like humans using prior knowledge, common sense and physics ✔️NVIDIA Isaac GR00T N1.6, an open robot foundation model, enables humanoids to understand ambiguous instructions Leading robotics developers including Agility Robotics, Lightwheel, Mentee Robotics, UniversalRobots, and Wandelbots are adopting simulation technologies and libraries to accelerate physical AI development and deployment. Omniverse Ambassador Dylan Tobin built an AI chatbot trained on Isaac Sim workflows, helping devs navigate Omniverse faster. Read the full blog 👉
0:13

Building robots that can effectively operate alongside human workers is difficult. 🛠️ Advances in open-source physics, open foundation models, and frameworks are helping accelerate physical #AI deployment. ✔️ Newton Physics Engine, an open-source GPU-powered simulation built on OpenUSD, speeds up robot learning for advanced manipulation and mobility. ✔️ NVIDIA Cosmos Reason, an open reasoning vision language model, gives robots the ability to think like humans using prior knowledge, common sense and physics ✔️NVIDIA Isaac GR00T N1.6, an open robot foundation model, enables humanoids to understand ambiguous instructions Leading robotics developers including Agility Robotics, Lightwheel, Mentee Robotics, UniversalRobots, and Wandelbots are adopting simulation technologies and libraries to accelerate physical AI development and deployment. Omniverse Ambassador Dylan Tobin built an AI chatbot trained on Isaac Sim workflows, helping devs navigate Omniverse faster. Read the full blog 👉

NVIDIA

48,500 просмотров • 9 месяцев назад

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:
0:16

RL is painfully slow 😭 — bottlenecked by super-long CoT rollout. 🔭 Sparse attention should help, but naive sparse rollout hits a brutal efficiency–stability tradeoff: A tedious trial-and-error sparsity sweep for each dense policy is required before an actual RL run. 🐤Sparrow chirps no more pain! Introduce Sparrow: Sparse Rollout for stable and efficient long-context RL. Sparrow finds that: 💡As long as we keep the tail distribution mismatch throughout the sparse rollout above a critical threshold, the RL training will be stable. 💡Even cooler! Through comprehensive control studies of Qwen3-1.7B, 4B, 8B thinking models RL with 40K rollout max length, the critical threshold stays constant across model sizes. 💡Sparrow then finds the optimal dynamic sparse schedule to reach the threshold with minimal cost. 💡Sparrow's findings are empirically validated to generalize in Qwen3-14B, and hold on both Math and Coding RL. 🐤Sparrow empirically helps achieve 2.2× / 2.4× / 2.0× rollout speedup on Qwen3 1.7B / 4B / 8B thinking models, while keeping training stability over extended RL steps. We release the 🐤bird in the following formats. [1/n] Paper: Code: Blog:

Infini-AI-Lab

77,891 просмотров • 1 месяц назад

Career Update 🌱 Incredibly excited to share that I've joined (YC F25) as a Design Engineer. I'll be building and shipping AI products that make complex work feel effortless. Here's to creating, shipping, and learning with an incredible team✨
0:13

Career Update 🌱 Incredibly excited to share that I've joined (YC F25) as a Design Engineer. I'll be building and shipping AI products that make complex work feel effortless. Here's to creating, shipping, and learning with an incredible team✨

priya bhatt

22,290 просмотров • 16 дней назад

put the new models in game and worked on a bunch of stuff, including basic follower AI for the overworld, baked lightmaps, cloth physics, and item pickups! [#gamedev | #indiegame | #unity3d]
0:17

put the new models in game and worked on a bunch of stuff, including basic follower AI for the overworld, baked lightmaps, cloth physics, and item pickups! [#gamedev | #indiegame | #unity3d]

gonju

31,508 просмотров • 9 месяцев назад

Do you have experience in microscopy, image analysis, fluid dynamics, network theory, electrophysiology, and/or machine learning? Join us as we track the flows & structure of mycorrhizal networks. We have a biophysics post-doc position open in Amsterdam
0:16

Do you have experience in microscopy, image analysis, fluid dynamics, network theory, electrophysiology, and/or machine learning? Join us as we track the flows & structure of mycorrhizal networks. We have a biophysics post-doc position open in Amsterdam

Toby Kiers

110,349 просмотров • 3 лет назад

Jolt Physics v5.1.0 is out! It includes improvements to soft bodies, characters and vehicles. It also has a ton of bug fixes. Release notes:
0:14

Jolt Physics v5.1.0 is out! It includes improvements to soft bodies, characters and vehicles. It also has a ton of bug fixes. Release notes:

Jorrit Rouwe

32,438 просмотров • 1 год назад

UPDATE: Investigators now focusing a lot of attention on a particular room inside Nancy Guthrie’s home with latex gloves taking Photos and removing several boxes of items. KVOA News 4 Tucson
0:14

UPDATE: Investigators now focusing a lot of attention on a particular room inside Nancy Guthrie’s home with latex gloves taking Photos and removing several boxes of items. KVOA News 4 Tucson

Monica Garcia

296,150 просмотров • 5 месяцев назад

This is the best AI agent-first notes app I've found. It's called . It has the potential to be a productized version of Karpathy's "LLM Wiki" knowledge bases. Here's what I did: 1. Imported "Learning, Fast and Slow" a Continual Learning paper 2. Asked OpenKnowledge to create a visual explainer 3. Read the explainer and had Claude explain to me 4. Created a new section of my own understanding 5. Saved the durable version in my Obsidian vault because it's markdown Free and open-source. Absolutely incredible learning tool.
0:14

This is the best AI agent-first notes app I've found. It's called . It has the potential to be a productized version of Karpathy's "LLM Wiki" knowledge bases. Here's what I did: 1. Imported "Learning, Fast and Slow" a Continual Learning paper 2. Asked OpenKnowledge to create a visual explainer 3. Read the explainer and had Claude explain to me 4. Created a new section of my own understanding 5. Saved the durable version in my Obsidian vault because it's markdown Free and open-source. Absolutely incredible learning tool.

Dan McAteer

16,683 просмотров • 21 дней назад