Vai Viswanathan's banner

Vai Viswanathan

@vai_viswanathan • 1,847 subscribers

PHYSICAL AI exploring @southpkcommons investing @ https://t.co/qeVE4byi9t prev. https://t.co/QJrkahGGVy, @canaryaero

Shorts

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly

36,039 просмотров

For years Yann LeCun has argued that generative video models can't truly learn physics. DeepMind's Physics-IQ benchmark proved him right, reporting "a striking lack of physical understanding in current generative video models". The best one scored just 29.5%. This CVPR 2026 paper finds the fix in LeCun's own playbook. "Inference-time Physics Alignment" doesn't retrain the generator. It steers a video model's denoising at inference using a reward from VJEPA-2, LeCun's Joint Embedding Predictive Architecture. - Won first place in the ICCV 2025 PhysicsIQ Challenge with a 62.64% score, beating the previous state of the art by 7.42%. - Repurpose VJEPA-2's "surprise" score as a reward, then search and rank multiple candidate denoising trajectories at test time. - Why it matters: It's a neat vindication of LeCun's thesis. The pixel-prediction generator alone doesn't get physics, but the JEPA world model approach he champions supplies the physics the video model lacks.

For years Yann LeCun has argued that generative video models can't truly learn physics. DeepMind's Physics-IQ benchmark proved him right, reporting "a striking lack of physical understanding in current generative video models". The best one scored just 29.5%. This CVPR 2026 paper finds the fix in LeCun's own playbook. "Inference-time Physics Alignment" doesn't retrain the generator. It steers a video model's denoising at inference using a reward from VJEPA-2, LeCun's Joint Embedding Predictive Architecture. - Won first place in the ICCV 2025 PhysicsIQ Challenge with a 62.64% score, beating the previous state of the art by 7.42%. - Repurpose VJEPA-2's "surprise" score as a reward, then search and rank multiple candidate denoising trajectories at test time. - Why it matters: It's a neat vindication of LeCun's thesis. The pixel-prediction generator alone doesn't get physics, but the JEPA world model approach he champions supplies the physics the video model lacks.

44,327 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

Vai Viswanathan

32,053 просмотров • 24 дней назад

32,000+ hours of open robot data 🤖 ANYONE can get started training their own robotics models!

32,000+ hours of open robot data 🤖 ANYONE can get started training their own robotics models!

Vai Viswanathan

13,368 просмотров • 1 месяц назад

Больше нет контента для загрузки