Vai Viswanathan's banner

Vai Viswanathan

@vai_viswanathan • 1,877 subscribers

PHYSICAL AI prev. https://t.co/QJrkahGGVy, @canaryaero

Shorts

I didn't believe it until I tried it myself! This only took 30 minutes

I didn't believe it until I tried it myself! This only took 30 minutes

33,719 görüntüleme

For years Yann LeCun has argued that generative video models can't truly learn physics. DeepMind's Physics-IQ benchmark proved him right, reporting "a striking lack of physical understanding in current generative video models". The best one scored just 29.5%. This CVPR 2026 paper finds the fix in LeCun's own playbook. "Inference-time Physics Alignment" doesn't retrain the generator. It steers a video model's denoising at inference using a reward from VJEPA-2, LeCun's Joint Embedding Predictive Architecture. - Won first place in the ICCV 2025 PhysicsIQ Challenge with a 62.64% score, beating the previous state of the art by 7.42%. - Repurpose VJEPA-2's "surprise" score as a reward, then search and rank multiple candidate denoising trajectories at test time. - Why it matters: It's a neat vindication of LeCun's thesis. The pixel-prediction generator alone doesn't get physics, but the JEPA world model approach he champions supplies the physics the video model lacks.

For years Yann LeCun has argued that generative video models can't truly learn physics. DeepMind's Physics-IQ benchmark proved him right, reporting "a striking lack of physical understanding in current generative video models". The best one scored just 29.5%. This CVPR 2026 paper finds the fix in LeCun's own playbook. "Inference-time Physics Alignment" doesn't retrain the generator. It steers a video model's denoising at inference using a reward from VJEPA-2, LeCun's Joint Embedding Predictive Architecture. - Won first place in the ICCV 2025 PhysicsIQ Challenge with a 62.64% score, beating the previous state of the art by 7.42%. - Repurpose VJEPA-2's "surprise" score as a reward, then search and rank multiple candidate denoising trajectories at test time. - Why it matters: It's a neat vindication of LeCun's thesis. The pixel-prediction generator alone doesn't get physics, but the JEPA world model approach he champions supplies the physics the video model lacks.

44,407 görüntüleme

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly

You pretrained a robot policy on millions of camera frames. Now you want to add a force sensor. Do you really have to retrain on everything from scratch? MuSe adds a force-torque sensor to a frozen vision-only policy using a tiny amount of contact data. - lifts contact-rich task success (peg insertion 60% to 87%, vase wiping 33% to 77%) - fuse the new sensor both early (shared token space) and late (cross-attention), train the policy as a world model that predicts future video, future force, and actions together, and replay old vision-only data with the force input masked to prevent forgetting. - adding new sensor modalities to pre-trained World Model can be cheap & improve performance significantly

36,059 görüntüleme

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

A world model controlling a drone flying outdoors at 100Hz! 🚁 - Cuts position-tracking error 26-38% and attitude error up to 54% versus predictive baselines - Trained in simulation and deployed zero-shot on hardware. - predicts future dynamics in latent space instead of forecasting raw states, dodging the compounding-error drift that wrecks autoregressive models. A physics-inspired “prober” maps frozen latents back to real position, velocity, and attitude for an onboard MPPI controller. - Why it matters: The first example of JEPA world models surviving the messiness of outdoor deployment. The whole model is ~9K parameters, small enough to run on an embedded flight computer 🧠

A world model controlling a drone flying outdoors at 100Hz! 🚁 - Cuts position-tracking error 26-38% and attitude error up to 54% versus predictive baselines - Trained in simulation and deployed zero-shot on hardware. - predicts future dynamics in latent space instead of forecasting raw states, dodging the compounding-error drift that wrecks autoregressive models. A physics-inspired “prober” maps frozen latents back to real position, velocity, and attitude for an onboard MPPI controller. - Why it matters: The first example of JEPA world models surviving the messiness of outdoor deployment. The whole model is ~9K parameters, small enough to run on an embedded flight computer 🧠

Vai Viswanathan

52,773 görüntüleme • 3 ay önce

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

We have nearly 40,000 hours of open robotics data on Check it out and train your own robotics policy or world model

Vai Viswanathan

32,320 görüntüleme • 2 ay önce

32,000+ hours of open robot data 🤖 ANYONE can get started training their own robotics models!

32,000+ hours of open robot data 🤖 ANYONE can get started training their own robotics models!

Vai Viswanathan

14,193 görüntüleme • 3 ay önce

Daha fazla içerik yok.