Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing PAN 2: an open foundation world model for physical intelligence. A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space. Towards AI that simulates “what if?” before it acts. More details:

2,059,535 Aufrufe • vor 6 Tagen •via X (Twitter)

19 Kommentare

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

PAN 2 employs generative latent prediction (GLP). It reasons in two steps. A pre-trained multimodal LLM, the conceptual reasoning backbone, works out what happens and when. A 29B latent diffusion model, the perceptual reasoning backbone, trained from scratch, fills in how it looks and moves. More details in:

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

PAN 2 is open under Apache 2.0: weights, training code and inference code. Predict the next world on your own machine with vLLM, Hugging Face Diffusers or FastVideo, or train it yourself. • Training code: • vLLM: • Hugging Face Diffusers: • FastVideo: • Weights:

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

PAN 2 is grounded in time. Timing and order change the outcome. PAN 2 simulates what will happen, with the timing and order you set, and each step builds on the last one. That’s the base for multi-step simulation that can guide an agent’s decisions: an agent can only trust a simulation that gets every step right. Below: same surfer, same wave, different pop-up times. Watch which one works.

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

PAN 2 is grounded in physics. Its worlds obey physical laws, respect real-world constraints, and have a life of their own. Below: milk, ink and oil are each added to a tube of water. Milk clouds, ink sinks and spreads, and oil floats. PAN 2 simulates the world, not just paints pixels.

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

PAN 2 is grounded in space. Move the camera, and the world it shows doesn't change. Objects keep their identity and state, even when covered or out of view. An agent can't plan in a world that rearranges itself whenever it looks away. Below: a child stacks two blocks, then turns to look at the cat, which wanders around on its own. When the child looks back, the blocks are still stacked.

Profilbild von Institute of Foundation Models
Institute of Foundation Modelsvor 6 Tagen

Ever played Jenga and wondered, "What if I pull this one?" PAN 2 lets you see before you pull. Change one thing in a scene and see how outcome changes. Simulations like this let agents learn and plan without costly, slow or irreversible real-world trials.

Profilbild von Kaitee
Kaiteevor 6 Tagen

does it handle rope or cables at all?

Profilbild von Dawna Yi
Dawna Yivor 6 Tagen

hmmm what’s the major difference reasoning gives to this model compared to diffusion video models??

Profilbild von Ella Tech & Tool
Ella Tech & Toolvor 6 Tagen

World model predicting what happens after each action sounds wonderfully powerful

Profilbild von Aaliya
Aaliyavor 6 Tagen

impressed predicting the outcome of actions before executing them is a big step for AI agents.

Profilbild von AI_Explorer
AI_Explorervor 6 Tagen

can an agent call it in a loop while it works?

Profilbild von GPUMonk
GPUMonkvor 6 Tagen

@grok is this something like JEPA ?

Profilbild von Historic Vids
Historic Vidsvor 6 Tagen

Does it know a ball rolls differently on carpet than on tile?

Profilbild von Sharon Riley
Sharon Rileyvor 6 Tagen

Predicting outcomes before acting could change everything ahead.

Profilbild von Mikael
Mikaelvor 6 Tagen

What counts as a new combination? like a familiar object in a room it's never seen?

Profilbild von Amelia
Ameliavor 6 Tagen

does camera motion count as an action? like "pan left while the cup falls

Profilbild von Neo Kim
Neo Kimvor 6 Tagen

34.0% of full plans and per IFM that's still more than any other model tested. full plans are hard

Profilbild von el.cine
el.cinevor 6 Tagen

this great solution for consistency issue in video generation

Profilbild von Chase Dimond | Email Marketing Nerd 📧
Chase Dimond | Email Marketing Nerd 📧vor 6 Tagen

per IFM, 8.7% of steps without the conceptual reasoner vs 75.1% with it. wild gap pumped to be working with y'all

Ähnliche Videos