Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Cosmos Policy Fine-Tuning Video Models for Visuomotor Control and Planning

_akhaliq's profile picture

AK

525,948 subscribers

10,355 Aufrufe • vor 8 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von AK
AKvor 8 Monaten

paper:

Profilbild von Moo Jin Kim
Moo Jin Kimvor 8 Monaten

Thank you for sharing our work! 🙏

Profilbild von Nathan Wang
Nathan Wangvor 8 Monaten

Video generation models aren’t just for making clips anymore—they’re becoming the robot brain itself. The part that really clicked for me in this NVIDIA/Stanford paper is the simplicity. Instead of bolting on complex control modules, they just treat robot actions as "latent frames" within the video sequence. Because of this, a single fine-tuned video model (Cosmos-Predict2) acts as the policy, the world simulator, and the value function all at once. It’s hitting 98.5% success on the LIBERO benchmark just by "dreaming" the right future frames and actions together. Makes you wonder—is the path to AGI just a really, really good video predictor?

Ähnliche Videos

Introducing OFT—an Optimized Fine-Tuning recipe for VLAs! Fine-tuning OpenVLA w/ OFT, we see: -25-50x faster inference ⚡️ -SOTA 97.1% avg SR in LIBERO 💪 -high-freq control w/ 7B model on real bimanual robot -outperforms π₀, RDT-1B, DiT Policy, MDT, Diffusion Policy, ACT 🧵👇
0:25

Introducing OFT—an Optimized Fine-Tuning recipe for VLAs! Fine-tuning OpenVLA w/ OFT, we see: -25-50x faster inference ⚡️ -SOTA 97.1% avg SR in LIBERO 💪 -high-freq control w/ 7B model on real bimanual robot -outperforms π₀, RDT-1B, DiT Policy, MDT, Diffusion Policy, ACT 🧵👇

Moo Jin Kim

84,359 Aufrufe • vor 1 Jahr

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️
0:36

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

merve

16,733 Aufrufe • vor 1 Jahr

This is a research preview. It requires more prompt engineering than previous models - but the generations are breathtaking. We’ll continue fine-tuning to improve reliability and control.
0:25

This is a research preview. It requires more prompt engineering than previous models - but the generations are breathtaking. We’ll continue fine-tuning to improve reliability and control.

ElevenLabs

41,102 Aufrufe • vor 1 Jahr

intensity, precision, conditioning and passion.. fine tuning the fine tuning 🤼‍♀️
1:27

intensity, precision, conditioning and passion.. fine tuning the fine tuning 🤼‍♀️

Nattie

55,627 Aufrufe • vor 1 Jahr

.Fireworks AI is helping teams scale AI applications with a production platform for open-weight models, supporting world-class inference, fine-tuning, and enterprise AI workloads.
0:46

.Fireworks AI is helping teams scale AI applications with a production platform for open-weight models, supporting world-class inference, fine-tuning, and enterprise AI workloads.

Microsoft Azure

15,485 Aufrufe • vor 2 Monaten

Fine-tuning for tomorrow 🔧 #ItalianGP
0:22

Fine-tuning for tomorrow 🔧 #ItalianGP

SuperFile Trackhouse MotoGP Team

14,120 Aufrufe • vor 3 Monaten

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️
0:38

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️

NVIDIA Robotics

14,404 Aufrufe • vor 7 Monaten

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.
0:27

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.

Robots Digest 🤖

11,076 Aufrufe • vor 3 Monaten

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]
0:25

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 Aufrufe • vor 2 Jahren

Fine Tuning is in progress for better visibility and understanding Greater Chennai Corporation
0:35

Fine Tuning is in progress for better visibility and understanding Greater Chennai Corporation

KGB

18,063 Aufrufe • vor 8 Monaten

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move
3:58

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,697 Aufrufe • vor 1 Jahr

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source, 51k+ stars 🌟
0:46

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source, 51k+ stars 🌟

Akshay 🚀

60,627 Aufrufe • vor 1 Jahr

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source with 50k stars!
0:46

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source with 50k stars!

Avi Chawla

557,650 Aufrufe • vor 1 Jahr

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.
1:16

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

Standard Intelligence

72,463 Aufrufe • vor 7 Monaten

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️
0:35

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️

NVIDIA Newsroom

47,166 Aufrufe • vor 3 Monaten

Conversation w/ Arthur Mensch CEO of Mistral AI Figma *Mistral models *OS & opencore AI *Fine tuning, context windows,... *Vertical AI models *EU & French startup scene
38:26

Conversation w/ Arthur Mensch CEO of Mistral AI Figma *Mistral models *OS & opencore AI *Fine tuning, context windows,... *Vertical AI models *EU & French startup scene

Elad Gil

76,854 Aufrufe • vor 2 Jahren

Stephen Meyer explains the origins of the fine tuning argument for God and why it’s so compelling.
3:30

Stephen Meyer explains the origins of the fine tuning argument for God and why it’s so compelling.

Darwin to Jesus

19,617 Aufrufe • vor 3 Monaten

Rosy Persona by Art Director Joe Zheng explores lighting and animation using UE 5.6 Metahuman with vibrant pink-hued setups. Unreal Engine allows for fine-tuning real-time Metahuman rendering and light control, which is perfect for stylized looks 👩‍🦰
0:56

Rosy Persona by Art Director Joe Zheng explores lighting and animation using UE 5.6 Metahuman with vibrant pink-hued setups. Unreal Engine allows for fine-tuning real-time Metahuman rendering and light control, which is perfect for stylized looks 👩‍🦰

Unreal Engine

52,641 Aufrufe • vor 1 Jahr