Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Cosmos Policy Fine-Tuning Video Models for Visuomotor Control and Planning

_akhaliq's profile picture

AK

525,948 subscribers

10,355 görüntüleme • 8 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

3 Yorum

AK profil fotoğrafı
AK8 ay önce

paper:

Moo Jin Kim profil fotoğrafı
Moo Jin Kim8 ay önce

Thank you for sharing our work! 🙏

Nathan Wang profil fotoğrafı
Nathan Wang8 ay önce

Video generation models aren’t just for making clips anymore—they’re becoming the robot brain itself. The part that really clicked for me in this NVIDIA/Stanford paper is the simplicity. Instead of bolting on complex control modules, they just treat robot actions as "latent frames" within the video sequence. Because of this, a single fine-tuned video model (Cosmos-Predict2) acts as the policy, the world simulator, and the value function all at once. It’s hitting 98.5% success on the LIBERO benchmark just by "dreaming" the right future frames and actions together. Makes you wonder—is the path to AGI just a really, really good video predictor?

Benzer Videolar

Introducing OFT—an Optimized Fine-Tuning recipe for VLAs! Fine-tuning OpenVLA w/ OFT, we see: -25-50x faster inference ⚡️ -SOTA 97.1% avg SR in LIBERO 💪 -high-freq control w/ 7B model on real bimanual robot -outperforms π₀, RDT-1B, DiT Policy, MDT, Diffusion Policy, ACT 🧵👇
0:25

Introducing OFT—an Optimized Fine-Tuning recipe for VLAs! Fine-tuning OpenVLA w/ OFT, we see: -25-50x faster inference ⚡️ -SOTA 97.1% avg SR in LIBERO 💪 -high-freq control w/ 7B model on real bimanual robot -outperforms π₀, RDT-1B, DiT Policy, MDT, Diffusion Policy, ACT 🧵👇

Moo Jin Kim

84,359 görüntüleme • 1 yıl önce

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️
0:36

IN: video fine-tuning support for AI at Meta's V-JEPA 2 in HF transformers 🔥 it comes with > fine-tuning notebook > four models fine-tuned on Diving48 and SSv2 dataset > FastRTC demo on V-JEPA2 SSv2 (see below) we're looking forward to see fine-tuned V-JEPA2 models on Hub ⏯️

merve

16,733 görüntüleme • 1 yıl önce

This is a research preview. It requires more prompt engineering than previous models - but the generations are breathtaking. We’ll continue fine-tuning to improve reliability and control.
0:25

This is a research preview. It requires more prompt engineering than previous models - but the generations are breathtaking. We’ll continue fine-tuning to improve reliability and control.

ElevenLabs

41,102 görüntüleme • 1 yıl önce

intensity, precision, conditioning and passion.. fine tuning the fine tuning 🤼‍♀️
1:27

intensity, precision, conditioning and passion.. fine tuning the fine tuning 🤼‍♀️

Nattie

55,627 görüntüleme • 1 yıl önce

.Fireworks AI is helping teams scale AI applications with a production platform for open-weight models, supporting world-class inference, fine-tuning, and enterprise AI workloads.
0:46

.Fireworks AI is helping teams scale AI applications with a production platform for open-weight models, supporting world-class inference, fine-tuning, and enterprise AI workloads.

Microsoft Azure

15,485 görüntüleme • 2 ay önce

Fine-tuning for tomorrow 🔧 #ItalianGP
0:22

Fine-tuning for tomorrow 🔧 #ItalianGP

SuperFile Trackhouse MotoGP Team

14,120 görüntüleme • 3 ay önce

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️
0:38

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️

NVIDIA Robotics

14,404 görüntüleme • 7 ay önce

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.
0:27

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.

Robots Digest 🤖

11,076 görüntüleme • 3 ay önce

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]
0:25

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 görüntüleme • 2 yıl önce

Fine Tuning is in progress for better visibility and understanding Greater Chennai Corporation
0:35

Fine Tuning is in progress for better visibility and understanding Greater Chennai Corporation

KGB

18,063 görüntüleme • 8 ay önce

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move
3:58

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,697 görüntüleme • 1 yıl önce

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source, 51k+ stars 🌟
0:46

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source, 51k+ stars 🌟

Akshay 🚀

60,627 görüntüleme • 1 yıl önce

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source with 50k stars!
0:46

Fine-tune 100+ LLMs directly from a UI! LLaMA-Factory lets you train and fine-tune open-source LLMs and VLMs without writing any code. Supports 100+ models, multimodal fine-tuning, PPO, DPO, experiment tracking, and much more! 100% open-source with 50k stars!

Avi Chawla

557,650 görüntüleme • 1 yıl önce

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.
1:16

True computer use is fully general. FDM-1 uses arrow keys on a computer to steer a car in San Francisco with less than 1 hour of fine-tuning data. The action policy is critical: tuning FDM-1 to drive gets much higher accuracy than tuning just the video encoder on the same data.

Standard Intelligence

72,463 görüntüleme • 7 ay önce

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️
0:35

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️

NVIDIA Newsroom

47,166 görüntüleme • 3 ay önce

Conversation w/ Arthur Mensch CEO of Mistral AI Figma *Mistral models *OS & opencore AI *Fine tuning, context windows,... *Vertical AI models *EU & French startup scene
38:26

Conversation w/ Arthur Mensch CEO of Mistral AI Figma *Mistral models *OS & opencore AI *Fine tuning, context windows,... *Vertical AI models *EU & French startup scene

Elad Gil

76,854 görüntüleme • 2 yıl önce

Stephen Meyer explains the origins of the fine tuning argument for God and why it’s so compelling.
3:30

Stephen Meyer explains the origins of the fine tuning argument for God and why it’s so compelling.

Darwin to Jesus

19,617 görüntüleme • 3 ay önce

Rosy Persona by Art Director Joe Zheng explores lighting and animation using UE 5.6 Metahuman with vibrant pink-hued setups. Unreal Engine allows for fine-tuning real-time Metahuman rendering and light control, which is perfect for stylized looks 👩‍🦰
0:56

Rosy Persona by Art Director Joe Zheng explores lighting and animation using UE 5.6 Metahuman with vibrant pink-hued setups. Unreal Engine allows for fine-tuning real-time Metahuman rendering and light control, which is perfect for stylized looks 👩‍🦰

Unreal Engine

52,641 görüntüleme • 1 yıl önce