正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🤔Want a principled way to RL your diffusion model? Check Data-regularized Reinforcement Learning (DDRL)! Post-train NVIDIA #Cosmos World Foundation models with a million GPU hours! 🤯 Novel formulation ➡️ Theoretically integrates SFT into RL ➡️ Robust to Reward Hacking 🛑 Details: #DDRL #Diffusion #RL #NVIDIA #Cosmos

Haotian Ye

1,321 subscribers

78,565 次观看 • 11 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 条评论

Jiarui Liu 的头像

Jiarui Liu10 个月前

@nvidia Nice work! In our recent paper on GRPO training for LLMs, we have a similar philosophy that unifies RL and SFT into a single objective:

Haotian Ye 的头像

Haotian Ye10 个月前

@nvidia that's so cool!

Charles 的头像

Charles10 个月前

@nvidia greate work! when will it be open sourced?

Haotian Ye 的头像

Haotian Ye10 个月前

@nvidia very soon :)

Himanshu Kumar 的头像

Himanshu Kumar10 个月前

@nvidia Wow, Haotian, that DDRL approach sounds quite promising, especially for robust reward hacking, isn't it?

Haotian Ye 的头像

Haotian Ye10 个月前

@nvidia yeah! reward hacking is a big motivation.

Yuanzhi 的头像

Yuanzhi10 个月前

@nvidia Very cool work!!

相关视频

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.

Sawyer Merritt

254,398 次观看 • 1 年前

We teamed up with NVIDIA and Matthew Berman to teach you how to do Reinforcement Learning! Learn about: - RL environments, reward functions & reward hacking - Training OpenAI gpt-oss to automatically solve 2048 - Local Windows training with NVIDIA_AI_PC RTX GPUs - How RLVR (verifiable rewards) works - How to interpret RL metrics like KL Divergence Full video tutorial:

We teamed up with NVIDIA and Matthew Berman to teach you how to do Reinforcement Learning! Learn about: - RL environments, reward functions & reward hacking - Training OpenAI gpt-oss to automatically solve 2048 - Local Windows training with NVIDIA_AI_PC RTX GPUs - How RLVR (verifiable rewards) works - How to interpret RL metrics like KL Divergence Full video tutorial:

Unsloth AI

54,307 次观看 • 10 个月前

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

34,102 次观看 • 4 个月前

NVIDIA just introduced Cosmos, a platform for world foundation models designed for robotics. ⦿ It features advanced tokenizers, an AI-accelerated data pipeline, and integration with NVIDIA Omniverse. Humanoid makers 1X, Figure, and Agility are among the first to adopt Cosmos. ⦿ Cosmos generates synthetic, physics-based data, accelerating model training and customization. ⦿ It also features a CUDA-accelerated data processing pipeline that enables developers to process, curate, and label 20 million hours of videos in 14 days using the NVIDIA Blackwell platform.

NVIDIA just introduced Cosmos, a platform for world foundation models designed for robotics. ⦿ It features advanced tokenizers, an AI-accelerated data pipeline, and integration with NVIDIA Omniverse. Humanoid makers 1X, Figure, and Agility are among the first to adopt Cosmos. ⦿ Cosmos generates synthetic, physics-based data, accelerating model training and customization. ⦿ It also features a CUDA-accelerated data processing pipeline that enables developers to process, curate, and label 20 million hours of videos in 14 days using the NVIDIA Blackwell platform.

The Humanoid Hub

129,481 次观看 • 1 年前

RL-100 Performant Robotic Manipulation with Real-World Reinforcement Learning

RL-100 Performant Robotic Manipulation with Real-World Reinforcement Learning

AK

15,364 次观看 • 1 年前

We open-sourced QeRL — Quantization-enhanced Reinforcement Learning ! 🧠 4-bit quantized RL training 💪 Train a 32B LLM on a single H100 GPU ⚙️ 1.7× faster overall training 🎯 Accuracy on par with bfloat16-level accuracy 🔥 Supports NVFP4 quantization format Moreover, we show that quantization helps exploration in RL training. Paper: Code: #NVIDIA #AIResearch #ReinforcementLearning #Quantization #LLM #EfficientAI

We open-sourced QeRL — Quantization-enhanced Reinforcement Learning ! 🧠 4-bit quantized RL training 💪 Train a 32B LLM on a single H100 GPU ⚙️ 1.7× faster overall training 🎯 Accuracy on par with bfloat16-level accuracy 🔥 Supports NVFP4 quantization format Moreover, we show that quantization helps exploration in RL training. Paper: Code: #NVIDIA #AIResearch #ReinforcementLearning #Quantization #LLM #EfficientAI

Yukang Chen

69,896 次观看 • 1 年前

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️

NVIDIA Cosmos 3 launches with a full model family — Cosmos Super for highest-accuracy robotics and AV post-training, Cosmos Nano for high-speed video and action reasoning, and Cosmos Edge for real-time edge inference. Read the release ➡️

NVIDIA Newsroom

47,166 次观看 • 5 个月前

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

Sergey Levine

153,232 次观看 • 1 年前

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Satpal Singh Rathore

46,010 次观看 • 1 年前

RobotMDM, by Disney Research, combines diffusion-based motion generation with RL to produce physics-aware humanoid motions from text prompts. Trained on human motion data with a reward surrogate for physical feasibility, it ensures realistic motions.

RobotMDM, by Disney Research, combines diffusion-based motion generation with RL to produce physics-aware humanoid motions from text prompts. Trained on human motion data with a reward surrogate for physical feasibility, it ensures realistic motions.

The Humanoid Hub

22,943 次观看 • 2 年前

🤝 Arcee.ai, an American AI lab, built its frontier open model family on NVIDIA Blackwell Ultra. Its flagship MoE model, Trinity-Large-Thinking, was RL post-trained using NVIDIA NeMo open libraries. Arcee optimized agentic model inference with NVIDIA Dynamo and vLLM, along with NVIDIA accelerated networking, to deliver low token cost, achieving: ✅ 3T+ tokens served on OpenRouter in its first two months ✅ $0.90 per 1M output tokens ✅ 2nd on PinchBench for open agentic models Explore Arcee's open models, powered by NVIDIA ➡️

🤝 Arcee.ai, an American AI lab, built its frontier open model family on NVIDIA Blackwell Ultra. Its flagship MoE model, Trinity-Large-Thinking, was RL post-trained using NVIDIA NeMo open libraries. Arcee optimized agentic model inference with NVIDIA Dynamo and vLLM, along with NVIDIA accelerated networking, to deliver low token cost, achieving: ✅ 3T+ tokens served on OpenRouter in its first two months ✅ $0.90 per 1M output tokens ✅ 2nd on PinchBench for open agentic models Explore Arcee's open models, powered by NVIDIA ➡️

NVIDIA AI Infrastructure

27,377 次观看 • 3 个月前

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,696 次观看 • 1 年前

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

447,483 次观看 • 7 个月前

literally one of the best introductions to reinforcement learning today. only within 90 minutes. > framing of rl problems: agent, environment, states, actions, rewards > grid maze: world models vs model-free > eval methods: monte carlo, td, sarsa > scaling up: dqn, actor/critic the video walks through multiple learning considerations and helps build an intuitive understanding of rl problems. Check this out!

literally one of the best introductions to reinforcement learning today. only within 90 minutes. > framing of rl problems: agent, environment, states, actions, rewards > grid maze: world models vs model-free > eval methods: monte carlo, td, sarsa > scaling up: dqn, actor/critic the video walks through multiple learning considerations and helps build an intuitive understanding of rl problems. Check this out!

Aarno

48,893 次观看 • 3 个月前

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Neuralink

18,410 次观看 • 1 年前

1/ Introducing RL Swarm’s new backend: GenRL. A modular reinforcement learning library built for distributed, fault-tolerant training - now powering RL Swarm from the ground up. 🧵

1/ Introducing RL Swarm’s new backend: GenRL. A modular reinforcement learning library built for distributed, fault-tolerant training - now powering RL Swarm from the ground up. 🧵

gensyn

82,215 次观看 • 1 年前

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation without requiring any paired human-robot data or online RL 🙂 1/n

Homanga Bharadhwaj

33,088 次观看 • 1 年前

Nvidia announces Stable Diffusion Models Accelerated with NVIDIA TensorRT featuring a1111 Gradio demo demo:

Nvidia announces Stable Diffusion Models Accelerated with NVIDIA TensorRT featuring a1111 Gradio demo demo:

AK

112,490 次观看 • 2 年前