Загрузка видео...

Не удалось загрузить видео

На главную

NVIDIA Cosmos Reason 2 is here. 🥳 An open, highly accurate reasoning vision language model for physical AI, featuring: ✅ Improved spatio-temporal understanding and timestamp precision ✅ Flexible deployment with 2B and 8B model sizes ✅ Long-context reasoning with up to 256K tokens ✅ Expanded visual perception across complex...

NVIDIAAIDev's profile picture

NVIDIA AI Developer

108,815 subscribers

45,677 просмотров • 8 месяцев назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

"Bring me the healthiest snack." The robot goes to the kitchen and gets the snack, fully autonomously. This is the first public demo of NVIDIA's Isaac GR00T N1.6 foundation model presented by Yuke Zhu at CoRL 2025. The previous versions focused only on bimanual stationary manipulation; the N1.6 unlocks the entire kinematic range of the robot. The latest release of the open Isaac GR00T N1.6 VLA model will be available soon on Hugging Face. It integrates with NVIDIA Cosmos Reason, an open, customizable reasoning vision language model that turns vague instructions into step-by-step plans.
2:29

"Bring me the healthiest snack." The robot goes to the kitchen and gets the snack, fully autonomously. This is the first public demo of NVIDIA's Isaac GR00T N1.6 foundation model presented by Yuke Zhu at CoRL 2025. The previous versions focused only on bimanual stationary manipulation; the N1.6 unlocks the entire kinematic range of the robot. The latest release of the open Isaac GR00T N1.6 VLA model will be available soon on Hugging Face. It integrates with NVIDIA Cosmos Reason, an open, customizable reasoning vision language model that turns vague instructions into step-by-step plans.

The Humanoid Hub

42,823 просмотров • 1 год назад

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️
0:38

Cosmos Policy just dropped for robotics. 🤖 Cutting edge research is turning a world foundation model into a unified robot brain that can see, predict, and act—no extra action heads, no complicated control stack. Read our blog on Hugging Face ➡️ Want to get hands-on with Cosmos (Reason, Predict, Policy, Cookbook)? Join the Cosmos Cookoff, sponsored by Nebius and Milestone Systems ➡️

NVIDIA Robotics

14,404 просмотров • 7 месяцев назад

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.
0:22

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

NVIDIA AI

211,856 просмотров • 2 месяцев назад

Explore where physical AI is headed next with the people shaping it. On Feb. 10 at 9 a.m. PT, join Ming‑Yu, Vice President of Research at NVIDIA, and Chubby♨️ from Superintelligence, for a conversation on how NVIDIA Cosmos reframes AI for a world of robots, AVs, and vision-driven systems. We'll dig into: ✅ Cosmos' origins and designing models for the physical world ✅ The scientific ideas and architectures behind Cosmos and its open models ✅ How open, reasoning-capable physical AI is changing how teams prototype, deploy, and iterate ✅ What a future of AI systems that understand space, time, and causality could unlock If you’re thinking into real‑world AI, this is the conversation to catch ➡️
0:39

Explore where physical AI is headed next with the people shaping it. On Feb. 10 at 9 a.m. PT, join Ming‑Yu, Vice President of Research at NVIDIA, and Chubby♨️ from Superintelligence, for a conversation on how NVIDIA Cosmos reframes AI for a world of robots, AVs, and vision-driven systems. We'll dig into: ✅ Cosmos' origins and designing models for the physical world ✅ The scientific ideas and architectures behind Cosmos and its open models ✅ How open, reasoning-capable physical AI is changing how teams prototype, deploy, and iterate ✅ What a future of AI systems that understand space, time, and causality could unlock If you’re thinking into real‑world AI, this is the conversation to catch ➡️

NVIDIA AI Developer

26,468 просмотров • 7 месяцев назад

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖
0:33

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯 Why is Physical AI much harder than building a chatbot? Understanding the world is not enough, robots need to predict it and act inside it. That's the idea behind NVIDIA Cosmos 3: → Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal. Previous systems often stitched these capabilities together using separate models. Cosmos 3 combines them into a single architecture with two components: ▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context. ▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints. So Cosmos 3 moves from: Perception → Model A Prediction → Model B Actions → Model C to: Perception + Prediction + Actions → One unified system. The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions: 1. What is happening? 2. What will happen next? 3. What should I do? That's a big shift from today's AI, which mostly focuses on generating text. And check the benchmarks! Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

Turing Post

13,836 просмотров • 3 месяцев назад

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.
2:30

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

45,595 просмотров • 1 месяц назад

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:
0:54

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:

Tencent Hy

34,317 просмотров • 5 месяцев назад

✨ Meet our new open family of models: NVIDIA Nemotron 3 Open in weights, data, tools, and training, Nemotron 3 is built for multi-agent apps and features: • An efficient hybrid Mamba‑Transformer MoE architecture • 1M token context for long-term memory and improved reasoning • Multi‑environment reinforcement learning via NeMo Gym for advanced skill adaptation Plus NVFP4 pre-training, latent MoE, 1T tokens of data, and more. 📗Read the details in our tech blog: 🤗 Try the model on Hugging Face:
1:35

✨ Meet our new open family of models: NVIDIA Nemotron 3 Open in weights, data, tools, and training, Nemotron 3 is built for multi-agent apps and features: • An efficient hybrid Mamba‑Transformer MoE architecture • 1M token context for long-term memory and improved reasoning • Multi‑environment reinforcement learning via NeMo Gym for advanced skill adaptation Plus NVFP4 pre-training, latent MoE, 1T tokens of data, and more. 📗Read the details in our tech blog: 🤗 Try the model on Hugging Face:

NVIDIA AI Developer

69,008 просмотров • 9 месяцев назад

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️
2:10

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,490 просмотров • 1 год назад

Physics AI 🌊 Jensen just unveiled the NVIDIA Isaac GR00T Reference Humanoid Robot at GTC Taipei--the world’s first open-source humanoid robot reference design Built on Jetson Thor edge computing and the Isaac GR00T open platform, it features a Unitree H2 Plus body with Sharpa five-finger dexterous hands and 75 degrees of freedom. Even before Physical AI truly explodes, NVIDIA has built a complete end-to-end full-stack infrastructure: >chips (Jetson Thor + Blackwell), >synthetic data (GR00T-Dreams), >world models (Cosmos), >foundation models (GR00T N-series VLA), >high-fidelity simulation platforms (Isaac Sim + Isaac Lab + Omniverse). Dozens of humanoid robot companies ,including Agility Robotics, Aglie robotics,Boston Dynamics, Figure, 1X, NEURA Robotics, XPENG Robotics, and more are already running large-scale daily simulation training on this platform, rapidly iterating perception, reasoning, and actions in virtual environments, speeding up the jump from lab prototypes to real factory deployment. By providing an open reference design and full-stack tools, NVIDIA has dramatically lowered the barrier, letting every robotics team join the Physical AI wave with low cost and high efficiency.
4:31

Physics AI 🌊 Jensen just unveiled the NVIDIA Isaac GR00T Reference Humanoid Robot at GTC Taipei--the world’s first open-source humanoid robot reference design Built on Jetson Thor edge computing and the Isaac GR00T open platform, it features a Unitree H2 Plus body with Sharpa five-finger dexterous hands and 75 degrees of freedom. Even before Physical AI truly explodes, NVIDIA has built a complete end-to-end full-stack infrastructure: >chips (Jetson Thor + Blackwell), >synthetic data (GR00T-Dreams), >world models (Cosmos), >foundation models (GR00T N-series VLA), >high-fidelity simulation platforms (Isaac Sim + Isaac Lab + Omniverse). Dozens of humanoid robot companies ,including Agility Robotics, Aglie robotics,Boston Dynamics, Figure, 1X, NEURA Robotics, XPENG Robotics, and more are already running large-scale daily simulation training on this platform, rapidly iterating perception, reasoning, and actions in virtual environments, speeding up the jump from lab prototypes to real factory deployment. By providing an open reference design and full-stack tools, NVIDIA has dramatically lowered the barrier, letting every robotics team join the Physical AI wave with low cost and high efficiency.

CyberRobo

17,092 просмотров • 3 месяцев назад

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:
1:24

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:

NVIDIA AI Developer

138,518 просмотров • 8 месяцев назад

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:
0:44

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,079,976 просмотров • 3 месяцев назад

Come Into The Future With Us - GamerHash AI ChatBot Open Beta Is Now Officially Live! GamerHash AI is proud to present our latest release - a highly efficient AI ChatBot powered by NVIDIA. We’ve been working tirelessly to deliver a comprehensive and high quality #DePIN-powered product for our network of gamers. Here are some details about it: ✅ Lightning Fast - featuring the efficient AI at Meta Llama3 (8B) model! ✅ 30+ Languages Operability - with seamless transitioning! ✅ Private & Local - we take decentralization and privacy very seriously! ✅ No Censorship - we are dedicated to high transparency of our products! ✅ Easily Expandable Future - our future roadmap includes Personas, multi-modality & custom models functionality! Wait no longer & download our GamerHash AI App to start using the newest #AI module! 👇
0:34

Come Into The Future With Us - GamerHash AI ChatBot Open Beta Is Now Officially Live! GamerHash AI is proud to present our latest release - a highly efficient AI ChatBot powered by NVIDIA. We’ve been working tirelessly to deliver a comprehensive and high quality #DePIN-powered product for our network of gamers. Here are some details about it: ✅ Lightning Fast - featuring the efficient AI at Meta Llama3 (8B) model! ✅ 30+ Languages Operability - with seamless transitioning! ✅ Private & Local - we take decentralization and privacy very seriously! ✅ No Censorship - we are dedicated to high transparency of our products! ✅ Easily Expandable Future - our future roadmap includes Personas, multi-modality & custom models functionality! Wait no longer & download our GamerHash AI App to start using the newest #AI module! 👇

GamerHash AI

32,479 просмотров • 2 лет назад

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )
0:44

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )

Omer Shlomovits

36,648 просмотров • 5 месяцев назад

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️
2:03

Our vision is for AI that uses world models to adapt in new and dynamic environments and efficiently learn new skills. We’re sharing V-JEPA 2, a new world model with state-of-the-art performance in visual understanding and prediction. V-JEPA 2 is a 1.2 billion-parameter model, trained on video, that can enable zero-shot planning in robots—allowing them to plan and execute tasks in unfamiliar environments. Learn more about V-JEPA 2 ➡️ As we continue working toward our goal of achieving advanced machine intelligence (AMI), we’re also releasing three new benchmarks for evaluating how well existing models can reason about the physical world from video. Learn more and download the new benchmarks ➡️

AI at Meta

310,676 просмотров • 1 год назад

GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.
0:31

GR00T 1.7 just landed in LeRobot 🟢 We upgraded our NVIDIA Robotics GR00T integration to 1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. It swaps the previous VLM for Cosmos-Reason2-2B feeding a flow-matching action head, and we parity-tested it against NVIDIA's own Isaac GR00T implementation: same inputs, same outputs. Flash-attention is now optional too, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly. Isaac Teleop is now in the ecosystem too, letting you drive an SO-101 with a VR controller over CloudXR/OpenXR.

LeRobot

19,572 просмотров • 2 месяцев назад

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵
2:21

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

467,237 просмотров • 1 год назад