Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Simulation drives robotics progress, but how do we close the reality gap? Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes! Thread 🧵

alescontrela's profile picture

Alejandro Escontrela

1,392 subscribers

108,029 Aufrufe • vor 7 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality.
1:07

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality.

NVIDIA GeForce UK

2,469,798 Aufrufe • vor 3 Monaten

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality. Learn More →
1:10

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality. Learn More →

NVIDIA GeForce

95,192,233 Aufrufe • vor 3 Monaten

🔦 NVIDIA DLSS 5 Spotlight 🔦 NVIDIA DLSS 5 is coming to Resident Evil Requiem. DLSS 5 infuses pixels with photorealistic lighting and materials to bridge the gap between rendering and reality in Raccoon City.
0:23

🔦 NVIDIA DLSS 5 Spotlight 🔦 NVIDIA DLSS 5 is coming to Resident Evil Requiem. DLSS 5 infuses pixels with photorealistic lighting and materials to bridge the gap between rendering and reality in Raccoon City.

NVIDIA GeForce UK

69,065 Aufrufe • vor 3 Monaten

🔦 NVIDIA DLSS 5 Spotlight 🔦 NVIDIA DLSS 5 is coming to Resident Evil Requiem. DLSS 5 infuses pixels with photorealistic lighting and materials to bridge the gap between rendering and reality in Raccoon City. Learn more →
0:25

🔦 NVIDIA DLSS 5 Spotlight 🔦 NVIDIA DLSS 5 is coming to Resident Evil Requiem. DLSS 5 infuses pixels with photorealistic lighting and materials to bridge the gap between rendering and reality in Raccoon City. Learn more →

NVIDIA GeForce

1,528,242 Aufrufe • vor 3 Monaten

NVIDIA today unveiled DLSS 5, which they describe it as their most significant breakthrough in computer graphics since 2018. Using AI, "DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality."
1:04

NVIDIA today unveiled DLSS 5, which they describe it as their most significant breakthrough in computer graphics since 2018. Using AI, "DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality."

Nic Cruz Patane

13,998 Aufrufe • vor 3 Monaten

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵
1:31

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵

Max Fu

169,218 Aufrufe • vor 2 Monaten

Introducing CerebrasCoder! An open-source app that generates websites with Llama3.3-70b from as fast as you can type. 100% free and open-source.
2:08

Introducing CerebrasCoder! An open-source app that generates websites with Llama3.3-70b from as fast as you can type. 100% free and open-source.

Steve Krouse

240,119 Aufrufe • vor 1 Jahr

Introducing the Modern Fiberoptic Drone Framework (MFDF), an open-source next-gen fiber optic framework for tactical drones. Designed with MFDF is the K9 Fiber C2 System, an open-source NDAA-compliant C2 & video solution for the Genesis tactical drone family 🇺🇸(1/4).
0:41

Introducing the Modern Fiberoptic Drone Framework (MFDF), an open-source next-gen fiber optic framework for tactical drones. Designed with MFDF is the K9 Fiber C2 System, an open-source NDAA-compliant C2 & video solution for the Genesis tactical drone family 🇺🇸(1/4).

Canine Defense Technologies

70,200 Aufrufe • vor 6 Monaten

Phones can talk, but mine DRIVES MY CAR with an #openpilot fork 🤯 openpilot is an open-source, semi-automated driving software by comma Repost this and I’ll drop the how-to in a thread.
0:37

Phones can talk, but mine DRIVES MY CAR with an #openpilot fork 🤯 openpilot is an open-source, semi-automated driving software by comma Repost this and I’ll drop the how-to in a thread.

Andrei Radulescu

23,942 Aufrufe • vor 1 Jahr

Tired of waiting hours for humanoids to learn to walk? Our new technical report shows how to train sim-to-real humanoid locomotion in 15 minutes with FastSAC and FastTD3! The full pipeline is open-source in the newly released Holosoma codebase. Thread 🧵
0:22

Tired of waiting hours for humanoids to learn to walk? Our new technical report shows how to train sim-to-real humanoid locomotion in 15 minutes with FastSAC and FastTD3! The full pipeline is open-source in the newly released Holosoma codebase. Thread 🧵

Younggyo Seo

36,408 Aufrufe • vor 6 Monaten

How I discovered and exploited an Unauthenticated RCE in BYOB (Build Your Own Botnet), an open-source post-exploitation framework for students, researchers and developers with close to 9k stars on GitHub!
0:37

How I discovered and exploited an Unauthenticated RCE in BYOB (Build Your Own Botnet), an open-source post-exploitation framework for students, researchers and developers with close to 9k stars on GitHub!

chebuya

65,143 Aufrufe • vor 1 Jahr

🧵 Evaluating robot policies in the real world is slow, expensive, and hard to scale. During my internship at SceniX AI this summer, we had many discussions around the two key questions: how accurate must a simulator be for evaluation to be meaningful, and how do we get there? Our new framework, Real2Sim-Eval, takes a step toward that answer. By combining Gaussian Splatting for photorealistic rendering and soft-body digital twins for realistic dynamics, we make simulation predictive of real-world performance. 👉
1:26

🧵 Evaluating robot policies in the real world is slow, expensive, and hard to scale. During my internship at SceniX AI this summer, we had many discussions around the two key questions: how accurate must a simulator be for evaluation to be meaningful, and how do we get there? Our new framework, Real2Sim-Eval, takes a step toward that answer. By combining Gaussian Splatting for photorealistic rendering and soft-body digital twins for realistic dynamics, we make simulation predictive of real-world performance. 👉

Kaifeng Zhang

60,566 Aufrufe • vor 7 Monaten

Generalist robots need a generalist evaluator. But how do you test safety without breaking things? 💥 🌎 Introducing our new work from Google DeepMind: Evaluating Gemini Robotics Policies in a Veo World Simulator 🧵👇
0:48

Generalist robots need a generalist evaluator. But how do you test safety without breaking things? 💥 🌎 Introducing our new work from Google DeepMind: Evaluating Gemini Robotics Policies in a Veo World Simulator 🧵👇

Anirudha Majumdar

237,317 Aufrufe • vor 6 Monaten

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at
0:34

Excited to announce RoboCasa, a large-scale simulation framework of everyday tasks! We use generative AI tools to create diverse objects, scenes, and tasks. Simulation plays a pivotal role in our Data Pyramid for training generalist robots. Open-source at

Yuke Zhu

141,478 Aufrufe • vor 2 Jahren

Introducing Omni Octane. Octane is an open source, performance-optimized EVM framework launching with integrations from Celestia, Arbitrum, and . Octane’s unique design opens the door for new high performance L1 architectures and decentralized sequencers for L2s.
0:34

Introducing Omni Octane. Octane is an open source, performance-optimized EVM framework launching with integrations from Celestia, Arbitrum, and . Octane’s unique design opens the door for new high performance L1 architectures and decentralized sequencers for L2s.

Nomina

299,759 Aufrufe • vor 2 Jahren

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion Robotics to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla
1:06:07

Today, we're joined by Nikita Rudin, co-founder and CEO of Flexion Robotics to discuss the gap between current robotic capabilities and what’s required to deploy fully autonomous robots in the real world. Nikita explains how reinforcement learning and simulation have driven rapid progress in robot locomotion—and why locomotion is still far from “solved.” We dig into the sim2real gap, and how adding visual inputs introduces noise and significantly complicates sim-to-real transfer. We also explore the debate between end-to-end models and modular approaches, and why separating locomotion, planning, and semantics remains a pragmatic approach today. Nikita also introduces the concept of "real-to-sim", which uses real-world data to refine simulation parameters for higher fidelity training, discusses how reinforcement learning, imitation learning, and teleoperation data are combined to train robust policies for both quadruped and humanoid robots, and introduces Flexion's hierarchical approach that utilizes pre-trained Vision-Language Models (VLMs) for high-level task orchestration with Vision-Language-Action (VLA) models and low-level whole-body trackers. Finally, Nikita shares the behind-the-scenes in humanoid robot demos, his take on reinforcement learning in simulation versus the real world, the nuances of reward tuning, and offers practical advice for researchers and practitioners looking to get started in robotics today. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 04:07 - Is robot locomotion solved? 06:04 - Sim-to-real gap 08:58 - Adding semantics to policies 09:42 - Modular vs end-to-end architectures 10:29 - Planner model 12:21 - Adapting RL techniques from quadrupeds to humanoids 15:39 - Behind robot demos 18:09 - Humanoid robots in home environments 22:03 - Training approach 23:56 - VLA models 27:59 - Closing the sim-to-real gap 32:55 - Task orchestration using VLMs 36:38 - Tool use 38:10 - Model hierarchy 43:37 - Simulator versus simulation environment 44:57 - Combining imitation learning and reinforcement learning 46:42 - RL in real world versus RL in simulation 52:58 - Reward tuning and value functions in robotics 56:38 - Predictions 1:00:10 - Humanoids, quadropeds, and wheeled platforms 1:02:45 - Advice, recommended robot kits, and community pla

The TWIML AI Podcast

22,264 Aufrufe • vor 5 Monaten

Introducing Muscle v0 -- infinite degrees of freedom, from Daxo Robotics. A different mountain to climb - with a far more beautiful peak. We built this from the ground up: - Ultra-dexterous - Built for machine learning - Durable and robust More below (1/n)
1:37

Introducing Muscle v0 -- infinite degrees of freedom, from Daxo Robotics. A different mountain to climb - with a far more beautiful peak. We built this from the ground up: - Ultra-dexterous - Built for machine learning - Durable and robust More below (1/n)

Tom Zhang

273,359 Aufrufe • vor 11 Monaten

The missing piece for the ChatGPT moment of robotics is finally here Today, we are releasing phosphobot, an open-source app to control robots intuitively and train the most powerful AI models for the cheapest robots possible -> phospho-app/phosphobot 🧵 What we are building:
1:24

The missing piece for the ChatGPT moment of robotics is finally here Today, we are releasing phosphobot, an open-source app to control robots intuitively and train the most powerful AI models for the cheapest robots possible -> phospho-app/phosphobot 🧵 What we are building:

Pierre-Louis Biojout (PLB)

35,828 Aufrufe • vor 1 Jahr

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:
2:21

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,182 Aufrufe • vor 1 Jahr