正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

An interesting work on Physical AI: PhysX-Omni. First unified sim-ready generation framework for rigid, deformable, and articulated objects, with a diverse dataset and new benchmark. 🌐 💻 📦

Ziqi Huang

3,022 subscribers

27,240 次观看 • 4 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

5 条评论

EB1A Experts 的头像

EB1A Experts4 个月前

Very interesting direction for simulation-ready physical AI systems.

Yash 的头像

Yash4 个月前

It's very useful for the robotics I believe

Saïd Aitmbarek 的头像

Saïd Aitmbarek4 个月前

fascinating stuff

Elodie 3DFontaine 的头像

Elodie 3DFontaine4 个月前

Sim-ready is good, but I need to inspect the geometry. Pretty meshes are fine, but let me see CAD-level topology, not renderings dressed for physics.

Rhy 的头像

Rhy4 个月前

love the concept of sim ready 3Dmodels - whats the 3D gen model built on top of?

相关视频

Together with the Ego4D consortium, we recently released Ego-Exo4D: A diverse, large-scale multi-modal, multi-view, video dataset and benchmark. Learn more about the work ➡️ Access the dataset ➡️ This work could help to advance AI models' understanding of complex human skills & enable new applications for AR systems, robotics & more.

Together with the Ego4D consortium, we recently released Ego-Exo4D: A diverse, large-scale multi-modal, multi-view, video dataset and benchmark. Learn more about the work ➡️ Access the dataset ➡️ This work could help to advance AI models' understanding of complex human skills & enable new applications for AR systems, robotics & more.

AI at Meta

78,091 次观看 • 2 年前

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

Ant Ling

21,365 次观看 • 1 年前

LiteRT is here. The universal framework for on-device AI. 📱💻 ✨ 1.4x faster GPU performance ✨ Unified NPU acceleration ✨ Seamless PyTorch and JAX support ✨ Ready for Gemma and gen AI Build faster, simpler, and everywhere. Read the blog:

LiteRT is here. The universal framework for on-device AI. 📱💻 ✨ 1.4x faster GPU performance ✨ Unified NPU acceleration ✨ Seamless PyTorch and JAX support ✨ Ready for Gemma and gen AI Build faster, simpler, and everywhere. Read the blog:

Google for Developers

194,484 次观看 • 8 个月前

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

📢 Announcing one of the most exciting works from us this year on scalable robot policy evaluation through real-to-sim transfer, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is orders of magnitude slower than the development of language agents. We propose a new framework where simulation performance strongly correlates with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing state-of-the-art correlation between simulation and reality for deformable object manipulation. It provides a scalable and reproducible evaluation engine for robot learning. 🌐

Yunzhu Li

39,960 次观看 • 11 个月前

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,425 次观看 • 1 年前

The latest update from NemoVideo sets a new benchmark for AI-powered video creation. With NemoVideo now integrated with Seedance 2, the workflow is seamless—structured scripting, refined visual generation, and production-ready output within minutes. A meaningful advancement for creators focused on efficiency and quality. Explore it here: #NemoVideo #Seedance2

The latest update from NemoVideo sets a new benchmark for AI-powered video creation. With NemoVideo now integrated with Seedance 2, the workflow is seamless—structured scripting, refined visual generation, and production-ready output within minutes. A meaningful advancement for creators focused on efficiency and quality. Explore it here: #NemoVideo #Seedance2

Aryan Rakib

250,353 次观看 • 7 个月前

Robotics needs data. OVER delivers it. Robots aren’t just machines anymore, they’re becoming autonomous agents. The secret? A new generation of Physical AI Models. And the fuel? A global dataset built for scale. Let’s dive in 👇🧵

Robotics needs data. OVER delivers it. Robots aren’t just machines anymore, they’re becoming autonomous agents. The secret? A new generation of Physical AI Models. And the fuel? A global dataset built for scale. Let’s dive in 👇🧵

Over the Reality 🌐

24,985 次观看 • 1 年前

🚀 Struggling with the lack of high-quality data for AI-driven human-object interaction research? We've got you covered! Introducing HUMOTO, a groundbreaking 4D dataset for human-object interaction, developed with a combination of wearable motion capture, SOTA 6D pose estimation vision models, LLM, and the professional refining works of multiple animation studios. HUMOTO features: ✅ Over 700 diverse daily activities ✅ Interactions with 60+ objects, 70+ articulated parts. ✅ Fine-grained text annotations ✅ Detailed hand and finger movements We hope HUMOTO will fuel your Humanoid AI research and drive new advancements! For research or commercial license inquiries, please contact yizho@adobe.com. Explore the dataset: 👉 HUMOTO Dataset Website Learn more: 👉 HUMOTO Project Page Jiaxin Lu Qixing Huang #AI #HumanObjectInteraction #HumanoidAI #MotionCapture #HUMOTO #AdobeResearch

🚀 Struggling with the lack of high-quality data for AI-driven human-object interaction research? We've got you covered! Introducing HUMOTO, a groundbreaking 4D dataset for human-object interaction, developed with a combination of wearable motion capture, SOTA 6D pose estimation vision models, LLM, and the professional refining works of multiple animation studios. HUMOTO features: ✅ Over 700 diverse daily activities ✅ Interactions with 60+ objects, 70+ articulated parts. ✅ Fine-grained text annotations ✅ Detailed hand and finger movements We hope HUMOTO will fuel your Humanoid AI research and drive new advancements! For research or commercial license inquiries, please contact [email protected]. Explore the dataset: 👉 HUMOTO Dataset Website Learn more: 👉 HUMOTO Project Page Jiaxin Lu Qixing Huang #AI #HumanObjectInteraction #HumanoidAI #MotionCapture #HUMOTO #AdobeResearch

Yi Zhou

71,609 次观看 • 1 年前

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵

Max Fu

180,628 次观看 • 6 个月前

Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt. VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties. We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation. Website: 👇🧵(1/8)

Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt. VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties. We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation. Website: 👇🧵(1/8)

Nicholas Pfaff

90,335 次观看 • 8 个月前

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Zhiyang (Frank) Dou

14,704 次观看 • 1 年前

Onto Ultiverse's Next Chapter: On-Chain Procedural Content Generation (PCG)📢 We are entering a new phase with our AI On-Chain PCG framework, transitioning from a gaming platform to an infrastructure provider and ecosystem catalyst for the entire Web3 domain. Read more below🧵

Onto Ultiverse's Next Chapter: On-Chain Procedural Content Generation (PCG)📢 We are entering a new phase with our AI On-Chain PCG framework, transitioning from a gaming platform to an infrastructure provider and ecosystem catalyst for the entire Web3 domain. Read more below🧵

Ultiverse

141,565 次观看 • 2 年前

Yesterday we announced we are joining forces with Stereolabs to accelerate the development of a unified sensing and perception platform for Physical AI We're combining high-resolution digital lidar with stereo cameras and AI compute to offer the first end-to-end sensing and perception stack, enabling machines to sense, think, act, and learn faster and with more precision To the autonomy engineers building the next-generation on Physical AI: we're here to accelerate your development from prototype to production. Welcome to the next chapter of Ouster 🤖

Yesterday we announced we are joining forces with Stereolabs to accelerate the development of a unified sensing and perception platform for Physical AI We're combining high-resolution digital lidar with stereo cameras and AI compute to offer the first end-to-end sensing and perception stack, enabling machines to sense, think, act, and learn faster and with more precision To the autonomy engineers building the next-generation on Physical AI: we're here to accelerate your development from prototype to production. Welcome to the next chapter of Ouster 🤖

Ouster

14,291 次观看 • 8 个月前

China is set to host the first humanoid robot arena competition in late May, focusing on direct physical confrontation between full-size bipedal robots. Participants will demonstrate real-time control, stability, and force capability under high-impact conditions—marking a new benchmark for physical AI in open competition.

China is set to host the first humanoid robot arena competition in late May, focusing on direct physical confrontation between full-size bipedal robots. Participants will demonstrate real-time control, stability, and force capability under high-impact conditions—marking a new benchmark for physical AI in open competition.

Reborn

93,132 次观看 • 1 年前

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,152 次观看 • 14 天前

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,200 次观看 • 1 年前

AI Guidance for Physical Work David Lieb Much of the AI conversation focuses on which desk jobs will get replaced. But for physical work, AI can't yet act in the world. With multimodal models and ubiquitous hardware, there’s an opportunity to give workers live AI coaching that dramatically reduces training time and unlocks skilled labor at scale.

AI Guidance for Physical Work David Lieb Much of the AI conversation focuses on which desk jobs will get replaced. But for physical work, AI can't yet act in the world. With multimodal models and ubiquitous hardware, there’s an opportunity to give workers live AI coaching that dramatically reduces training time and unlocks skilled labor at scale.

Y Combinator

26,533 次观看 • 8 个月前

You told us you’re running multiple AI agents and wanted a better UX. We listened and shipped it! Here’s what’s new in the latest Visual Studio Code release: 🗂️ Unified agent sessions workspace for local, background, and cloud agents 💻 Claude and Codex support for local and cloud agents 🔀 Parallel subagents 🌐 Integrated browser And more...

You told us you’re running multiple AI agents and wanted a better UX. We listened and shipped it! Here’s what’s new in the latest Visual Studio Code release: 🗂️ Unified agent sessions workspace for local, background, and cloud agents 💻 Claude and Codex support for local and cloud agents 🔀 Parallel subagents 🌐 Integrated browser And more...

Visual Studio Code

589,052 次观看 • 8 个月前

These new hands are the perfect companion for Atlas. With 13 degrees of freedom, they are directly actuated, built for high fidelity simulation to enable sim-to-real RL, and designed from the ground up for dexterous physical work. Learn how we built robot hands for modern AI and real work:

These new hands are the perfect companion for Atlas. With 13 degrees of freedom, they are directly actuated, built for high fidelity simulation to enable sim-to-real RL, and designed from the ground up for dexterous physical work. Learn how we built robot hands for modern AI and real work:

Boston Dynamics

366,008 次观看 • 8 天前