正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🚦 Excited introducing Urban-Sim — our new simulator presented at #CVPR2025 as a highlight paper! ⚡️ Fast training with IsaacSim backend 🏙️ Diverse 3D assets for rich urban scenes 🤖 Towards generalizable robots in dynamic urban environments. Webpage:

Bolei Zhou

9,179 subscribers

18,280 次观看 • 1 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 条评论

Arrogant Bill 的头像

Arrogant Bill1 年前

Really impressed by Urban-Sim! Fast, diverse simulations like this will push next-gen robotics and real-world applications further. Looking forward to seeing what emerges from this platform.

Page to Pixel Publishing 的头像

Page to Pixel Publishing2 年前

The Art of Flight is a homage to 80s/90s arcade action shmups with a fresh twist on the genre. Pilot multiple ships at the same time to take on oncoming waves of enemies in this fast paced space shooter. Wishlist on Steam today!

Daniel Ortega 的头像

Daniel Ortega1 年前

Esto sí es dar un salto real: simuladores que acercan la robótica urbana a la calle, no solo a pruebas académicas. Felicidades, así se innova de verdad.

相关视频

We release a new urban simulator, MetaUrban, to support research on AI agents for micromobility. The work will be presented at #ICLR2025, and the demo code can run on any laptop. Webpage: Code: Paper:

We release a new urban simulator, MetaUrban, to support research on AI agents for micromobility. The work will be presented at #ICLR2025, and the demo code can run on any laptop. Webpage: Code: Paper:

Bolei Zhou

30,344 次观看 • 1 年前

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

Chuang Gan

90,599 次观看 • 1 年前

Introducing 6STR Underground; a new standalone urban street racing experience built for FiveM, in collaboration with the team behind Arena. Jump in for casual street races, host meets in free roam, climb the ranks in competitive modes, chase down racers as law enforcement, or immerse yourself in a cinematic storytelling experience. Find out how to get involved in our first closed beta:

Introducing 6STR Underground; a new standalone urban street racing experience built for FiveM, in collaboration with the team behind Arena. Jump in for casual street races, host meets in free roam, climb the ranks in competitive modes, chase down racers as law enforcement, or immerse yourself in a cinematic storytelling experience. Find out how to get involved in our first closed beta:

WiseGuy

77,859 次观看 • 2 个月前

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

We have seen a lot of legged robots doing navigation in the wild. But how about mobile manipulation in the wild? I have been pushing the direction of learning a unified, efficient, and dynamic 3D representation of scenes (for navigation) and objects (for manipulation) for the past two years. And now we have GeFF --- our large-scale, generalizable feature field, that combines the speed of a feed-forward neural network with the rich semantics from Foundation Models, to handle dynamically changing scenes, and enable open-ended, language-grounded scene and object understanding.

Xiaolong Wang

43,093 次观看 • 2 年前

Pilot readiness isn't often talked about, but it's one of the most critical paths to commercial service. This timelapse shows a key piece of that work coming together: our CAE-built full-motion simulator, installed at Joby's flight test facility. Designed as a digital twin of our aircraft, these high-fidelity simulators provide a means of compliance and a robust tool to help prepare our pilots for the demands of high-volume operations in urban environments. From crated components to a fully assembled rig, and soon, the environment where Joby pilots will train for electric air taxi operations.

Pilot readiness isn't often talked about, but it's one of the most critical paths to commercial service. This timelapse shows a key piece of that work coming together: our CAE-built full-motion simulator, installed at Joby's flight test facility. Designed as a digital twin of our aircraft, these high-fidelity simulators provide a means of compliance and a robust tool to help prepare our pilots for the demands of high-volume operations in urban environments. From crated components to a fully assembled rig, and soon, the environment where Joby pilots will train for electric air taxi operations.

Joby Aviation

24,628 次观看 • 4 个月前

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇

🚨 New Paper Hybrid Neural World Models. We build neural world models for discontinuous environments and our method discovers when they're making error, with no ground truth. Our neural models run physics fast, leaping many simulator steps in a single pass. How we find points where not to trust it: ask the model for the same future two ways, and the disagreement between the two answers marks where it's failing without needing labels. The same recipe holds on three systems with nothing in common: a chemical wave, compressible gas with shocks, rigid bodies colliding in 3D. 👇

Lossfunk

15,171 次观看 • 3 个月前

Autonomous driving through tight, dynamic, stochastic, and adversarial traffic-dynamics on sub-urban roads in India, as well as through partially unstructured environments. This demos showcases the robustness of our motion planning and decision making algorithmic frameworks in enabling #autonomousdriving through seamlessly through such traffic and environmental scenarios. The vehicle starts from a generic open environment at the temple, where there are no traffic-rules to abide by. It then exits the region and assumes a generic autonomous navigation behaviour, negotiating complex traffic scenes. At various points it can be seen that the other vehicles (bikes, autos, bicyclists, and cars) didn't abide by any traffic-rule and moved in crisscross fashion, presenting adversarial scenarios, challenging our autonomous vehicle at Swaayatt Robots to take care of the collision avoidance. This classical motion planning and decision making algorithmic framework is being further scaled up with deep #reinforcementlearning, which will practically solve the sub-urban traffic-dynamics and environment negotiation for #autonomousvehicles in India and throughout the world as well. This demo was done at the Kankali Kali Mata mandir in the city of Bhopal. This demo was a culmination of our prior works and demos: off-roads, on-roads, bidirectional traffic negotiation in single lane roads, and toll-plaza navigation. We have taken up the arduous task of solving the Level-4 autonomous driving by the end of 2024, globally. #machinelearning #deeplearning

Autonomous driving through tight, dynamic, stochastic, and adversarial traffic-dynamics on sub-urban roads in India, as well as through partially unstructured environments. This demos showcases the robustness of our motion planning and decision making algorithmic frameworks in enabling #autonomousdriving through seamlessly through such traffic and environmental scenarios. The vehicle starts from a generic open environment at the temple, where there are no traffic-rules to abide by. It then exits the region and assumes a generic autonomous navigation behaviour, negotiating complex traffic scenes. At various points it can be seen that the other vehicles (bikes, autos, bicyclists, and cars) didn't abide by any traffic-rule and moved in crisscross fashion, presenting adversarial scenarios, challenging our autonomous vehicle at Swaayatt Robots to take care of the collision avoidance. This classical motion planning and decision making algorithmic framework is being further scaled up with deep #reinforcementlearning, which will practically solve the sub-urban traffic-dynamics and environment negotiation for #autonomousvehicles in India and throughout the world as well. This demo was done at the Kankali Kali Mata mandir in the city of Bhopal. This demo was a culmination of our prior works and demos: off-roads, on-roads, bidirectional traffic negotiation in single lane roads, and toll-plaza navigation. We have taken up the arduous task of solving the Level-4 autonomous driving by the end of 2024, globally. #machinelearning #deeplearning

Sanjeev Sharma

186,291 次观看 • 2 年前

We are excited to share our #CORL2024 paper (oral) on "Learning Quadruped Locomotion Using Differentiable Simulation" done in collaboration with Sangbae Kim Massachusetts Institute of Technology (MIT). We present a new way to learn to walk in minutes without parallelization, outperforming PPO in sample efficiency! PDF: Video: We present a new framework for learning quadruped locomotion. By leveraging differentiable simulation for policy optimization, our approach achieves fast convergence and stable training, significantly outperforming model-free #ReinforcementLearning methods like PPO in sample efficiency. The key enabler is to combine a high-fidelity, non-differentiable simulator for forward dynamics with a simplified surrogate model for gradient backpropagation. Our framework enables learning quadruped walking in simulation in minutes without parallelization. When augmented with GPU parallelization, our approach allows the quadruped robot to master diverse locomotion skills on challenging terrains in minutes. This work highlights one of the first successful real-world applications of differentiable simulation for quadruped robots, offering a compelling alternative to traditional RL methods. Kudos to Yunlong Song! UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) Massachusetts Institute of Technology (MIT)MechE

We are excited to share our #CORL2024 paper (oral) on "Learning Quadruped Locomotion Using Differentiable Simulation" done in collaboration with Sangbae Kim Massachusetts Institute of Technology (MIT). We present a new way to learn to walk in minutes without parallelization, outperforming PPO in sample efficiency! PDF: Video: We present a new framework for learning quadruped locomotion. By leveraging differentiable simulation for policy optimization, our approach achieves fast convergence and stable training, significantly outperforming model-free #ReinforcementLearning methods like PPO in sample efficiency. The key enabler is to combine a high-fidelity, non-differentiable simulator for forward dynamics with a simplified surrogate model for gradient backpropagation. Our framework enables learning quadruped walking in simulation in minutes without parallelization. When augmented with GPU parallelization, our approach allows the quadruped robot to master diverse locomotion skills on challenging terrains in minutes. This work highlights one of the first successful real-world applications of differentiable simulation for quadruped robots, offering a compelling alternative to traditional RL methods. Kudos to Yunlong Song! UZH Science University of Zurich UZH Space Hub UZH IfI European Research Council (ERC) Massachusetts Institute of Technology (MIT)MechE

Davide Scaramuzza

15,554 次观看 • 1 年前

Kled now owns exclusive rights for one of the largest aerial drone footage catalogs in the United States. A multi million dollar catalog consisting of: -750,000,000+ unique aerial frames -5,000+ hours of real world flight -100,000+ distinct aerial scenes & environments -Nationwide spatial coverage (urban, suburban, rural, coastal) -High altitude + low altitude perspectives -Continuous motion sequences built for temporal learning If any AI models want to learn from this content, they have to go through us. We’re beyond excited to enter this new territory of data. We’ve already begun conversations with hundreds of the leading game studios, simulation teams, and world building engine developers pushing the next generation of AI generated environments.

Kled now owns exclusive rights for one of the largest aerial drone footage catalogs in the United States. A multi million dollar catalog consisting of: -750,000,000+ unique aerial frames -5,000+ hours of real world flight -100,000+ distinct aerial scenes & environments -Nationwide spatial coverage (urban, suburban, rural, coastal) -High altitude + low altitude perspectives -Continuous motion sequences built for temporal learning If any AI models want to learn from this content, they have to go through us. We’re beyond excited to enter this new territory of data. We’ve already begun conversations with hundreds of the leading game studios, simulation teams, and world building engine developers pushing the next generation of AI generated environments.

Avi Patel

50,154 次观看 • 8 个月前

🎥 Introducing Split-then-Merge: A new video composition framework! This approach enables the composition of any foreground video with any background video. Unlike conventional methods that rely on annotated datasets or handcrafted rules, Split-then-Merge (StM) splits a large unlabeled corpus of videos into dynamic foreground and background layers, then merges them to learn how dynamic subjects interact with diverse scenes. Work done in collaboration with team members at Google: Du Tran (Du Tran) , Yujia Chen (Yujia Chen) , Prof. Ming-Hsuan Yang (Ming-Hsuan Yang), Vincent Chu: and my advisor at UIUC (Siebel School of Computing and Data Science): Prof. James M. Rehg (James Matthew Rehg). I will be attending NeurIPS, San Diego and would be happy to chat more! 🔗Project Webpage: 📄Paper:

🎥 Introducing Split-then-Merge: A new video composition framework! This approach enables the composition of any foreground video with any background video. Unlike conventional methods that rely on annotated datasets or handcrafted rules, Split-then-Merge (StM) splits a large unlabeled corpus of videos into dynamic foreground and background layers, then merges them to learn how dynamic subjects interact with diverse scenes. Work done in collaboration with team members at Google: Du Tran (Du Tran) , Yujia Chen (Yujia Chen) , Prof. Ming-Hsuan Yang (Ming-Hsuan Yang), Vincent Chu: and my advisor at UIUC (Siebel School of Computing and Data Science): Prof. James M. Rehg (James Matthew Rehg). I will be attending NeurIPS, San Diego and would be happy to chat more! 🔗Project Webpage: 📄Paper:

Özgür Kara

16,061 次观看 • 9 个月前

🚨The future of flight is here under President Donald J. Trump with eight new pilot projects as a part of our Advanced Air Mobility pilot program: 🛬 Safely test futuristic aircraft that will RADICALLY CHANGE the way people and products move 🚁 Partners will use these aircraft for all kinds of scenarios, including: urban air taxis, regional travel, cargo logistics, and emergency medicine 🇺🇸 This will create one of the WORLD’s largest testing environments for next-generation aircraft in states all across America It all begins THIS SUMMER!

🚨The future of flight is here under President Donald J. Trump with eight new pilot projects as a part of our Advanced Air Mobility pilot program: 🛬 Safely test futuristic aircraft that will RADICALLY CHANGE the way people and products move 🚁 Partners will use these aircraft for all kinds of scenarios, including: urban air taxis, regional travel, cargo logistics, and emergency medicine 🇺🇸 This will create one of the WORLD’s largest testing environments for next-generation aircraft in states all across America It all begins THIS SUMMER!

Secretary Sean Duffy

668,364 次观看 • 6 个月前

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Flexion

72,471 次观看 • 1 个月前

#SB19SaW | Elements in Motion: D-DAY —Dubai to Doha From the sands of Dubai to the waves that rise beyond — we’ve come full circle, but the journey is far from over. Tonight, as the echoes of the Arena fade, the current carries a new beginning. With hearts full of pride, A’TIN UAE sends our love across the sea. Qatar, are you ready for the next wave? 🌊 🎬 Production Cast & Credits Executive Producer: Sherill Timan Production Manager: Rhea Galido Concept Creators: Sherill Timan & Maricel Briones Creative Producer: Sherill Timan Camera / Videographers: Dawn Julius Bleza, Elvin Mark Santos & Monique Villalon Film Editors: Dawn Julius Bleza & Sherill Timan Special Participation: Fans, Non-Fans & DLM Team 🎥 Produced & Presented by Urban Grid Group ME – FZC (UGG) SB19 Official Urban Grid Group #SB19 #SB19SimulaAtWakas #SB19xUGG #SB19SaWWorldTourDOHA #ATINUAE #ElementsInMotion

#SB19SaW | Elements in Motion: D-DAY —Dubai to Doha From the sands of Dubai to the waves that rise beyond — we’ve come full circle, but the journey is far from over. Tonight, as the echoes of the Arena fade, the current carries a new beginning. With hearts full of pride, A’TIN UAE sends our love across the sea. Qatar, are you ready for the next wave? 🌊 🎬 Production Cast & Credits Executive Producer: Sherill Timan Production Manager: Rhea Galido Concept Creators: Sherill Timan & Maricel Briones Creative Producer: Sherill Timan Camera / Videographers: Dawn Julius Bleza, Elvin Mark Santos & Monique Villalon Film Editors: Dawn Julius Bleza & Sherill Timan Special Participation: Fans, Non-Fans & DLM Team 🎥 Produced & Presented by Urban Grid Group ME – FZC (UGG) SB19 Official Urban Grid Group #SB19 #SB19SimulaAtWakas #SB19xUGG #SB19SaWWorldTourDOHA #ATINUAE #ElementsInMotion

djsherahz ティマン ™️

16,347 次观看 • 10 个月前

🚀✨ A Year of 3D Development! AI Solutions for Fashion ✨🚀 Reflecting on an incredible year of intensive 3D development – a journey where innovation met determination. Back in the day, we navigated the challenges without the aid of AI solutions to craft cutting-edge fashion 3D assets. But guess what? Fast forward to now, and the game has completely changed! 🌐👗 Today, with the power of AI solutions at our fingertips, the process of creating fashion assets has skyrocketed in efficiency, clocking in at a jaw-dropping 10 times faster! 🚀💨 The marriage of creativity and artificial intelligence has truly revolutionized our approach, bringing forth a new era in 3D design. As we celebrate this milestone, we can't help but look ahead to the future. Just like the highly anticipated release of GTA 6 in 2025, Avatly is gearing up to set the pace for what's next. ⚡️🎮 Our commitment to innovation knows no bounds, and we're determined to outpace expectations, ensuring that Avatly stands as a trailblazer in the evolving landscape of 3D development. Join us on this thrilling journey where technology and creativity converge, propelling us into a future where the possibilities are as endless as our imagination! === BUY AVA TOKEN === 💎Bitmart: 💎Pancakeswap:

🚀✨ A Year of 3D Development! AI Solutions for Fashion ✨🚀 Reflecting on an incredible year of intensive 3D development – a journey where innovation met determination. Back in the day, we navigated the challenges without the aid of AI solutions to craft cutting-edge fashion 3D assets. But guess what? Fast forward to now, and the game has completely changed! 🌐👗 Today, with the power of AI solutions at our fingertips, the process of creating fashion assets has skyrocketed in efficiency, clocking in at a jaw-dropping 10 times faster! 🚀💨 The marriage of creativity and artificial intelligence has truly revolutionized our approach, bringing forth a new era in 3D design. As we celebrate this milestone, we can't help but look ahead to the future. Just like the highly anticipated release of GTA 6 in 2025, Avatly is gearing up to set the pace for what's next. ⚡️🎮 Our commitment to innovation knows no bounds, and we're determined to outpace expectations, ensuring that Avatly stands as a trailblazer in the evolving landscape of 3D development. Join us on this thrilling journey where technology and creativity converge, propelling us into a future where the possibilities are as endless as our imagination! === BUY AVA TOKEN === 💎Bitmart: 💎Pancakeswap:

Avatly 💎

15,428 次观看 • 2 年前

✨ Introducing 3 new ways to build with real-world imagery and AI → From the fine details of a city street to a bird's eye view of the planet, at Cloud Next we revealed new ways that businesses can use our Street View, aerial and satellite imagery, and tap into our comprehensive view of the physical world including: 🏙️ Use Maps Imagery Grounding in Gemini Enterprise Agent Platform to generate location-specific assets at scale, anchored in real-world Street View imagery 🌍 Unlock a new perspective with Aerial and Satellite Insights, part of Earth AI, in BigQuery and Agent Platform to quickly extract detailed info about our changing planet 🛰️ Build custom apps with Aerial and Satellite models in Model Garden from Earth AI that easily identify specific objects or scenes Read the blog at the link above and sign up for early access.

✨ Introducing 3 new ways to build with real-world imagery and AI → From the fine details of a city street to a bird's eye view of the planet, at Cloud Next we revealed new ways that businesses can use our Street View, aerial and satellite imagery, and tap into our comprehensive view of the physical world including: 🏙️ Use Maps Imagery Grounding in Gemini Enterprise Agent Platform to generate location-specific assets at scale, anchored in real-world Street View imagery 🌍 Unlock a new perspective with Aerial and Satellite Insights, part of Earth AI, in BigQuery and Agent Platform to quickly extract detailed info about our changing planet 🛰️ Build custom apps with Aerial and Satellite models in Model Garden from Earth AI that easily identify specific objects or scenes Read the blog at the link above and sign up for early access.

Google Maps Platform

12,355 次观看 • 4 个月前

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 次观看 • 3 年前

This is deeply shocking and distressing. The urban middle class in India is rapidly adopting creche culture, but this video reveals a heartbreaking reality. A small child is brutally beaten and traumatized by another child of same age for nearly 10 minutes without pause, including multiple bites. The caregivers, or ayas, entrusted with our children are often poorly paid and lack proper training. I do not hold the child at fault, as they are too young to understand such harm, but the parents and guardians must be accountable. Can anyone help explain what might lead to such violent behavior at such a tender age?

This is deeply shocking and distressing. The urban middle class in India is rapidly adopting creche culture, but this video reveals a heartbreaking reality. A small child is brutally beaten and traumatized by another child of same age for nearly 10 minutes without pause, including multiple bites. The caregivers, or ayas, entrusted with our children are often poorly paid and lack proper training. I do not hold the child at fault, as they are too young to understand such harm, but the parents and guardians must be accountable. Can anyone help explain what might lead to such violent behavior at such a tender age?

Lt Col Sushil Singh Sheoran, Veteran

11,119 次观看 • 2 个月前