Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

🚀 New work [ICML 2026]: Structured 4D Latent Predictive Model for Robot Planning. Can robots plan by imagining future 3D structure, not just pixels? Multi-view obs + language → 3D latent rollouts → 3D-consistent subgoals for robot actions.

Zhiyi Li

1,188 subscribers

99,836 görüntüleme • 4 ay önce •via X (Twitter)

Bilim & Teknoloji Eğitim

Anya Rossi• Live Now

Private livecam show

13 Yorum

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(1/7) Why 3D in planning? A general-purpose robot in the real world must reason about where objects are, how they move, and how actions change the scene. 2D pixels can look plausible but be geometrically inconsistent. For control, predictions need 3D structures.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(2/7) Key idea: predict future 3D structure directly. Multi-view observations are encoded into a structured 3D latent, then rolled forward over time conditioned on text instructions.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(7/7) Takeaway: future 3D prediction is a useful bridge between generative world models and robot control. Please visit our project page for more results, visualizations, and details:

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(4/7) Generation results: the model unrolls language-conditioned 4D futures across tasks, from stacking and insertion to longer-horizon manipulation. Rendered views and point clouds stay coherent over time, showing strong 3D consistency.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(3/7) How it works: reconstruct current 3D latent, use a Single Dynamics Model for coarse structure and a Latent Generator for details, then decode future latents into point clouds/3D Gaussians, which are then converted to robot actions by inverse dynamics.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(5/7) Generalization: for robot planning, small unseen changes—lighting, noise, background color, or camera pose—can break policies. Structured 3D rollouts make the planner more robust under these shifts.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

(6/7) Real world: we train from block-in-basket demos with 4 RGB-D cameras. Predicted 3D subgoals are registered to recover gripper poses, then executed with training-free point cloud registration-based motion planning, showing the pipeline can transfer beyond simulation.

Zhiyi Li profil fotoğrafı

Zhiyi Li4 ay önce

Many thanks to my advisor @du_yilun and all coauthors @PeilinWu64 @xshenhan @ruojin8!

Gerard Sans | Axiom 🇬🇧 profil fotoğrafı

Gerard Sans | Axiom 🇬🇧4 ay önce

can’t beat this

AI Mastery Guide profil fotoğrafı

AI Mastery Guide4 ay önce

Planning in 3D structure instead of raw pixels feels like a much closer match to how humans actually imagine actions.

GarethTheDreamer profil fotoğrafı

GarethTheDreamer4 ay önce

Structured 4D latent space for planning, really elegant approach! 🚀 Love that it generalizes to novel viewpoints without retraining

oscar profil fotoğrafı

oscar4 ay önce

We provide guest posting backlinks on high-quality sites at affordable rates. I have Websites with Domain Authority 50 and organic traffic 200K/month

Taeyoon Kim profil fotoğrafı

Taeyoon Kim4 ay önce

Really enjoyed this!

Benzer Videolar

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

Champ Controllable and Consistent Human Image Animation with 3D Parametric Guidance In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion

AK

194,605 görüntüleme • 2 yıl önce

What happens when you try to carry the weight of a 3D printer as a 3D Printed tiny robot. But still amazing 🤩 work done by manufacturedmotion spotted at Open Sauce 3D Printed robots are now carrying 3D Printers! 📕 #3dprinting #robotics #robot #maker #prusa

What happens when you try to carry the weight of a 3D printer as a 3D Printed tiny robot. But still amazing 🤩 work done by manufacturedmotion spotted at Open Sauce 3D Printed robots are now carrying 3D Printers! 📕 #3dprinting #robotics #robot #maker #prusa

Ebokify

11,505 görüntüleme • 3 ay önce

🔥 New Feature: Multi-View Images in One Click! 🎨 Turn text/image → consistent multi-views (powered by Flux Kontext) → high-accuracy 3D models in seconds! 🚀 Boost your 3D workflow today:

🔥 New Feature: Multi-View Images in One Click! 🎨 Turn text/image → consistent multi-views (powered by Flux Kontext) → high-accuracy 3D models in seconds! 🚀 Boost your 3D workflow today:

MeshyAI

10,461 görüntüleme • 1 yıl önce

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,583 görüntüleme • 3 yıl önce

How can we generate high-fidelity, complex 3D scenes? Quan Meng's LT3SD decomposes 3D scenes into latent tree representations, with diffusion on the latent trees enabling seamless infinite 3D scene synthesis! w/ Lei Li, Matthias Niessner

How can we generate high-fidelity, complex 3D scenes? Quan Meng's LT3SD decomposes 3D scenes into latent tree representations, with diffusion on the latent trees enabling seamless infinite 3D scene synthesis! w/ Lei Li, Matthias Niessner

Angela Dai

29,469 görüntüleme • 2 yıl önce

Apple drops a new 3D model generator, LiTo > Learns geometry + appearance together in a unified 3D latent space > Turns RGB-D images into a compact surface light field > Recreates specular highlights under complex lighting > Generates 3D objects from a single image with input-consistent materials Big step toward more realistic 3D generation

Apple drops a new 3D model generator, LiTo > Learns geometry + appearance together in a unified 3D latent space > Turns RGB-D images into a compact surface light field > Recreates specular highlights under complex lighting > Generates 3D objects from a single image with input-consistent materials Big step toward more realistic 3D generation

AI Search

13,078 görüntüleme • 5 ay önce

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Everyone is scaling VLAs with more robot data. TiPToP shows another path. No robot training, no policy learning. Just RGB + language → 3D scene → GPU TAMP planner → trajectory. Foundation models + planning alone can run real manipulation tasks.

Robots Digest 🤖

10,425 görüntüleme • 7 ay önce

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction

🚀 Our NeurIPS '24 work, Large Spatial Model (LSM), is here! LSM performs semantic 3D reconstruction in just 0.1s, processing unposed data via feed-forward 3D reconstruction. 👉It leverages large-scale 3D datasets with minimal annotations, defining a 3D latent space. We are continuously exploring how this explicit 3D representation can further enhance reasoning and robotic learning. 🔗 Try our online Gradio demo with your own data at #NeurIPS2024 #3DReconstruction

Zhiwen(Aaron) Fan

44,051 görüntüleme • 2 yıl önce

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 görüntüleme • 1 yıl önce

Fuchikoma Microbit caster wheel robot. Almost entirely 3D printed. Future plan may include ultrasonic sensors for self navigation. Designed and built by yours truly. #robotics #Microbit #Ghostintheshell

Fuchikoma Microbit caster wheel robot. Almost entirely 3D printed. Future plan may include ultrasonic sensors for self navigation. Designed and built by yours truly. #robotics #Microbit #Ghostintheshell

Lewis RK ( Mechalomania )

65,357 görüntüleme • 1 yıl önce

Introducing WorldCrafter: a world model with persistent memory—not rigid 3D geometry, not just latent memory, but a “soft skeleton” of 3D-aware latents. The result: striking consistency as you explore, look away, and return—even in dynamic scenes.

Introducing WorldCrafter: a world model with persistent memory—not rigid 3D geometry, not just latent memory, but a “soft skeleton” of 3D-aware latents. The result: striking consistency as you explore, look away, and return—even in dynamic scenes.

Ying Shan

28,026 görüntüleme • 1 ay önce

Cosmos Policy turns a pretrained video diffusion model into a robot controller. Instead of redesigning the architecture, it injects robot state, actions, and values directly as latent frames inside the video model

Cosmos Policy turns a pretrained video diffusion model into a robot controller. Instead of redesigning the architecture, it injects robot state, actions, and values directly as latent frames inside the video model

Robots Digest 🤖

22,933 görüntüleme • 9 ay önce

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

Tencent HY

134,410 görüntüleme • 1 yıl önce

Map2World. turns text descriptions and sketch maps into full 3D worlds. -multi-diffusion in SLAT latent space for global consistency; - beaats SynCity - supports arbitrary world sizes.

Map2World. turns text descriptions and sketch maps into full 3D worlds. -multi-diffusion in SLAT latent space for global consistency; - beaats SynCity - supports arbitrary world sizes.

Wildminder

15,679 görüntüleme • 6 ay önce

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.

Another day, another humanoid robot from china AGIBOT introduces GO-1, a generalist foundation model that integrates a vision-language model with a latent planner for enhanced long-horizon and dexterous manipulation.

Chubby♨️

23,513 görüntüleme • 1 yıl önce

Introducing PointZero—a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: 🔥 Zero-shot 3D dynamics 🔥 Action-conditioned 3D dynamics (post-training) 🔥 Imitation learning (post-training) Details and links 👇 1/6

Introducing PointZero—a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: 🔥 Zero-shot 3D dynamics 🔥 Action-conditioned 3D dynamics (post-training) 🔥 Imitation learning (post-training) Details and links 👇 1/6

Bardienus Duisterhof

29,174 görüntüleme • 1 ay önce

A new paradigm for robot learning that unifies latent world modeling and action generation. Astribot has introduced Lumo-2, a latent world-action model for robot control. - Validated on a real robot through 20+ intricate chores, extending beyond simple benchmark testing. - A 3-stage recipe that progressively aligns actions with world dynamics, then vision, then language - Action quality depends on the geometry of the latent space, not just reconstruction accuracy - BAR (block-wise decoding) speeds up inference by 2.71x compared to standard autoregressive decoding - Shows cross-embodiment transfer without specialized transfer learning Results: - Beats π0.5 and Fast-WAM in temporal reasoning, physical understanding, long-horizon, and dexterous manipulation - Lumo-2's VLM consistently outperforms Qwen-3.5 4B on embodied benchmarks

A new paradigm for robot learning that unifies latent world modeling and action generation. Astribot has introduced Lumo-2, a latent world-action model for robot control. - Validated on a real robot through 20+ intricate chores, extending beyond simple benchmark testing. - A 3-stage recipe that progressively aligns actions with world dynamics, then vision, then language - Action quality depends on the geometry of the latent space, not just reconstruction accuracy - BAR (block-wise decoding) speeds up inference by 2.71x compared to standard autoregressive decoding - Shows cross-embodiment transfer without specialized transfer learning Results: - Beats π0.5 and Fast-WAM in temporal reasoning, physical understanding, long-horizon, and dexterous manipulation - Lumo-2's VLM consistently outperforms Qwen-3.5 4B on embodied benchmarks

The Humanoid Hub

20,623 görüntüleme • 3 ay önce

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

250,112 görüntüleme • 3 yıl önce