Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Latent Spatial Memory for Video World Models

_akhaliq's profile picture

AK

507,762 subscribers

22,584 Aufrufe • vor 19 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

MindJourney Test-Time Scaling with World Models for Spatial Reasoning
0:43

MindJourney Test-Time Scaling with World Models for Spatial Reasoning

AK

61,714 Aufrufe • vor 11 Monaten

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:
0:47

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:

Tencent Hy

20,792 Aufrufe • vor 3 Monaten

Video of my opening keynote for the 2026 World Modeling Workshop at Mila, Quebec AI Institute: simple but powerful ways of using world models and their latent space (sorry for sound problem 0:20-0:43). Details and references in: The Neural World Model Boom
31:16

Video of my opening keynote for the 2026 World Modeling Workshop at Mila, Quebec AI Institute: simple but powerful ways of using world models and their latent space (sorry for sound problem 0:20-0:43). Details and references in: The Neural World Model Boom

Jürgen Schmidhuber

126,709 Aufrufe • vor 2 Monaten

You’ve used language models, image models, video models, and voice models. Now it’s time for world models, thanks to World Labs.
3:42

You’ve used language models, image models, video models, and voice models. Now it’s time for world models, thanks to World Labs.

Amir Haghighat

838,859 Aufrufe • vor 4 Monaten

ทำไมผู้หญิงชอบหลงทาง อ่าน Google Maps ไม่ออก บางคนจำทางเก่ง เดินไปไหนก็ไม่หลง แต่บางคนต่อให้เปิดแผนที่ก็ยังงงว่าต้องเลี้ยวซ้ายหรือขวา นี่คือเรื่อง Spatial Memory หรือความจำเชิงพื้นที่ของสมอง Spatial Memory คือความสามารถในการจำตำแหน่ง วัตถุ และเส้นทางรอบตัว สมองส่วน Hippocampus จะสร้าง “แผนที่ในหัว” หรือ Cognitive Map ให้เราใช้ในการนำทาง หากสมองส่วนนี้ไม่แข็งแรง เราจะหลงทางง่าย และอาจเป็นสัญญาณเริ่มต้นของสมองเสื่อม วิธีฝึก Spatial Memory 1.สังเกต Landmark รอบตัวเวลาเดินทาง 2.วาดแผนที่ในหัว หรือหมุนแผนที่ให้ตรงกับทิศทางที่เราเดิน 3.ออกกำลังกายแบบแอโรบิกเพื่อกระตุ้น Hippocampus 4.นอนหลับเพียงพอและลดความเครียด เพื่อให้สมองเรียนรู้ทิศทางได้ดีที่สุด การฝึก Spatial Memory ไม่ใช่แค่ช่วยให้ไม่หลงทาง แต่ยังเป็นการป้องกันสมองเสื่อมในอนาคต #BrainLongevity #TheLongevist #หมออั๋น #longevity
0:53

ทำไมผู้หญิงชอบหลงทาง อ่าน Google Maps ไม่ออก บางคนจำทางเก่ง เดินไปไหนก็ไม่หลง แต่บางคนต่อให้เปิดแผนที่ก็ยังงงว่าต้องเลี้ยวซ้ายหรือขวา นี่คือเรื่อง Spatial Memory หรือความจำเชิงพื้นที่ของสมอง Spatial Memory คือความสามารถในการจำตำแหน่ง วัตถุ และเส้นทางรอบตัว สมองส่วน Hippocampus จะสร้าง “แผนที่ในหัว” หรือ Cognitive Map ให้เราใช้ในการนำทาง หากสมองส่วนนี้ไม่แข็งแรง เราจะหลงทางง่าย และอาจเป็นสัญญาณเริ่มต้นของสมองเสื่อม วิธีฝึก Spatial Memory 1.สังเกต Landmark รอบตัวเวลาเดินทาง 2.วาดแผนที่ในหัว หรือหมุนแผนที่ให้ตรงกับทิศทางที่เราเดิน 3.ออกกำลังกายแบบแอโรบิกเพื่อกระตุ้น Hippocampus 4.นอนหลับเพียงพอและลดความเครียด เพื่อให้สมองเรียนรู้ทิศทางได้ดีที่สุด การฝึก Spatial Memory ไม่ใช่แค่ช่วยให้ไม่หลงทาง แต่ยังเป็นการป้องกันสมองเสื่อมในอนาคต #BrainLongevity #TheLongevist #หมออั๋น #longevity

The Longevist

283,914 Aufrufe • vor 10 Monaten

experimenting with realtime latent consistency models in Figma
0:30

experimenting with realtime latent consistency models in Figma

Jordan Singer

374,693 Aufrufe • vor 2 Jahren

AnchorWeave World-Consistent Video Generation with Retrieved Local Spatial Memories paper:
0:24

AnchorWeave World-Consistent Video Generation with Retrieved Local Spatial Memories paper:

AK

23,312 Aufrufe • vor 4 Monaten

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]
0:25

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]

Bardienus Duisterhof

63,508 Aufrufe • vor 16 Tagen

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:
0:25

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:

Xingang Pan

15,902 Aufrufe • vor 6 Monaten

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.
1:16:13

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

abdel

34,523 Aufrufe • vor 25 Tagen

Real World AI doesn’t fail because of models. It fails because it lacks context. Project Jade shows true spatial intelligence through an embodied AI experience on Snap Spectacles. 👓 Ready for the next AI wave? Niantic Spatial is already here. (1/4) 🔗
1:00

Real World AI doesn’t fail because of models. It fails because it lacks context. Project Jade shows true spatial intelligence through an embodied AI experience on Snap Spectacles. 👓 Ready for the next AI wave? Niantic Spatial is already here. (1/4) 🔗

Niantic Spatial 🌎

16,381 Aufrufe • vor 5 Monaten

try out the Gradio Demo for AudioLDM: Text-to-Audio Generation with Latent Diffusion Models on Hugging Face demo:
1:36

try out the Gradio Demo for AudioLDM: Text-to-Audio Generation with Latent Diffusion Models on Hugging Face demo:

AK

82,137 Aufrufe • vor 3 Jahren

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.
1:00

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.

asim ᯅ

49,156 Aufrufe • vor 23 Tagen

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:
0:32

Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:

Chuang Gan

43,265 Aufrufe • vor 1 Jahr

India’s PM’s got latent Full video -
1:36

India’s PM’s got latent Full video -

Official PeeingHuman

17,783 Aufrufe • vor 6 Monaten

What does JEPA actually learn? We can finally prove it 🌍 So excited to share our theory of identifiable World Models: LeJEPA recovers the latent variables of the world. Plan in the learned World Model as if it were real, same shortest path. 📄:
0:31

What does JEPA actually learn? We can finally prove it 🌍 So excited to share our theory of identifiable World Models: LeJEPA recovers the latent variables of the world. Plan in the learned World Model as if it were real, same shortest path. 📄:

David Klindt

212,929 Aufrufe • vor 1 Monat

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:
0:54

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:

Tencent Hy

34,317 Aufrufe • vor 2 Monaten

Announcing a new Memory system for robots on Dimensional Robots in production generate thousands of hours of video, lidar, odometry, far too large to fit into your Agent context SpatialMemory2 builds a multimodal data store in latent space for your Agents Fully open source
0:24

Announcing a new Memory system for robots on Dimensional Robots in production generate thousands of hours of video, lidar, odometry, far too large to fit into your Agent context SpatialMemory2 builds a multimodal data store in latent space for your Agents Fully open source

stash

228,166 Aufrufe • vor 2 Monaten

Announcing the Video Editor in Spatial Media Toolkit. Effortlessly create spatial videos or photo slideshows with music, all privately on-device. Seamlessly integrates with our powerful 2D-to-3D conversion for stunning spatial storytelling.
0:47

Announcing the Video Editor in Spatial Media Toolkit. Effortlessly create spatial videos or photo slideshows with music, all privately on-device. Seamlessly integrates with our powerful 2D-to-3D conversion for stunning spatial storytelling.

Ryan Gordon

12,353 Aufrufe • vor 1 Jahr