Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🔥Spatial intelligence requires world generation, and now we have the first comprehensive evaluation benchmark📏 for it! Introducing WorldScore: Unifying evaluation for 3D, 4D, and video models on world generation! 🧵1/7 Web: arxiv:

Koven_Yu's profile picture

Hong-Xing (Koven) Yu

2,376 subscribers

66,876 Aufrufe • vor 1 Jahr •via X (Twitter)

Nachrichten & PolitikWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🔥Spatial intelligence needs fast, *interactive* 3D world generation 🎮 — introducing WonderWorld: generating 3D scenes interactively following your movement and content requests, and see them in <10 seconds! 🧵1/6 Web: arXiv:
0:10

🔥Spatial intelligence needs fast, *interactive* 3D world generation 🎮 — introducing WonderWorld: generating 3D scenes interactively following your movement and content requests, and see them in <10 seconds! 🧵1/6 Web: arXiv:

Hong-Xing (Koven) Yu

169,184 Aufrufe • vor 2 Jahren

Voice AI has an evaluation problem. Models look strong on public benchmarks, then collapse on real-world audio. Introducing a recipe-driven evaluation framework for low-resource languages, real-world audio, and production failure modes. Details ↓
0:10

Voice AI has an evaluation problem. Models look strong on public benchmarks, then collapse on real-world audio. Introducing a recipe-driven evaluation framework for low-resource languages, real-world audio, and production failure modes. Details ↓

Poseidon

35,201 Aufrufe • vor 3 Monaten

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 Aufrufe • vor 1 Jahr

With Hunyuan3D World Model 1.0 now released and open-sourced, we're excited to showcase the technical highlights behind this impressive innovation: ✅360° Panoramic Generation: Creates complete, immersive “world scenes”, far beyond localized views. ✅Explorable 3D Scene Generation: Generates diverse, spatially consistent 3D worlds from text/image for truly immersive exploration. ✅Interactive/Editable: Achieves separation of foreground objects, background terrain, ground, and sky, for seamless secondary editing. ✅Exportable Mesh: Generated scenes can be exported as 3D meshes for direct import into mainstream game engines and modeling software. ✅Industry-Leading SOTA Evaluation: Surpasses state-of-the-art open-source models in generation quality. As the industry's first open-source model for physical simulation and explorable world generation, Hunyuan3D World Model 1.0 aims to foster a collaborative community ecosystem with developers and enthusiasts. ✨ Try it now: 🤗 Hugging Face:
0:07

With Hunyuan3D World Model 1.0 now released and open-sourced, we're excited to showcase the technical highlights behind this impressive innovation: ✅360° Panoramic Generation: Creates complete, immersive “world scenes”, far beyond localized views. ✅Explorable 3D Scene Generation: Generates diverse, spatially consistent 3D worlds from text/image for truly immersive exploration. ✅Interactive/Editable: Achieves separation of foreground objects, background terrain, ground, and sky, for seamless secondary editing. ✅Exportable Mesh: Generated scenes can be exported as 3D meshes for direct import into mainstream game engines and modeling software. ✅Industry-Leading SOTA Evaluation: Surpasses state-of-the-art open-source models in generation quality. As the industry's first open-source model for physical simulation and explorable world generation, Hunyuan3D World Model 1.0 aims to foster a collaborative community ecosystem with developers and enthusiasts. ✨ Try it now: 🤗 Hugging Face:

Tencent Hy

23,240 Aufrufe • vor 1 Jahr

We are introducing v0-1.5-md and v0-1.5-lg, our newest models for UI generation and web development. Available today on Coming to the API next week.
0:11

We are introducing v0-1.5-md and v0-1.5-lg, our newest models for UI generation and web development. Available today on Coming to the API next week.

v0

131,019 Aufrufe • vor 1 Jahr

I'm excited to announce that a year after launching and growing into the largest benchmark for Generative 3D, 3D Arena now has an official academic paper! 3D Arena: An Open Platform for Generative 3D Evaluation
0:10

I'm excited to announce that a year after launching and growing into the largest benchmark for Generative 3D, 3D Arena now has an official academic paper! 3D Arena: An Open Platform for Generative 3D Evaluation

dylan

31,021 Aufrufe • vor 1 Jahr

Announcing Neo-1: the world’s most advanced atomistic foundation model, unifying structure prediction and all-atom de novo generation for the first time - to decode and design the structure of life 🧵(1/10)
0:12

Announcing Neo-1: the world’s most advanced atomistic foundation model, unifying structure prediction and all-atom de novo generation for the first time - to decode and design the structure of life 🧵(1/10)

Proxima

332,298 Aufrufe • vor 1 Jahr

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)
0:13

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)

Shuang Li

67,720 Aufrufe • vor 1 Jahr

world models have their comfyui now. introducing alakazam studio, the first local, open source UI for world models. •⁠ ⁠sentence goes in => you edit the state graph => play •⁠ ⁠on a live model, yours, running in your browser •⁠ ⁠⁠apache-2.0, fully open source fork it and point it at your model 👇
0:19

world models have their comfyui now. introducing alakazam studio, the first local, open source UI for world models. •⁠ ⁠sentence goes in => you edit the state graph => play •⁠ ⁠on a live model, yours, running in your browser •⁠ ⁠⁠apache-2.0, fully open source fork it and point it at your model 👇

hugo

26,101 Aufrufe • vor 1 Monat

AI’s next frontier is Spatial Intelligence, a technology that will turn seeing into reasoning, perception into action, and imagination into creation. But what is it? Why does it matter? How do we build it? And how can we use it? Today, I want to share with you my thoughts on building and using world models to unlock spatial intelligence in this essay below. 1/n
0:11

AI’s next frontier is Spatial Intelligence, a technology that will turn seeing into reasoning, perception into action, and imagination into creation. But what is it? Why does it matter? How do we build it? And how can we use it? Today, I want to share with you my thoughts on building and using world models to unlock spatial intelligence in this essay below. 1/n

Fei-Fei Li

928,724 Aufrufe • vor 10 Monaten

High-resolution image and video generation is hitting a wall because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵
0:17

High-resolution image and video generation is hitting a wall because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵

Gordon Wetzstein

171,136 Aufrufe • vor 5 Monaten

My first PhD paper!🎉We learn *diffusion* models for code generation that learn to directly *edit* syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/n
0:12

My first PhD paper!🎉We learn *diffusion* models for code generation that learn to directly *edit* syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/n

Shreyas Kapur

742,687 Aufrufe • vor 2 Jahren

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):
0:16

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):

Xingang Pan

11,531 Aufrufe • vor 10 Monaten

This is some quietly impressive work on making video world models actually controllable in 4D space. VerseCrafter lets you take an input image, use something like Blender to animate the 3D camera path and object trajectories, then uses that to condition generation. Scribbling in 2D feels so crude in comparison. The authors represent everything in a shared 4D world state - static background as a point cloud, moving objects as 3D gaussian trajectories. The gaussians are an interesting choice because they capture position, shape, and orientation probabilistically rather than forcing rigid bounding boxes or category specific models like SMPL-X for human bodies. They bolt this onto frozen Wan2.1 with a lightweight adapter, so they get a strong video prior. They also built a pipeline to auto extract 4D annotations from real world videos to train this puppy. It doesn't look sexy yet, but IMO this is the interface video world models need - actual 3D authoring tools to exert control rather than crude scribbles and prompt incantations.
0:17

This is some quietly impressive work on making video world models actually controllable in 4D space. VerseCrafter lets you take an input image, use something like Blender to animate the 3D camera path and object trajectories, then uses that to condition generation. Scribbling in 2D feels so crude in comparison. The authors represent everything in a shared 4D world state - static background as a point cloud, moving objects as 3D gaussian trajectories. The gaussians are an interesting choice because they capture position, shape, and orientation probabilistically rather than forcing rigid bounding boxes or category specific models like SMPL-X for human bodies. They bolt this onto frozen Wan2.1 with a lightweight adapter, so they get a strong video prior. They also built a pipeline to auto extract 4D annotations from real world videos to train this puppy. It doesn't look sexy yet, but IMO this is the interface video world models need - actual 3D authoring tools to exert control rather than crude scribbles and prompt incantations.

Bilawal Sidhu

26,017 Aufrufe • vor 8 Monaten

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:
0:10

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:

Tencent Hy

20,581 Aufrufe • vor 8 Monaten

There's a lot of speculation about whether OpenAI's video generation model Sora has a 'physics engine' (bolstered by OAI's own claims about 'world simulation'). Like the debate about world models in LLMs, this question is both genuinely interesting and somewhat ill-defined. 🧵1/
0:20

There's a lot of speculation about whether OpenAI's video generation model Sora has a 'physics engine' (bolstered by OAI's own claims about 'world simulation'). Like the debate about world models in LLMs, this question is both genuinely interesting and somewhat ill-defined. 🧵1/

Raphaël Millière

617,455 Aufrufe • vor 2 Jahren

Real-time generation isn’t about predicting the next pixel, it requires understanding the structure and context of a 3D world. That’s the shift. Read more about our real-time game-native diffusion model on our blog. Moonlake's Reverie v0.1.0
0:11

Real-time generation isn’t about predicting the next pixel, it requires understanding the structure and context of a 3D world. That’s the shift. Read more about our real-time game-native diffusion model on our blog. Moonlake's Reverie v0.1.0

Moonlake

14,439 Aufrufe • vor 9 Monaten

Our CUDA-native genome generation stack can now generate a whole synthetic genome in just under 20 minutes on eight H200 GPUs from lium.io . That's roughly 440x faster than the traditional CPU-bound workflows. This opens the door to population-scale synthetic genome generation for benchmarking, variant calling evaluation, and genomic AI.
0:10

Our CUDA-native genome generation stack can now generate a whole synthetic genome in just under 20 minutes on eight H200 GPUs from lium.io . That's roughly 440x faster than the traditional CPU-bound workflows. This opens the door to population-scale synthetic genome generation for benchmarking, variant calling evaluation, and genomic AI.

Minos

16,787 Aufrufe • vor 2 Monaten