Loading video...

Video Failed to Load

Go Home

4D LangSplat 4D Language Gaussian Splatting via Multimodal Large Language Models

_akhaliq's profile picture

AK

504,320 subscribers

30,848 views • 1 year ago •via X (Twitter)

EducationScience & TechnologyNews & Politics
LIVE

Anya Rossi• Live Now

Private livecam show

3 Comments

AK's profile picture
AK1 year ago

discuss:

AssemblyAI's profile picture
AssemblyAI1 year ago

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

Universa's profile picture
Universa1 year ago

Exciting to see advancements in 4D language Gaussian splatting via multimodal large language models! The intersection of AI and language processing is a key area of focus for Universa, and initiatives like Universa Earth aim to drive innovation in this space.

Related Videos

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.
0:29

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.

MrNeRF

10,953 views • 1 year ago

a one minute explanation of 4d gaussian splatting
0:52

a one minute explanation of 4d gaussian splatting

dylan

186,592 views • 2 years ago

Long live bullet time. The future of sports is 4d gaussian splatting.
0:32

Long live bullet time. The future of sports is 4d gaussian splatting.

Bilawal Sidhu

127,960 views • 3 months ago

TALKPLAY Multimodal Music Recommendation with Large Language Models
1:36

TALKPLAY Multimodal Music Recommendation with Large Language Models

AK

20,864 views • 1 year ago

Wow! 4D Gaussian Splatting on Apple Vision Pro looks INSANE 🤯 This feels like the future
0:30

Wow! 4D Gaussian Splatting on Apple Vision Pro looks INSANE 🤯 This feels like the future

Ben Geskin

27,539 views • 1 year ago

Wow! 4DV releases new examples of their 4D Gaussian Splatting tech. The renderer used is @PlayCanvas.
0:44

Wow! 4DV releases new examples of their 4D Gaussian Splatting tech. The renderer used is @PlayCanvas.

Will Eastcott

44,017 views • 6 months ago

LangSplatV2 High-dimensional 3D Language Gaussian Splatting with 450+ FPS
2:57

LangSplatV2 High-dimensional 3D Language Gaussian Splatting with 450+ FPS

AK

46,826 views • 1 year ago

🤩Apple opensources MGIE! Now one can take random pictures w. iPhone & edit w. language! Guiding Instruction-based Image Editing via Multimodal Large Language Models #ICLR2024 spotlight: Apple repo Gradio
0:34

🤩Apple opensources MGIE! Now one can take random pictures w. iPhone & edit w. language! Guiding Instruction-based Image Editing via Multimodal Large Language Models #ICLR2024 spotlight: Apple repo Gradio

William Wang

125,022 views • 2 years ago

4D Gaussian Splatting on the web just leveled up. ✨ Announcing the integration of Gracia with the open-source @PlayCanvas Engine. 🧵
1:02

4D Gaussian Splatting on the web just leveled up. ✨ Announcing the integration of Gracia with the open-source @PlayCanvas Engine. 🧵

Will Eastcott

151,370 views • 3 months ago

4D Gaussian Splatting is mindblowing. How is this level of fidelity and realism able to run so flawlessly on a web browser?
1:16

4D Gaussian Splatting is mindblowing. How is this level of fidelity and realism able to run so flawlessly on a web browser?

NikTek

278,850 views • 3 months ago

Oh we talking 4D gaussian splats? 4DV.ai
0:30

Oh we talking 4D gaussian splats? 4DV.ai

Wren

340,157 views • 5 months ago

The Blender 4D Gaussian Splatting project by Zhi Wang is now live on Blender's Demo Files page, complete with two preconfigured scenes. Download here:
0:20

The Blender 4D Gaussian Splatting project by Zhi Wang is now live on Blender's Demo Files page, complete with two preconfigured scenes. Download here:

80 LEVEL

32,860 views • 8 months ago

3D gaussian splatting + time (video) = 4D gaussian splatting is now possible This means in the near future you'll be able to fly around on your phone in any video you recorded A lot like the "braindance" in Cyberpunk 2077
0:29

3D gaussian splatting + time (video) = 4D gaussian splatting is now possible This means in the near future you'll be able to fly around on your phone in any video you recorded A lot like the "braindance" in Cyberpunk 2077

@levelsio

590,061 views • 1 year ago

RynnWorld-4D 4D Embodied World Models for Robotic Manipulation
1:41

RynnWorld-4D 4D Embodied World Models for Robotic Manipulation

AK

33,467 views • 22 days ago

omg.. this cant be real China’s 4DV AI just dropped 4D Gaussian Splatting, you can turn 2D video into 4D with sound.. imagine.. we will be able to change camera angle, zoom in/out while watching movies 5 examples:
0:25

omg.. this cant be real China’s 4DV AI just dropped 4D Gaussian Splatting, you can turn 2D video into 4D with sound.. imagine.. we will be able to change camera angle, zoom in/out while watching movies 5 examples:

el.cine

3,611,813 views • 1 year ago

Large language models:
0:46

Large language models:

Liv

91,909 views • 7 months ago

Large Language Models Visualized
1:06

Large Language Models Visualized

ₕₐₘₚₜₒₙ

85,984 views • 1 year ago

LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS Contributions: • LangSplatV2 achieves real-time performance with 476.2 FPS for high-dimensional feature splatting and 384.6 FPS for 3D open-vocabulary text querying. • Delivers a 42× speedup and 47× boost in performance compared to LangSplat. • Improves query accuracy while drastically reducing inference time. • Replaces the heavyweight decoder in LangSplat with a sparse coefficient field, removing the main performance bottleneck. • Introduces a CUDA-optimized sparse coefficient splatting method, enabling fast and high-quality rendering of high-dimensional features. • Enables scalable 3D language interaction in complex scenes, opening up real-time applications previously not possible with LangSplat.
1:13

LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS Contributions: • LangSplatV2 achieves real-time performance with 476.2 FPS for high-dimensional feature splatting and 384.6 FPS for 3D open-vocabulary text querying. • Delivers a 42× speedup and 47× boost in performance compared to LangSplat. • Improves query accuracy while drastically reducing inference time. • Replaces the heavyweight decoder in LangSplat with a sparse coefficient field, removing the main performance bottleneck. • Introduces a CUDA-optimized sparse coefficient splatting method, enabling fast and high-quality rendering of high-dimensional features. • Enables scalable 3D language interaction in complex scenes, opening up real-time applications previously not possible with LangSplat.

MrNeRF

14,261 views • 1 year ago

Stream-Omni Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
0:37

Stream-Omni Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model

AK

12,374 views • 1 year ago