Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

cursor tracking on 3D CSS surfaces 👨‍🍳 inverse projection mappin' to calculate proximity in 3D space and apply an eased 0-1 value 🤙 the key: measuring the 4 corners, check the reveal 👇 (ideal for 2D canvas w/ 3D transforms)

jhey ʕ•ᴥ•ʔ

146,411 subscribers

123,007 views • 10 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs 1. Load any 3D scene and hit scan 2. Get 2D detections from VLM 3. Cluster outputs & project into 3D world space 4. Save as a persistent 3D semantic layer Inspired by Alexander Chen's experiments with gemini visual intelligence. Just had to try to lift it from 2D to 3D!

Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs 1. Load any 3D scene and hit scan 2. Get 2D detections from VLM 3. Cluster outputs & project into 3D world space 4. Save as a persistent 3D semantic layer Inspired by Alexander Chen's experiments with gemini visual intelligence. Just had to try to lift it from 2D to 3D!

Bilawal Sidhu

56,910 views • 2 months ago

3D for Hana is here. The first canvas that combines 3D and 2D in real-time. Convert any shape to a 3D model in a non-destructive process. All systems (vectors, booleans, layouts, events, etc.) work together.

3D for Hana is here. The first canvas that combines 3D and 2D in real-time. Convert any shape to a 3D model in a non-destructive process. All systems (vectors, booleans, layouts, events, etc.) work together.

Spline

114,540 views • 8 months ago

Combining 2D and 3D in After Effects. In Foldspace, not every surface needs to be bent. Keep one flat for 2D artwork, and let the others exist in 3D space. This allows you to combine 2D and 3D objects in a single animation with no masks or tricks needed.

Combining 2D and 3D in After Effects. In Foldspace, not every surface needs to be bent. Keep one flat for 2D artwork, and let the others exist in 3D space. This allows you to combine 2D and 3D objects in a single animation with no masks or tricks needed.

Motion Princess

21,666 views • 4 months ago

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

AK

41,736 views • 1 year ago

Thank you Figma for building the best 3D software ever! Jokes aside, this is probably the first and only 3D engine built in Figma. It uses mathematical transformations to project 3D coordinates on a 2D plane (your screen) Check it here:

Thank you Figma for building the best 3D software ever! Jokes aside, this is probably the first and only 3D engine built in Figma. It uses mathematical transformations to project 3D coordinates on a 2D plane (your screen) Check it here:

DoubleGlitch 🇺🇦

106,168 views • 3 years ago

SRB2 without the thok really is peak 3D Sonic. Boost mode is perhaps the best additive mechanic to Sonic we've ever seen and the drop dash aids in turning corners whilst maintaining it and the insta shield acts as an aid to landing on enemies in 3D. This is the ideal moveset 👌

SRB2 without the thok really is peak 3D Sonic. Boost mode is perhaps the best additive mechanic to Sonic we've ever seen and the drop dash aids in turning corners whilst maintaining it and the insta shield acts as an aid to landing on enemies in 3D. This is the ideal moveset 👌

Caramel Smoothie

14,738 views • 1 year ago

Always have an SVG goo filter ready to go in your UI toolkit 🎯 .container { filter: url(#goo); } One line of CSS on the container + the SVG filter 🤙 Throw a border on for "Fake 3D" 👀 SVG filter code below 👇

Always have an SVG goo filter ready to go in your UI toolkit 🎯 .container { filter: url(#goo); } One line of CSS on the container + the SVG filter 🤙 Throw a border on for "Fake 3D" 👀 SVG filter code below 👇

jhey ʕ•ᴥ•ʔ

189,799 views • 2 years ago

Check this out!🔥 Modified version of TripoSR (open source 2D to 3D tool). This version outputs USDZ directly for Apple Vision Pro.🥽 You can easily create 3D objects in AVP with one click & view them in your space! Modified repo in the thread 🧵👇

Check this out!🔥 Modified version of TripoSR (open source 2D to 3D tool). This version outputs USDZ directly for Apple Vision Pro.🥽 You can easily create 3D objects in AVP with one click & view them in your space! Modified repo in the thread 🧵👇

Blaine Brown

48,111 views • 2 years ago

CSS card parallax with pointermove + custom props 🧑‍🍳 <div class=ring style="--i: 2"> .ring { translate: calc(var(--px) * (var(--i) * -2rem)) calc(var(--py) * (var(--i) * 2rem)); } trick: pointer mapping to rotate card in 3D + translate rings 2D for the illusion 🤙 check it 👇

CSS card parallax with pointermove + custom props 🧑‍🍳 <div class=ring style="--i: 2"> .ring { translate: calc(var(--px) * (var(--i) * -2rem)) calc(var(--py) * (var(--i) * 2rem)); } trick: pointer mapping to rotate card in 3D + translate rings 2D for the illusion 🤙 check it 👇

jhey ʕ•ᴥ•ʔ

306,875 views • 9 months ago

🛸 IMPOSTORS HAVE ENTERED ANOTHER DIMENSION 🛸 🤯 3D traits re-imagined in 2D 📸 Use the Previewer to see YOURS 👽 New website ⏰Metadata will update on 10/31 with 2D<>3D Toggle 🧵👇 1/5

🛸 IMPOSTORS HAVE ENTERED ANOTHER DIMENSION 🛸 🤯 3D traits re-imagined in 2D 📸 Use the Previewer to see YOURS 👽 New website ⏰Metadata will update on 10/31 with 2D<>3D Toggle 🧵👇 1/5

Impostors

67,965 views • 2 years ago

Zero Hand Tracking is here! Working on the OS and sensor integration. Accesibility is key, that's why apps are HTML/CSS/JS 2D web apps render as panels in front of you. It's really easy to make this kind of app for Zero 3D apps, like below, run over WebXR, here using #threejs

Zero Hand Tracking is here! Working on the OS and sensor integration. Accesibility is key, that's why apps are HTML/CSS/JS 2D web apps render as panels in front of you. It's really easy to make this kind of app for Zero 3D apps, like below, run over WebXR, here using #threejs

Miroslav Kotalík

13,123 views • 1 year ago

3d visual positioning experiment -- look at the alignment between the 3d mesh and the live camera view. Truly feels magical -- like x-ray vision. Workflow: 1. Scanned a street in 15 mins w/ xgrids 2. Localized against that scan at night, in real-time, while sitting in a car The xgrids scanner (w/ rgb + lidar) is perfect to build maps for humans (3d gaussian splats). Then multiset ai makes it easy to build machine-readable maps for AR and robotics to figure out exactly where they are in 3d space with cm-level accuracy.

3d visual positioning experiment -- look at the alignment between the 3d mesh and the live camera view. Truly feels magical -- like x-ray vision. Workflow: 1. Scanned a street in 15 mins w/ xgrids 2. Localized against that scan at night, in real-time, while sitting in a car The xgrids scanner (w/ rgb + lidar) is perfect to build maps for humans (3d gaussian splats). Then multiset ai makes it easy to build machine-readable maps for AR and robotics to figure out exactly where they are in 3d space with cm-level accuracy.

Bilawal Sidhu

27,278 views • 8 months ago

Did you know that in Blender you can easily bend objects in edit mode by pressing Shift+W? ✨ 1. Select the geometry to bend from 2. Press Shift+S,2 to place the 3d cursor there 3. Select the geometry to bend 4. Press Shift+W to bend! Important: when you're pressing Shift+W, hover your mouse over the geo you're trying to bend, like shown in the video! Bonus: by placing the 3d cursor slightly off center and holding ALT (to disable clamping) while bending, you can easily create spirals! #b3d

Did you know that in Blender you can easily bend objects in edit mode by pressing Shift+W? ✨ 1. Select the geometry to bend from 2. Press Shift+S,2 to place the 3d cursor there 3. Select the geometry to bend 4. Press Shift+W to bend! Important: when you're pressing Shift+W, hover your mouse over the geo you're trying to bend, like shown in the video! Bonus: by placing the 3d cursor slightly off center and holding ALT (to disable clamping) while bending, you can easily create spirals! #b3d

passivestar

1,050,384 views • 3 years ago

📢 Intrinsic Image Fusion for Multi-View 3D Material Reconstruction 📢 We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures 3) optimize low-dimensional parameters for physically-grounded reconstructions. The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset! 🌍 🎥 Great work by Peter Kocsis Lukas Höllein!

📢 Intrinsic Image Fusion for Multi-View 3D Material Reconstruction 📢 We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures 3) optimize low-dimensional parameters for physically-grounded reconstructions. The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset! 🌍 🎥 Great work by Peter Kocsis Lukas Höllein!

Matthias Niessner

23,225 views • 7 months ago

Volumetric, animated 3D noise shader block in Unity. Where each frame is 2D space, and the 3rd dimension is time. So that, measurement of change within a slice is 'space', and measurement of change across slices is 'time'. As a 3D block, you see the noise animate into the 3rd dimension, but when looking at only a single slice, it appears to evolve in place. Likewise, for 4D... If: this were 3D noise, then: the 4th dimension would be whatever processing/frames/states of 3D data.

Volumetric, animated 3D noise shader block in Unity. Where each frame is 2D space, and the 3rd dimension is time. So that, measurement of change within a slice is 'space', and measurement of change across slices is 'time'. As a 3D block, you see the noise animate into the 3rd dimension, but when looking at only a single slice, it appears to evolve in place. Likewise, for 4D... If: this were 3D noise, then: the 4th dimension would be whatever processing/frames/states of 3D data.

Mirza Beig

126,907 views • 7 months ago

// PendingReality showing a closer look at the elizaOS x Unreal Engine plugin — 1/ Import On Chain Character [Emergence] 2/ Activate Agentic Brain [Eliza] This is an evolution for agents + npcs.. flat 2D surfaces >> expansive 3D worlds

// PendingReality showing a closer look at the elizaOS x Unreal Engine plugin — 1/ Import On Chain Character [Emergence] 2/ Activate Agentic Brain [Eliza] This is an evolution for agents + npcs.. flat 2D surfaces >> expansive 3D worlds

Green Loom Association

45,184 views • 1 year ago

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen

Matthias Niessner

25,198 views • 1 month ago

Tripo3D MCP is here, allowing Blender/Cursor/Claude to integrate directly to generate 3D models in your #vibecoding project🫡 Check out an example with the Blender addon👇 Tutorial: Tripo MCP is on: #blendermcp #vibecoding #tripo3d

Tripo3D MCP is here, allowing Blender/Cursor/Claude to integrate directly to generate 3D models in your #vibecoding project🫡 Check out an example with the Blender addon👇 Tutorial: Tripo MCP is on: #blendermcp #vibecoding #tripo3d

Tripo🔜SIGGRAPH2026

136,260 views • 1 year ago

3D Object Tracking without Training Data? In our nature Machine Intelligence paper ( we recast 3D tracking as an inverse neural rendering task where we fit a scene graph to an image that best explains this image. The method generalizes to completely unseen datasets and is explainable. Project and Code: Fun collaboration between Princeton Computer Science and Torc Robotics, with Julian Ost and Tanushree Banerjee leading this project.

3D Object Tracking without Training Data? In our nature Machine Intelligence paper ( we recast 3D tracking as an inverse neural rendering task where we fit a scene graph to an image that best explains this image. The method generalizes to completely unseen datasets and is explainable. Project and Code: Fun collaboration between Princeton Computer Science and Torc Robotics, with Julian Ost and Tanushree Banerjee leading this project.

Felix Heide

27,858 views • 1 year ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 views • 3 years ago