Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

improving my 3D hand tracking template by adding depth-anything-v2 using mediapipe for X/Y movement and depth-anything for Z movement here i'm controlling a threejs scene in realtime with input from a single webcam

AA

25,902 subscribers

28,276 views • 3 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

I created this using Claude + 3D animation: 1. Ask Claude for a scroll-based cinematic animation pla 2. Generate prompts for depth, parallax, and camera movement 3. Install Three.js + GSAP + ScrollTrigger 4. Use scroll to control zoom, scale, and scene transitions 5. Add layered images for 3D depth illusion 6. Apply smooth easing, lighting, and shadows for realism 7. Refine until it feels like one continuous scene Save this

I created this using Claude + 3D animation: 1. Ask Claude for a scroll-based cinematic animation pla 2. Generate prompts for depth, parallax, and camera movement 3. Install Three.js + GSAP + ScrollTrigger 4. Use scroll to control zoom, scale, and scene transitions 5. Add layered images for 3D depth illusion 6. Apply smooth easing, lighting, and shadows for realism 7. Refine until it feels like one continuous scene Save this

FHILY👑

380,219 views • 2 months ago

🤯 Depth videos perfectly solve false flags on reference videos — and can perfectly recreate both martial arts and dance! Turned a Guan Dao martial arts clip into a Depth motion reference, swapped the fighter for a market auntie, and moved the scene to a T-junction inside a local market 🤣 🌟 Workflow: 1. Pick a reference clip under 15 seconds 2. Convert it to Depth with Depth Anything V2 3. Generate a new character + scene 4. Feed everything into Seedance with the prompt below 🌟 Depth video conversion: You can build a local Depth converter with Codex — prompt in the comments. This goes way beyond trending dances. Martial arts, polearms, and other complex movements can all be transferred cleanly with Depth! Workflow + Prompt below 👇

🤯 Depth videos perfectly solve false flags on reference videos — and can perfectly recreate both martial arts and dance! Turned a Guan Dao martial arts clip into a Depth motion reference, swapped the fighter for a market auntie, and moved the scene to a T-junction inside a local market 🤣 🌟 Workflow: 1. Pick a reference clip under 15 seconds 2. Convert it to Depth with Depth Anything V2 3. Generate a new character + scene 4. Feed everything into Seedance with the prompt below 🌟 Depth video conversion: You can build a local Depth converter with Codex — prompt in the comments. This goes way beyond trending dances. Martial arts, polearms, and other complex movements can all be transferred cleanly with Depth! Workflow + Prompt below 👇

Larus Canus

44,006 views • 2 days ago

I'm starting to get a hang of Kling 3, you can't do anything with fast movement, but for slower shots it's SOTA in quality, especially with my own special sauce.

I'm starting to get a hang of Kling 3, you can't do anything with fast movement, but for slower shots it's SOTA in quality, especially with my own special sauce.

Ian Sharar

46,749 views • 4 months ago

Create a unique-looking depth effect without adding any geometry to your scene in one click with Parallax It! add-on for Blender 🔶. Get it here: #b3d #blender #blender3d #blendercommunity #parallaxocclusion

Create a unique-looking depth effect without adding any geometry to your scene in one click with Parallax It! add-on for Blender 🔶. Get it here: #b3d #blender #blender3d #blendercommunity #parallaxocclusion

80 LEVEL

48,543 views • 2 years ago

YOYOOYO sliding in for #PortfolioDay !! I'm Simples, a 3D animator from Tennessee with nearly 4 years in the industry. I love playing with fast freaks, goofy creatures and anything that goes BOING BOING! I'm looking for work!! Gimme a holler: simplesthingss@gmail.com

YOYOOYO sliding in for #PortfolioDay !! I'm Simples, a 3D animator from Tennessee with nearly 4 years in the industry. I love playing with fast freaks, goofy creatures and anything that goes BOING BOING! I'm looking for work!! Gimme a holler: [email protected]

Simples

33,655 views • 1 year ago

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Ilir Aliu

53,911 views • 3 months ago

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

ハシモトタクマ / FunTech

27,192 views • 7 months ago

Saw a dot grid effect from Antonin Brener.work on Insta and wanted to create something similar for typing in an input. The input border effect is using border-beam by Jakub Antalik Try here:

Saw a dot grid effect from Antonin Brener.work on Insta and wanted to create something similar for typing in an input. The input border effect is using border-beam by Jakub Antalik Try here:

Zsolt Kacso

21,084 views • 2 months ago

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

MrNeRF

22,698 views • 1 year ago

From 30 years ago, a spacecraft I designed and rendered using POVRay for an animation I created portraying the diversion of a comet using a nuclear weapon. This wasn't designed in a modeling program, it was hand drafted and the geometry transcribed into POV Scene description language by hand.

From 30 years ago, a spacecraft I designed and rendered using POVRay for an animation I created portraying the diversion of a comet using a nuclear weapon. This wasn't designed in a modeling program, it was hand drafted and the geometry transcribed into POV Scene description language by hand.

Scott Manley

36,089 views • 5 months ago

Depth Anything 3 now runs as pure C++/ggml (ggml) . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !

Depth Anything 3 now runs as pure C++/ggml (ggml) . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !

Ettore Di Giacinto

34,581 views • 1 month ago

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Pablo Vela

29,904 views • 11 months ago

Rise and Bork !☀️ I've been experimenting with 3D programs since 2023 ! I recently got Rebocap and tested with a dance(ya ya ying) that didn't track so well ! I think XR Animator tracks really well with just using a not so good webcam. But with Rebocap you can really see the excitement I have for this dance ! I still used XR Animator for my finger tracking in the Rebocap one though.

Rise and Bork !☀️ I've been experimenting with 3D programs since 2023 ! I recently got Rebocap and tested with a dance(ya ya ying) that didn't track so well ! I think XR Animator tracks really well with just using a not so good webcam. But with Rebocap you can really see the excitement I have for this dance ! I still used XR Animator for my finger tracking in the Rebocap one though.

Kimii Kimura|❔🎀 Support Pup of the Nation

17,104 views • 6 months ago

TIL gradients can be sampled from an image in Blender. Here I'm using it with a volumetric shader. thx to Andrew Price & celestialmaze for the tip🙏 (btw we are adding this feature to #EmberGen too!)

TIL gradients can be sampled from an image in Blender. Here I'm using it with a volumetric shader. thx to Andrew Price & celestialmaze for the tip🙏 (btw we are adding this feature to #EmberGen too!)

Jason Key

57,771 views • 2 years ago

From the Rush Center for Congenital and Structural Heart Disease we present a 49 Y/O M with a large secundum ASD discovered after a stroke. Closed using a 44 mm Gore Cardioform ASD occluder guided by 4D ICE using NUVISION Catheter with bi-plane and 3D.

From the Rush Center for Congenital and Structural Heart Disease we present a 49 Y/O M with a large secundum ASD discovered after a stroke. Closed using a 44 mm Gore Cardioform ASD occluder guided by 4D ICE using NUVISION Catheter with bi-plane and 3D.

clifford jay kavinsky

16,950 views • 3 years ago

my fav play of the night. CC gets free from a great screen by MHA. finds KK in the lane. KK doesnt panic, passes to MHA. defense is in a scramble and MHA finds Sophie for a wide open for a 3. beautiful ball movement

my fav play of the night. CC gets free from a great screen by MHA. finds KK in the lane. KK doesnt panic, passes to MHA. defense is in a scramble and MHA finds Sophie for a wide open for a 3. beautiful ball movement

E.A.🤴🏾

29,318 views • 2 months ago

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

Alex

48,189 views • 4 months ago

Speaking of Dreams.. still kinda crazy to me how perfect of a match a tool like that is for generative AI workflows. Intuitive game engine/creative sandbox that allows you to build almost anything, running underneath a Gen AI render. Sculpting, painting, interacting with your two hands in 3d space, here using two move controllers which by now is tech from 2010. I haven't seen ANY way to interact with 3d that allows for the same fast and intuitive sculpting without forcing you to wear a VR headset. I made this scene a few years ago in 2 and a half hour or so as part of a 1 scene pr. episode of The Last of Us, and ran it through a combo of Runway and Sora to get to this result.

Speaking of Dreams.. still kinda crazy to me how perfect of a match a tool like that is for generative AI workflows. Intuitive game engine/creative sandbox that allows you to build almost anything, running underneath a Gen AI render. Sculpting, painting, interacting with your two hands in 3d space, here using two move controllers which by now is tech from 2010. I haven't seen ANY way to interact with 3d that allows for the same fast and intuitive sculpting without forcing you to wear a VR headset. I made this scene a few years ago in 2 and a half hour or so as part of a 1 scene pr. episode of The Last of Us, and ran it through a combo of Runway and Sora to get to this result.

Martin Nebelong

19,510 views • 10 months ago