正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

improving my 3D hand tracking template by adding depth-anything-v2 using mediapipe for X/Y movement and depth-anything for Z movement here i'm controlling a threejs scene in realtime with input from a single webcam

AA

25,902 subscribers

28,276 次观看 • 3 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

I created this using Claude + 3D animation: 1. Ask Claude for a scroll-based cinematic animation pla 2. Generate prompts for depth, parallax, and camera movement 3. Install Three.js + GSAP + ScrollTrigger 4. Use scroll to control zoom, scale, and scene transitions 5. Add layered images for 3D depth illusion 6. Apply smooth easing, lighting, and shadows for realism 7. Refine until it feels like one continuous scene Save this

I created this using Claude + 3D animation: 1. Ask Claude for a scroll-based cinematic animation pla 2. Generate prompts for depth, parallax, and camera movement 3. Install Three.js + GSAP + ScrollTrigger 4. Use scroll to control zoom, scale, and scene transitions 5. Add layered images for 3D depth illusion 6. Apply smooth easing, lighting, and shadows for realism 7. Refine until it feels like one continuous scene Save this

FHILY👑

380,219 次观看 • 2 个月前

I'm starting to get a hang of Kling 3, you can't do anything with fast movement, but for slower shots it's SOTA in quality, especially with my own special sauce.

I'm starting to get a hang of Kling 3, you can't do anything with fast movement, but for slower shots it's SOTA in quality, especially with my own special sauce.

Ian Sharar

46,749 次观看 • 4 个月前

Create a unique-looking depth effect without adding any geometry to your scene in one click with Parallax It! add-on for Blender 🔶. Get it here: #b3d #blender #blender3d #blendercommunity #parallaxocclusion

Create a unique-looking depth effect without adding any geometry to your scene in one click with Parallax It! add-on for Blender 🔶. Get it here: #b3d #blender #blender3d #blendercommunity #parallaxocclusion

80 LEVEL

48,543 次观看 • 2 年前

YOYOOYO sliding in for #PortfolioDay !! I'm Simples, a 3D animator from Tennessee with nearly 4 years in the industry. I love playing with fast freaks, goofy creatures and anything that goes BOING BOING! I'm looking for work!! Gimme a holler: simplesthingss@gmail.com

YOYOOYO sliding in for #PortfolioDay !! I'm Simples, a 3D animator from Tennessee with nearly 4 years in the industry. I love playing with fast freaks, goofy creatures and anything that goes BOING BOING! I'm looking for work!! Gimme a holler: [email protected]

Simples

33,655 次观看 • 1 年前

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Robots can now reconstruct 3D scenes in real time from a single RGB camera. [📍 Projects page + paper] No depth sensor. No retraining. 30 FPS. Researchers at the Imperial College London introduced KV-Tracker, a training-free method that makes heavy models like π³ and Depth Anything 3 fast enough for real-time tracking. The idea is simple. These models use global self-attention, which is powerful but computationally expensive. KV-Tracker caches the key and value pairs from selected keyframes and reuses them for new frames. That cache becomes an implicit scene representation. Result: • Up to 30 FPS • 10 to 15x speedup • Accurate 6-DoF tracking on benchmarks like TUM RGB-D and 7-Scenes • Works with monocular RGB only It also supports object-level tracking with masks and allows saving the KV-cache for later reuse. For robotics, this reduces hardware constraints and moves real-time 3D perception closer to practical deployment. Credit to Marwan Taher (Marwan Taher) at Imperial’s Dyson Robotics Lab and many others who contributed to this! 📍 Save projects page + paper for later: Video: ——- if it matters in AI or Robotics you'll read it here first:

Ilir Aliu

53,911 次观看 • 3 个月前

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

ハシモトタクマ / FunTech

27,192 次观看 • 7 个月前

Saw a dot grid effect from Antonin Brener.work on Insta and wanted to create something similar for typing in an input. The input border effect is using border-beam by Jakub Antalik Try here:

Saw a dot grid effect from Antonin Brener.work on Insta and wanted to create something similar for typing in an input. The input border effect is using border-beam by Jakub Antalik Try here:

Zsolt Kacso

20,928 次观看 • 2 个月前

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

MrNeRF

22,698 次观看 • 1 年前

From 30 years ago, a spacecraft I designed and rendered using POVRay for an animation I created portraying the diversion of a comet using a nuclear weapon. This wasn't designed in a modeling program, it was hand drafted and the geometry transcribed into POV Scene description language by hand.

From 30 years ago, a spacecraft I designed and rendered using POVRay for an animation I created portraying the diversion of a comet using a nuclear weapon. This wasn't designed in a modeling program, it was hand drafted and the geometry transcribed into POV Scene description language by hand.

Scott Manley

36,089 次观看 • 5 个月前

Depth Anything 3 now runs as pure C++/ggml (ggml) . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !

Depth Anything 3 now runs as pure C++/ggml (ggml) . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !

Ettore Di Giacinto

34,581 次观看 • 1 个月前

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Some updates on the multiview vistadream pipeline with Rerun! Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.

Pablo Vela

29,849 次观看 • 11 个月前

Rise and Bork !☀️ I've been experimenting with 3D programs since 2023 ! I recently got Rebocap and tested with a dance(ya ya ying) that didn't track so well ! I think XR Animator tracks really well with just using a not so good webcam. But with Rebocap you can really see the excitement I have for this dance ! I still used XR Animator for my finger tracking in the Rebocap one though.

Rise and Bork !☀️ I've been experimenting with 3D programs since 2023 ! I recently got Rebocap and tested with a dance(ya ya ying) that didn't track so well ! I think XR Animator tracks really well with just using a not so good webcam. But with Rebocap you can really see the excitement I have for this dance ! I still used XR Animator for my finger tracking in the Rebocap one though.

Kimii Kimura|❔🎀 Support Pup of the Nation

17,104 次观看 • 6 个月前

From the Rush Center for Congenital and Structural Heart Disease we present a 49 Y/O M with a large secundum ASD discovered after a stroke. Closed using a 44 mm Gore Cardioform ASD occluder guided by 4D ICE using NUVISION Catheter with bi-plane and 3D.

From the Rush Center for Congenital and Structural Heart Disease we present a 49 Y/O M with a large secundum ASD discovered after a stroke. Closed using a 44 mm Gore Cardioform ASD occluder guided by 4D ICE using NUVISION Catheter with bi-plane and 3D.

clifford jay kavinsky

16,950 次观看 • 3 年前

TIL gradients can be sampled from an image in Blender. Here I'm using it with a volumetric shader. thx to Andrew Price & celestialmaze for the tip🙏 (btw we are adding this feature to #EmberGen too!)

TIL gradients can be sampled from an image in Blender. Here I'm using it with a volumetric shader. thx to Andrew Price & celestialmaze for the tip🙏 (btw we are adding this feature to #EmberGen too!)

Jason Key

57,771 次观看 • 2 年前

my fav play of the night. CC gets free from a great screen by MHA. finds KK in the lane. KK doesnt panic, passes to MHA. defense is in a scramble and MHA finds Sophie for a wide open for a 3. beautiful ball movement

my fav play of the night. CC gets free from a great screen by MHA. finds KK in the lane. KK doesnt panic, passes to MHA. defense is in a scramble and MHA finds Sophie for a wide open for a 3. beautiful ball movement

E.A.🤴🏾

29,318 次观看 • 2 个月前

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

For the facial animations in my game, I use a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:

Alex

48,189 次观看 • 4 个月前

Speaking of Dreams.. still kinda crazy to me how perfect of a match a tool like that is for generative AI workflows. Intuitive game engine/creative sandbox that allows you to build almost anything, running underneath a Gen AI render. Sculpting, painting, interacting with your two hands in 3d space, here using two move controllers which by now is tech from 2010. I haven't seen ANY way to interact with 3d that allows for the same fast and intuitive sculpting without forcing you to wear a VR headset. I made this scene a few years ago in 2 and a half hour or so as part of a 1 scene pr. episode of The Last of Us, and ran it through a combo of Runway and Sora to get to this result.

Speaking of Dreams.. still kinda crazy to me how perfect of a match a tool like that is for generative AI workflows. Intuitive game engine/creative sandbox that allows you to build almost anything, running underneath a Gen AI render. Sculpting, painting, interacting with your two hands in 3d space, here using two move controllers which by now is tech from 2010. I haven't seen ANY way to interact with 3d that allows for the same fast and intuitive sculpting without forcing you to wear a VR headset. I made this scene a few years ago in 2 and a half hour or so as part of a 1 scene pr. episode of The Last of Us, and ran it through a combo of Runway and Sora to get to this result.

Martin Nebelong

19,510 次观看 • 10 个月前

🌟 "SHOULD I BUY AN IPHONE FOR TRACKING?" 🌟 tl;dr at bottom I've been using a facecam and Nvidia tracking for a long time and upgrading to a used iphone 13 combined with vbridger, the difference is HUGE. Here is my take on it! Why is Facecam > iPhone? ✅️ More affordable, esp for those using android phones ✅️ More convenient. If you launch Vtubestudio, there's a setting where your webcam automatically turns on, which is great. ✅️ Can track pretty well in the dark IF you already have a good webcam for night tracking. ❌️❌️ Stiff tracking at times ❌️ Not good at tracking specific mouth movement Why iPhone > Facecam? ✅️✅️ You can make the most of your model, since movement along the X and Y axes are a lot more accurate and wider. Also tracks eyes and overall face better. ✅️ More EXPRESSIONS. If your rigging allows for it, things like cheek puff, and tongue are able to be tracked. As far as I know, I cannot do this on facecam. ❌️❌️ WAY more expensive or requires that you have an iPhone already. Needs more set-up (need phone stand right in front of you, need to hook your phone up to a charger at all times, phone could possibly overheat as well if it's old, so you might need a cooler). TL;DR For me, if you have an extra 200 to spare for WAY better tracking, I would 1000% recommend buying a used iPhone on Amazon. iPhone X is the BARE minumum, I would recommend 12/13 so that your phone does not overheat. I do not need to use a cooler for my used iPhone 13. Being able to use my rigging to its fullest makes the model feel so so so different (in a good way). Feel free to reply with any questions, I will try to answer them!

🌟 "SHOULD I BUY AN IPHONE FOR TRACKING?" 🌟 tl;dr at bottom I've been using a facecam and Nvidia tracking for a long time and upgrading to a used iphone 13 combined with vbridger, the difference is HUGE. Here is my take on it! Why is Facecam > iPhone? ✅️ More affordable, esp for those using android phones ✅️ More convenient. If you launch Vtubestudio, there's a setting where your webcam automatically turns on, which is great. ✅️ Can track pretty well in the dark IF you already have a good webcam for night tracking. ❌️❌️ Stiff tracking at times ❌️ Not good at tracking specific mouth movement Why iPhone > Facecam? ✅️✅️ You can make the most of your model, since movement along the X and Y axes are a lot more accurate and wider. Also tracks eyes and overall face better. ✅️ More EXPRESSIONS. If your rigging allows for it, things like cheek puff, and tongue are able to be tracked. As far as I know, I cannot do this on facecam. ❌️❌️ WAY more expensive or requires that you have an iPhone already. Needs more set-up (need phone stand right in front of you, need to hook your phone up to a charger at all times, phone could possibly overheat as well if it's old, so you might need a cooler). TL;DR For me, if you have an extra 200 to spare for WAY better tracking, I would 1000% recommend buying a used iPhone on Amazon. iPhone X is the BARE minumum, I would recommend 12/13 so that your phone does not overheat. I do not need to use a cooler for my used iPhone 13. Being able to use my rigging to its fullest makes the model feel so so so different (in a good way). Feel free to reply with any questions, I will try to answer them!

Minori 🎀🍰💢 || bakaneko vampire :3

309,398 次观看 • 1 年前