Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

ReViV reconstructs viewer-centric human motion (body, hand, and gaze) and view-centric scene geometry (camera and depth) from a single egocentric RGB video in a unified feed-forward model. It formulates the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body... show more

AI Bites | YouTube Channel

2,433 subscribers

18,590 görüntüleme • 24 gün önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 görüntüleme • 1 yıl önce

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.

MrNeRF

24,729 görüntüleme • 1 yıl önce

🇨🇳 Another great Chinese Model, OmniHuman-1.5 from ByteDance Turns 1 image plus a voice track into expressive avatar video by pairing a System 1 and System 2 inspired planner with a Diffusion Transformer, Produces coherent motion for over 1 minute with moving camera and multi character scenes. Most avatar models move to the beat of the audio but miss meaning, so gestures feel generic and emotions feel shallow. The fix here is a Multimodal LLM planner that listens to the speech and drafts a structured plan describing intent, emotions, beats, and high level actions, which gives the motion engine clear semantic targets instead of only rhythm. The motion engine is a Multimodal Diffusion Transformer that fuses the plan with audio, the single reference image, and optional text prompts, then synthesizes continuous body, face, and head motion that matches both words and tone. A key trick is a Pseudo Last Frame, a synthetic target that summarizes the next expected state, which stabilizes fusion across modalities and keeps motion consistent over long spans. From just 1 image and speech, the system outputs speaking avatars with synchronized lips, context aware gestures, and continuous camera movement, and it also supports multi character interactions without manual choreography. Reported results show strong lip sync accuracy, high video quality, natural motion, and close match to text prompts, and the same setup works on nonhuman characters too.

Rohan Paul

63,859 görüntüleme • 11 ay önce

Single video → a reframeable 4D Gaussian Splatting scene. Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.

Single video → a reframeable 4D Gaussian Splatting scene. Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.

KIRI Engine - 3D Scanner App

30,603 görüntüleme • 1 ay önce

SAM 3D Body is a CVPR 2026 award candidate paper from AI at Meta model recovers a full 3D human body mesh from a single RGB image you can run it automatically, or guide the reconstruction with masks and 2D keypoints thx to Niels Rogge for awesome demo idea

SAM 3D Body is a CVPR 2026 award candidate paper from AI at Meta model recovers a full 3D human body mesh from a single RGB image you can run it automatically, or guide the reconstruction with masks and 2D keypoints thx to Niels Rogge for awesome demo idea

SkalskiP

72,500 görüntüleme • 2 ay önce

🤯 Depth can now recreate complex climbing and parkour motion this cleanly! Turned a fast climbing clip into a Depth motion reference, then swapped in a 20-year-old twin-tail student with a backpack while keeping the same concrete structure. Wall contact, weight shifts, jumps, climbing paths, and full-body movement all stay surprisingly clean and consistent! 🌟 Workflow: 1. Pick a reference clip under 15 seconds 2. Convert it to Depth with Depth Anything V2 3. Generate a new character + scene 4. Feed everything into Seedance with the prompt below 🌟 Depth video conversion: You can build a local Depth converter with Codex — prompt in the comments. Dance, martial arts, climbing, parkour, and other complex movements can all be transferred cleanly with Depth! Workflow + Prompt below 👇

🤯 Depth can now recreate complex climbing and parkour motion this cleanly! Turned a fast climbing clip into a Depth motion reference, then swapped in a 20-year-old twin-tail student with a backpack while keeping the same concrete structure. Wall contact, weight shifts, jumps, climbing paths, and full-body movement all stay surprisingly clean and consistent! 🌟 Workflow: 1. Pick a reference clip under 15 seconds 2. Convert it to Depth with Depth Anything V2 3. Generate a new character + scene 4. Feed everything into Seedance with the prompt below 🌟 Depth video conversion: You can build a local Depth converter with Codex — prompt in the comments. Dance, martial arts, climbing, parkour, and other complex movements can all be transferred cleanly with Depth! Workflow + Prompt below 👇

Larus Canus

106,715 görüntüleme • 27 gün önce

Tried generating a short cinematic scene with CapCut Seedance 2.5, and the results were impressive. The AI kept the character consistent across shots, produced clean camera movement, and delivered polished visuals from just a single prompt. Looking forward to the full release. #CapCut #Seedance25 #CreateWithCapCut

Tried generating a short cinematic scene with CapCut Seedance 2.5, and the results were impressive. The AI kept the character consistent across shots, produced clean camera movement, and delivered polished visuals from just a single prompt. Looking forward to the full release. #CapCut #Seedance25 #CreateWithCapCut

Sani Ai Tech

19,729 görüntüleme • 23 gün önce

THE DEPTH MAP TRICK THAT FIXED DANCE ACCURACY IN SEEDANCE 2.0 Feed the model a video of someone dancing and it tries to interpret everything- the person, the clothes, the lighting, the room, and somewhere in there, the movement. Feed it a depth map and there's nothing left to interpret but the motion. Most creators trying to transfer a dance to a character reference the source footage directly, then wonder why the choreography drifts. The problem isn't the model - it's that you handed it ten variables when you only wanted one. Here's the workflow 1. Lock the character reference in GPT Image 2 first -face, build, costume, so identity holds independently of whatever motion gets applied to it 2. Convert the source dance footage into a depth map instead of using the raw video -this strips out the original performer's appearance, clothing, and environment entirely 3. Feed the depth map as the motion reference and the character sheet as the identity reference- two separate inputs doing two separate jobs, not one input trying to do both 5. Let the depth map carry only spatial movement -the model receives body position and momentum with no competing information about who's moving or what they look like 6. Keep the character and motion inputs isolated throughout - the moment you mix appearance data into the motion reference, the model starts negotiating between two identities Why this works • Raw footage passes the model everything at once- performer, wardrobe, room, lighting -and the choreography competes with all of it for attention • A depth map is pure spatial information, so the only thing left to transfer is movement • Separating identity from motion means the character can stay locked while the dance stays accurate - normally you're trading one for the other • The accuracy gain isn't the model getting better, it's the model getting fewer decisions to make Use cases: ⁃ Dance and choreography transfer onto original characters ⁃ Motion capture-style workflows without motion capture ⁃ Any sequence where a specific movement needs to survive intact ⁃ Character showcase content built on existing performance footage The character sheet answers who's dancing. The depth map answers how - and keeping those two questions separate is the whole trick.

THE DEPTH MAP TRICK THAT FIXED DANCE ACCURACY IN SEEDANCE 2.0 Feed the model a video of someone dancing and it tries to interpret everything- the person, the clothes, the lighting, the room, and somewhere in there, the movement. Feed it a depth map and there's nothing left to interpret but the motion. Most creators trying to transfer a dance to a character reference the source footage directly, then wonder why the choreography drifts. The problem isn't the model - it's that you handed it ten variables when you only wanted one. Here's the workflow 1. Lock the character reference in GPT Image 2 first -face, build, costume, so identity holds independently of whatever motion gets applied to it 2. Convert the source dance footage into a depth map instead of using the raw video -this strips out the original performer's appearance, clothing, and environment entirely 3. Feed the depth map as the motion reference and the character sheet as the identity reference- two separate inputs doing two separate jobs, not one input trying to do both 5. Let the depth map carry only spatial movement -the model receives body position and momentum with no competing information about who's moving or what they look like 6. Keep the character and motion inputs isolated throughout - the moment you mix appearance data into the motion reference, the model starts negotiating between two identities Why this works • Raw footage passes the model everything at once- performer, wardrobe, room, lighting -and the choreography competes with all of it for attention • A depth map is pure spatial information, so the only thing left to transfer is movement • Separating identity from motion means the character can stay locked while the dance stays accurate - normally you're trading one for the other • The accuracy gain isn't the model getting better, it's the model getting fewer decisions to make Use cases: ⁃ Dance and choreography transfer onto original characters ⁃ Motion capture-style workflows without motion capture ⁃ Any sequence where a specific movement needs to survive intact ⁃ Character showcase content built on existing performance footage The character sheet answers who's dancing. The depth map answers how - and keeping those two questions separate is the whole trick.

Nexlow

85,251 görüntüleme • 1 ay önce

Split-screen test on Flux 3! Prompt : Split-screen video showing the same real-time action from two different camera angles. The screen is divided vertically into two equal halves. On the left side, show a static, wide overhead view from a ceiling-mounted camera in the back corner of the room. The full room layout is visible: a couch, coffee table, rug, bookshelves, and a tall shelf near the wall. The cat is perched on top of the shelf. At the 2-second mark, it suddenly jumps, flips mid-air above the furniture, and lands gracefully on the coffee table. On the right side, show the same action in perfect sync from a side, eye-level camera placed low beside the couch. The view includes the shelf, the space between, and the coffee table. The camera follows the cat’s leap, capturing body motion, twist, and landing.Both sides must be precisely synchronized, showing the exact same event from different angles with consistent lighting, physics, and timing.

Split-screen test on Flux 3! Prompt : Split-screen video showing the same real-time action from two different camera angles. The screen is divided vertically into two equal halves. On the left side, show a static, wide overhead view from a ceiling-mounted camera in the back corner of the room. The full room layout is visible: a couch, coffee table, rug, bookshelves, and a tall shelf near the wall. The cat is perched on top of the shelf. At the 2-second mark, it suddenly jumps, flips mid-air above the furniture, and lands gracefully on the coffee table. On the right side, show the same action in perfect sync from a side, eye-level camera placed low beside the couch. The view includes the shelf, the space between, and the coffee table. The camera follows the cat’s leap, capturing body motion, twist, and landing.Both sides must be precisely synchronized, showing the exact same event from different angles with consistent lighting, physics, and timing.

Umesh

87,894 görüntüleme • 25 gün önce

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

名画を立体的に鑑賞する方法 Nano Banana Pro (深度/法線マップ生成)→ WebGL(Three.js) スマホでも体験できます👉 【プロンプト】（法線マップ用） Generate a surface normal map from the input artwork image. Output one RGB image where each pixel encodes the surface normal direction in standard tangent-space format: •R = X direction •G = Y direction •B = Z direction Requirements: •Maintain the same resolution and aspect ratio as the input image. •Normal directions must follow the local geometry implied by brush strokes, shading, and contours. •Do NOT add new artistic elements or repaint the scene. •Only infer plausible micro-geometry and surface orientation. •Ensure smooth, consistent normal transitions without noise. （深度マップ用） Generate a depth map from the input artwork image. Output a single grayscale depth map where: •White = near to the viewer •Black = far from the viewer Requirements: •Maintain the same resolution and aspect ratio as the input image. •Depth must be smooth, continuous, and physically plausible. •Infer only geometric depth that could reasonably exist in the original artwork. •Do NOT add new objects or details. •Do NOT stylize, repaint, or reinterpret the scene. •Avoid texture noise; focus solely on depth structure.

ハシモトタクマ / FunTech

27,192 görüntüleme • 8 ay önce

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 görüntüleme • 1 yıl önce

MiniMax H3 Character Introduction Prompt: Use @[char ref] as the sole character reference. Preserve the exact identity, face, body proportions, hairstyle, outfit, colors, materials and overall silhouette of the character throughout the entire video. Do not redesign, simplify or replace any defining visual features. Create a cinematic character introduction focused on presence, silhouette, attitude and controlled motion. 0–4s Begin with a close shot of a defining lower-body or detail element such as boots, shoes, feet, hands, clothing hem or an important accessory. The character enters frame or settles into position. The camera slowly tracks upward while hair, clothing and secondary elements move naturally in the wind or environment. 4–8s Reveal more of the body with a medium or medium-wide shot from the back, side or three-quarter angle. The character stands in a calm, composed way inside the environment. The camera makes a smooth orbit, arc or lateral move to gradually reveal the character’s face and silhouette. 8–12s Move into a tight cinematic portrait or upper-body shot. The character performs one subtle signature action that fits their personality, such as lifting the chin, turning the head, adjusting clothing, brushing hair aside, opening a hand, looking toward camera, or shifting posture. Keep the motion minimal and intentional. The expression should match the character’s vibe. 12–15s End with a strong full-body hero shot that clearly presents the entire design and silhouette. Use a low-angle, eye-level or slightly dramatic framing depending on the character’s personality. The character settles into a natural final pose and holds it confidently for a clean final reveal. VISUAL DIRECTION Premium cinematic presentation. Match the visual medium and rendering style of @[char ref]. Emphasize clean silhouette, elegant staging, subtle secondary motion, believable hair and cloth movement, strong composition, atmospheric depth and polished lighting. The scene should feel like a high-end anime, game or film character introduction. CAMERA Use a clear progression from detail reveal to partial reveal to face reveal to full-body hero reveal. Camera movement should be smooth, controlled and intentional. Avoid chaotic motion. ENVIRONMENT Place the character in a fitting environment that supports their identity and mood. The background should enhance the character without distracting from them.

MiniMax H3 Character Introduction Prompt: Use @[char ref] as the sole character reference. Preserve the exact identity, face, body proportions, hairstyle, outfit, colors, materials and overall silhouette of the character throughout the entire video. Do not redesign, simplify or replace any defining visual features. Create a cinematic character introduction focused on presence, silhouette, attitude and controlled motion. 0–4s Begin with a close shot of a defining lower-body or detail element such as boots, shoes, feet, hands, clothing hem or an important accessory. The character enters frame or settles into position. The camera slowly tracks upward while hair, clothing and secondary elements move naturally in the wind or environment. 4–8s Reveal more of the body with a medium or medium-wide shot from the back, side or three-quarter angle. The character stands in a calm, composed way inside the environment. The camera makes a smooth orbit, arc or lateral move to gradually reveal the character’s face and silhouette. 8–12s Move into a tight cinematic portrait or upper-body shot. The character performs one subtle signature action that fits their personality, such as lifting the chin, turning the head, adjusting clothing, brushing hair aside, opening a hand, looking toward camera, or shifting posture. Keep the motion minimal and intentional. The expression should match the character’s vibe. 12–15s End with a strong full-body hero shot that clearly presents the entire design and silhouette. Use a low-angle, eye-level or slightly dramatic framing depending on the character’s personality. The character settles into a natural final pose and holds it confidently for a clean final reveal. VISUAL DIRECTION Premium cinematic presentation. Match the visual medium and rendering style of @[char ref]. Emphasize clean silhouette, elegant staging, subtle secondary motion, believable hair and cloth movement, strong composition, atmospheric depth and polished lighting. The scene should feel like a high-end anime, game or film character introduction. CAMERA Use a clear progression from detail reveal to partial reveal to face reveal to full-body hero reveal. Camera movement should be smooth, controlled and intentional. Avoid chaotic motion. ENVIRONMENT Place the character in a fitting environment that supports their identity and mood. The background should enhance the character without distracting from them.

Kōda

101,450 görüntüleme • 9 gün önce

Only one Adobe Firefly video prompt from our research hit production-ready on the first generation. Prompt in comments 👀 It looks you have to direct the camera as much as the model to get the most out of Firefly. Every other video session in our research needed a revision. Revising the first frame is where Firefly broke: >motion failures (temporal flicker, motion continuity, physics in motion) carried 60% of video's issue mix once designers re-prompted. TLDR: in Adobe Firefly, the camera does a lot of the work. Bake it into the first prompt and the video lands. Its by far the best time to be a creative.

Only one Adobe Firefly video prompt from our research hit production-ready on the first generation. Prompt in comments 👀 It looks you have to direct the camera as much as the model to get the most out of Firefly. Every other video session in our research needed a revision. Revising the first frame is where Firefly broke: >motion failures (temporal flicker, motion continuity, physics in motion) carried 60% of video's issue mix once designers re-prompted. TLDR: in Adobe Firefly, the camera does a lot of the work. Bake it into the first prompt and the video lands. Its by far the best time to be a creative.

ben @ CVPR

12,847 görüntüleme • 2 ay önce

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 görüntüleme • 1 yıl önce

I've been working a lot with SAM3 and the Momentum Human Rig (MHR). I finally integrated it into the data I'm working with Rerun. The progression I've taken looks as follows SAM3 + SAM3D-body on 1. a single image 2. a set of multiple images 3. a single video 4. A multiview video capture I took inspiration from the SAM3D-body paper and built a multiview fitting optimization pipeline. This pipeline involves using the 2D keypoints from the single-view pipeline, triangulating them, and employing an L1 loss between the 2D/3D keypoints. The temporal stability isn't great, so that's the next portion I'm going to focus on. One really frustrating thing about SAM3D-body is the lack of per-joint confidence values. It makes it harder to deal with occlusions. I'm probably going to need to use a separate model, or maybe add a confidence head.

I've been working a lot with SAM3 and the Momentum Human Rig (MHR). I finally integrated it into the data I'm working with Rerun. The progression I've taken looks as follows SAM3 + SAM3D-body on 1. a single image 2. a set of multiple images 3. a single video 4. A multiview video capture I took inspiration from the SAM3D-body paper and built a multiview fitting optimization pipeline. This pipeline involves using the 2D keypoints from the single-view pipeline, triangulating them, and employing an L1 loss between the 2D/3D keypoints. The temporal stability isn't great, so that's the next portion I'm going to focus on. One really frustrating thing about SAM3D-body is the lack of per-joint confidence values. It makes it harder to deal with occlusions. I'm probably going to need to use a separate model, or maybe add a confidence head.

Pablo Vela

42,267 görüntüleme • 7 ay önce

🔥 AI video creation is finally evolving beyond basic motion prompts. Try out Depth Map Control right here: Historically, the hardest part of the process has been maintaining scene consistency: • camera movement • object placement • spatial relationships Thanks to PixVerse Depth Map Control, you can now grab depth data from a reference video and merge it with a fresh image to explore highly directed video generation. The final outcome? Much smoother camera tracking. Highly cohesive visual scenes. It is a powerful workflow that hands creators actual authority over how their AI visuals shift and flow over time. 🎬

🔥 AI video creation is finally evolving beyond basic motion prompts. Try out Depth Map Control right here: Historically, the hardest part of the process has been maintaining scene consistency: • camera movement • object placement • spatial relationships Thanks to PixVerse Depth Map Control, you can now grab depth data from a reference video and merge it with a fresh image to explore highly directed video generation. The final outcome? Much smoother camera tracking. Highly cohesive visual scenes. It is a powerful workflow that hands creators actual authority over how their AI visuals shift and flow over time. 🎬

Elara Quinn

12,202 görüntüleme • 17 gün önce

We are pleased to announce the availability of Stable Video 4D, our very first video-to-video generation model that allows users to upload a single video and receive dynamic novel-view videos of eight new angles, delivering a new level of versatility and creativity. In conjunction with this announcement, we are releasing a comprehensive technical report detailing the methodologies, challenges, and breakthroughs achieved during the development of this model. Learn more about this release and access the report here:

We are pleased to announce the availability of Stable Video 4D, our very first video-to-video generation model that allows users to upload a single video and receive dynamic novel-view videos of eight new angles, delivering a new level of versatility and creativity. In conjunction with this announcement, we are releasing a comprehensive technical report detailing the methodologies, challenges, and breakthroughs achieved during the development of this model. Learn more about this release and access the report here:

Stability AI

131,173 görüntüleme • 2 yıl önce

30 minutes of video. Robot learns the task. Open-source, end-to-end. An open-source framework for training robot policies from only 30 minutes of human egocentric videos captured via Meta Aria glasses: Achieving zero-shot transfer to robots without any robot data collection. The method relies on Interaction-Centric Tokens that encode hand-object spatial relationships invariant to embodiment and viewpoint, supplemented by auxiliary objectives like object motion prediction and latent consistency to extract richer supervision signals from the same data. HumanEgo demonstrates strong cross-embodiment, cross-environment performance on bimanual tasks, outperforming baselines like ACT and teleop data while being trainable on a single RTX 4090 GPU. Thanks for sharing, Zhi (Leo) Wang. 📌 Website: Paper: Code: Video: ——- Weekly robotics and AI insights. Subscribe free:

30 minutes of video. Robot learns the task. Open-source, end-to-end. An open-source framework for training robot policies from only 30 minutes of human egocentric videos captured via Meta Aria glasses: Achieving zero-shot transfer to robots without any robot data collection. The method relies on Interaction-Centric Tokens that encode hand-object spatial relationships invariant to embodiment and viewpoint, supplemented by auxiliary objectives like object motion prediction and latent consistency to extract richer supervision signals from the same data. HumanEgo demonstrates strong cross-embodiment, cross-environment performance on bimanual tasks, outperforming baselines like ACT and teleop data while being trainable on a single RTX 4090 GPU. Thanks for sharing, Zhi (Leo) Wang. 📌 Website: Paper: Code: Video: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

17,077 görüntüleme • 2 ay önce

Video generated by Seedance 2.0 on Lart AI Try it there 👇🏻 Prompt 10-Second Cinematic Timeline Prompt — "The Train Escape" 0–2s | The Robbery A crowded 19th-century train station during golden hour. A masked thief steals a mysterious black briefcase and runs toward a departing steam train. Crowds scatter in panic. Cinematic lighting, realistic motion, dramatic atmosphere. Camera: Wide establishing shot → handheld chase shot. --- 2–4s | The Pursuit A fearless young female detective wearing a black trench coat sprints after the thief through the crowded platform, jumping over luggage carts and avoiding passengers. Steam fills the air as the train begins moving. Dialogue: "Stop! Drop the case!" Camera: Dynamic tracking shot with cinematic motion blur. --- 4–6s | The Leap The thief jumps onto the moving steam train. The detective runs at full speed and leaps onto the train moments before it leaves the station. Sparks fly from the tracks. Camera: Slow-motion side tracking shot transitioning to normal speed. --- 6–8s | The Fight On the roof of the speeding train, the detective and thief engage in an intense hand-to-hand fight. Strong wind blows their clothes as steam and sparks surround them. Mountains rush past in the background. Camera: Fast cinematic action shots with dramatic close-ups. --- 8–10s | The Victory The detective defeats the thief, recovers the briefcase, and stands on top of the speeding train as the sun sets behind distant mountains. The thief lies defeated. Camera: Epic wide aerial shot slowly pulling back. Atmosphere: Heroic, cinematic, action movie ending, ultra-realistic, 4K.

Video generated by Seedance 2.0 on Lart AI Try it there 👇🏻 Prompt 10-Second Cinematic Timeline Prompt — "The Train Escape" 0–2s | The Robbery A crowded 19th-century train station during golden hour. A masked thief steals a mysterious black briefcase and runs toward a departing steam train. Crowds scatter in panic. Cinematic lighting, realistic motion, dramatic atmosphere. Camera: Wide establishing shot → handheld chase shot. --- 2–4s | The Pursuit A fearless young female detective wearing a black trench coat sprints after the thief through the crowded platform, jumping over luggage carts and avoiding passengers. Steam fills the air as the train begins moving. Dialogue: "Stop! Drop the case!" Camera: Dynamic tracking shot with cinematic motion blur. --- 4–6s | The Leap The thief jumps onto the moving steam train. The detective runs at full speed and leaps onto the train moments before it leaves the station. Sparks fly from the tracks. Camera: Slow-motion side tracking shot transitioning to normal speed. --- 6–8s | The Fight On the roof of the speeding train, the detective and thief engage in an intense hand-to-hand fight. Strong wind blows their clothes as steam and sparks surround them. Mountains rush past in the background. Camera: Fast cinematic action shots with dramatic close-ups. --- 8–10s | The Victory The detective defeats the thief, recovers the briefcase, and stands on top of the speeding train as the sun sets behind distant mountains. The thief lies defeated. Camera: Epic wide aerial shot slowly pulling back. Atmosphere: Heroic, cinematic, action movie ending, ultra-realistic, 4K.

Stacy Lee 🇺🇸

22,899 görüntüleme • 1 ay önce