跳过导航
Learning Temporally Consistent Video Depth from Video Diffusion Priors
0:00 / 1:25

Learning Temporally Consistent Video Depth from Video Diffusion Priors

AK

@_akhaliq53万 位关注者

在 X 上查看
372 个赞下载原帖

81,666 次观看

Learning Temporally Consistent Video Depth from Video Diffusion Priors This work addresses the challenge of video depth estimation, which expects not only per-frame accuracy but, more importantly, cross-frame consistency. Instead of directly

9 条评论

评论来自 X (Twitter) 上的原帖

  • AK 的头像

    @_akhaliq

    developing a depth estimator from scratch, we reformulate the prediction task into a conditional generation problem. This allows us to leverage the prior knowledge embedded in existing video generation models, thereby reducing learn- ing difficulty and enhancing

    2 个赞

  • AK 的头像

    @_akhaliq

    generalizability. Concretely, we study how to tame the public Stable Video Diffusion (SVD) to predict reliable depth from input videos using a mixture of image depth and video depth datasets. We empirically confirm that a procedural training strategy - first optimizing the

    2 个赞

  • AK 的头像

    @_akhaliq

    spatial layers of SVD and then optimizing the temporal layers while keeping the spatial layers frozen - yields the best results in terms of both spatial accuracy and temporal consistency. We further examine the sliding window strategy for inference on arbitrarily long

    2 个赞

  • AK 的头像

    @_akhaliq

    videos. Our observations indicate a trade-off between efficiency and performance, with a one-frame overlap already producing favorable results. Extensive experimental results demonstrate the superiority of our approach, termed ChronoDepth, over existing

    3 个赞

  • AK 的头像

    @_akhaliq

    alternatives, particularly in terms of the temporal consistency of the estimated depth. Additionally, we highlight the benefits of more consistent video depth in two practical applications: depth-conditioned video generation and novel view synthesis.

    2 个赞

  • AK 的头像

    @_akhaliq

    paper page:

    16 个赞

  • AK 的头像

    @_akhaliq

    daily papers:

    4 个赞

  • Katy 的头像

    @NFkTys

    @NandoDF Amazing progress 🚀🔥

  • Abhinav Girdhar 的头像

    @AbhinavGirdhar

    Great insights on the new features and security measures! It's good to see continuous improvements in user experience and safety. Looking forward to more updates.