Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

SCoPE Sightline-Coordinate Positional Encoding for Video Diffusion Transformers model:

AK

530,834 subscribers

24,274 views • 2 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

5 Comments

Jane Austen2 months ago

SCoPE looks really interesting for video diffusion. Better positional encoding could make a big difference in how models handle motion and spatial consistency across longer video sequences. Definitely one to watch. 🔥

César Bonilla2 months ago

Mamma mia

Michael Yin2 months ago

Thanks for sharing SCoPE! The interactive demo is now live with few-step inference. Huggingface Space: TencentARC/scope-camera-video-generation

HU Wenbo2 months ago

Thx for sharing Code and weights are up, Apache-2.0:

dhinna ship .ico2 months ago

Does SCoPE's coordinate scheme reduce drift on clips longer than 16 frames?

Related Videos

"Cameras as Relative Positional Encoding" TLDR: comparison for conditioning transformers on cameras: token-level raymap, attention-level relative pose encodings, a (new) relative encoding Projective Positional Encoding -> camera frustums, (int|ext)insics for relative pos encoding

"Cameras as Relative Positional Encoding" TLDR: comparison for conditioning transformers on cameras: token-level raymap, attention-level relative pose encodings, a (new) relative encoding Projective Positional Encoding -> camera frustums, (int|ext)insics for relative pos encoding

Alexandre Morgand

17,837 views • 1 year ago

Meta presents Adaptive Caching for Faster Video Generation with Diffusion Transformers

Meta presents Adaptive Caching for Faster Video Generation with Diffusion Transformers

AK

53,129 views • 1 year ago

I spent months illustrating how Transformers actually work. Not just what they do, but why they’re built this way. The history, design choices, and intuition behind every layer. From RNNs → Attention → Multi-Head → FFNs → Positional Encoding. Here's everything I wish I knew:

I spent months illustrating how Transformers actually work. Not just what they do, but why they’re built this way. The history, design choices, and intuition behind every layer. From RNNs → Attention → Multi-Head → FFNs → Positional Encoding. Here's everything I wish I knew:

krupa

251,821 views • 1 year ago

📢📢📢 𝐀𝐂𝟑𝐃: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers TL;DR: for 3D camera control in generative video, it really helps knowing *which* part of your model you should mess with Internship by Sherwin Bahmani at Snap

📢📢📢 𝐀𝐂𝟑𝐃: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers TL;DR: for 3D camera control in generative video, it really helps knowing which part of your model you should mess with Internship by Sherwin Bahmani at Snap

Andrea Tagliasacchi

23,088 views • 1 year ago

FasterCache Training-Free Video Diffusion Model Acceleration with High Quality

FasterCache Training-Free Video Diffusion Model Acceleration with High Quality

AK

91,309 views • 2 years ago

Epona Autoregressive Diffusion World Model for Autonomous Driving

Epona Autoregressive Diffusion World Model for Autonomous Driving

AK

18,839 views • 1 year ago

A new frontier. Project Starlight—the first and only diffusion-based AI model for enhancing video. Launching soon.

A new frontier. Project Starlight—the first and only diffusion-based AI model for enhancing video. Launching soon.

Topaz Labs

22,375 views • 1 year ago

We introduce W.A.L.T, a diffusion model for photorealistic video generation. Our model is a transformer trained on image and video generation in a shared latent space. 🧵👇

We introduce W.A.L.T, a diffusion model for photorealistic video generation. Our model is a transformer trained on image and video generation in a shared latent space. 🧵👇

Agrim Gupta

431,358 views • 2 years ago

AnyFlow Any-Step Video Diffusion Model with On-Policy Flow Map Distillation

AnyFlow Any-Step Video Diffusion Model with On-Policy Flow Map Distillation

AK

14,152 views • 5 months ago

Video diffusion models generate high-quality videos but are too slow for interactive applications. We MIT CSAIL Adobe Research introduce CausVid, a fast autoregressive video diffusion model that starts playing the moment you hit "Generate"! A thread 🧵

Video diffusion models generate high-quality videos but are too slow for interactive applications. We MIT CSAIL Adobe Research introduce CausVid, a fast autoregressive video diffusion model that starts playing the moment you hit "Generate"! A thread 🧵

Tianwei Yin

84,074 views • 1 year ago

damn,.... this is so incredibly cool use case for discrete diffusion model

damn,.... this is so incredibly cool use case for discrete diffusion model

Simo Ryu

2,558,114 views • 1 year ago

📢Excited to be at #ICLR2025 for our paper: VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Poster: Thu 3-5:30 PM (#134) Website: Code: Also check out our #CVPR2025 follow-up AC3D:

📢Excited to be at #ICLR2025 for our paper: VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Poster: Thu 3-5:30 PM (#134) Website: Code: Also check out our #CVPR2025 follow-up AC3D:

Sherwin Bahmani

11,643 views • 1 year ago

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,881 views • 2 years ago

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

Chongjie Ye

28,181 views • 10 months ago

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

263,035 views • 3 years ago