I've done so many scrubbed video sequences on scroll over the years, that I thought it would be helpful to share some pro tips in this thread:

Generally the process looks like this: 1. Get your source video at 1.2-1.5x the width where it will be used, compress using a tool like 2. Use ffmpeg to cut the video into frames. Make sure to have a different frame set for desktop and mobile.

3. Do whatever you can do decrease the frame file size. If you have some frames that don't require transparency, they should be jpeg/webp. 4. Load said frames as blob images using a javascript webworker (see image) which moves the fetching and decoding off the main thread.

5. Once you have all your images back from the worker, draw the first frame to your canvas and watch for the section to be in view. 6. While in view, determine the percentage of the animation you've scrolled through, making sure to not draw the same frame twice in a row.

And that's pretty much it in terms of implementation. But it's worth mentioning that there are some optimizations that can happen before even getting to development. For example, consider advising your art directors to prioritize:

1. Shots with simple color palettes, avoiding complex image data. A mostly-black frame will be significantly smaller than a frame from a video with grass and leaves.

1a. Sequences that focus on the main subject.. Try to remove all background objects & scenery (replace with a solid color) 2. Smooth movements that can be cleanly expressed with lower framerates. Lots of fast movements/cuts require framerates that are hard to tie to scroll.

This is great! A video tutorial would be awesome, do you have one?

Nice tips, it comes down to loading speed and every small detail is important

Very insightful thank you! In your experience, is using a video file an option? I'm often battling weird behaviors/animation getting stuck when scrubbing a video

Wow, I've implemented this countless times, but I've learned a few tricks from your insights. Thanks for sharing!