Loading video...

Video Failed to Load

Go Home

I've been putting together an interactive tool called DiffusionLab for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.

alec_helbling's profile picture

Alec Helbling

10,370 subscribers

104,626 views • 1 year ago •via X (Twitter)

ArtsScience & TechnologyEducation
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇
0:15

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

Alec Helbling

73,076 views • 1 year ago

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber
0:18

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber

Robin Hawkes

26,525 views • 3 years ago

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.
0:19

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

AK

25,368 views • 2 years ago

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.
0:13

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Alec Helbling

428,834 views • 1 year ago

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.
0:16

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.

Amil Dravid

94,226 views • 2 years ago

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:
0:16

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:

Mark Neumann

18,188 views • 1 year ago

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:
0:16

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:

Kurzgesagt

100,348 views • 7 months ago

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D
0:16

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D

not yeucc

110,408 views • 10 months ago

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.
0:18

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.

Alec Helbling

41,360 views • 1 year ago

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.
0:10

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.

MrBounceBack.com

15,357 views • 6 days ago

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.
0:15

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.

Peter Holderrieth

474,010 views • 2 months ago

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.
0:14

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.

paula wehmeyer

38,652 views • 2 months ago

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.
0:14

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

Alec Helbling

279,011 views • 7 months ago

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.
0:18

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.

Dave Jeffery

483,208 views • 1 month ago

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇
0:19

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇

Charlie Clark

13,412 views • 5 months ago

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.
0:15

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.

Lost Media Busters

12,910 views • 11 months ago

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘
0:19

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘

Hoseokiepedia

19,234 views • 1 year ago

I found a much cleaner way to browse the newly released UFO files. The site is called Show Me UFOs, and it is significantly easier to use than the Department of War 🇺🇸 release page. You can sort the files by document type, agency, date, and even explore them through an interactive map. For anyone trying to actually dig through this material instead of just skim headlines, this is a genuinely useful tool. Link is in the attached comment. #ufox #ufotwitter
0:19

I found a much cleaner way to browse the newly released UFO files. The site is called Show Me UFOs, and it is significantly easier to use than the Department of War 🇺🇸 release page. You can sort the files by document type, agency, date, and even explore them through an interactive map. For anyone trying to actually dig through this material instead of just skim headlines, this is a genuinely useful tool. Link is in the attached comment. #ufox #ufotwitter

Tom Thompson🛸 (ＣＯＲＴＥＸ ＺＥＲＯ)

31,764 views • 1 month ago