Загрузка видео...

Не удалось загрузить видео

На главную

I've been putting together an interactive tool called DiffusionLab for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.

alec_helbling's profile picture

Alec Helbling

10,370 subscribers

104,626 просмотров • 1 год назад •via X (Twitter)

ИскусствоНаука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇
0:15

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

Alec Helbling

73,076 просмотров • 1 год назад

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber
0:18

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber

Robin Hawkes

26,525 просмотров • 3 лет назад

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.
0:19

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

AK

25,368 просмотров • 2 лет назад

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.
0:13

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Alec Helbling

428,834 просмотров • 1 год назад

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.
0:16

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.

Amil Dravid

94,226 просмотров • 2 лет назад

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:
0:16

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:

Kurzgesagt

100,348 просмотров • 7 месяцев назад

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:
0:16

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:

Mark Neumann

18,188 просмотров • 1 год назад

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D
0:16

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D

not yeucc

110,408 просмотров • 10 месяцев назад

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.
0:18

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.

Alec Helbling

41,360 просмотров • 1 год назад

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.
0:15

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.

Peter Holderrieth

474,010 просмотров • 2 месяцев назад

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.
0:10

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.

MrBounceBack.com

15,357 просмотров • 6 дней назад

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.
0:14

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.

paula wehmeyer

38,652 просмотров • 2 месяцев назад

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.
0:14

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

Alec Helbling

279,011 просмотров • 7 месяцев назад

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.
0:18

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.

Dave Jeffery

483,208 просмотров • 1 месяц назад

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇
0:19

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇

Charlie Clark

13,412 просмотров • 5 месяцев назад

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.
0:15

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.

Lost Media Busters

12,910 просмотров • 11 месяцев назад

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘
0:19

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘

Hoseokiepedia

19,234 просмотров • 1 год назад

Amazon said it’s in the "very, very early stages" of testing this robot in its facilities
0:18

Amazon said it’s in the "very, very early stages" of testing this robot in its facilities

World of Engineering

547,043 просмотров • 2 лет назад