正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

I've been putting together an interactive tool called DiffusionLab for explaining the geometric intuition behind diffusion and flow based generative models. Sampling is actually being done in the browser using Tensorflow.js! It is still in the very early stages.

Alec Helbling

10,370 subscribers

104,626 次观看 • 1 年前 •via X (Twitter)

艺术科学技术教育

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

I made a tool called Diffusion Explorer that lets you to train and visualize simple 2D diffusion and flow models live in the browser. You can draw your own distributions and observe how the generated samples converge during training. Try it live 👇

Alec Helbling

73,076 次观看 • 1 年前

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber

Still one of my fav projects I worked on in the day job – an interactive experience showing low-carbon technology for the home Custom 3D models created in Blender 🔶 and everything is rendered in the browser using Three.js and the awesome react-three-fiber

Robin Hawkes

26,525 次观看 • 3 年前

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

AK

25,368 次观看 • 2 年前

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Flow matching produces smooth, deterministic trajectories. In contrast, the sampling process of a diffusion model is chaotic, resembling the random motion of gas particles.

Alec Helbling

428,834 次观看 • 1 年前

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.

The latent space of earlier generative models like GANS can linearly encode concepts of the data. What if the data was model weights? We present weights2weights, a subspace in diffusion weights that behaves as an interpretable latent space over customized diffusion models.

Amil Dravid

94,226 次观看 • 2 年前

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:

There's a lot of both 2D and 3D effort that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:

Kurzgesagt

100,313 次观看 • 7 个月前

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:

This symmetric diffusion paper at ICLR is nice (simple idea in retrospect): SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models We'd actually implemented this idea internally at Orbital, and it works nicely even for very large crystal structures:

Mark Neumann

18,188 次观看 • 1 年前

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D

relating to Stocky, this is the game they're being made for! The game is called Meleeniacs and its in VERY early development. This is all subject to change, but I'm open to any questions about the plans for this game :D

not yeucc

110,408 次观看 • 10 个月前

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.

This is a fun interactive article explaining the intuition behind Gaussian Processes. Gaussian Processes allow you to construct a distribution over functions with a continuous domain. They are incredibly useful for applications like Bayesian regression.

Alec Helbling

41,360 次观看 • 1 年前

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.

We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI. 📖 Read the notes here: Joint work with Ezra Erives.

Peter Holderrieth

473,950 次观看 • 2 个月前

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.

Keir Starmer explaining how shunting off your kids early to a Breakfast Club is, wait for it, "bringing families and children together across the country". Clearly he isn't all there in the head.

MrBounceBack.com

15,065 次观看 • 2 天前

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.

Putting something together in NYC on April 21st. World models, policy learning, robotics - if you're building at the frontier - DM anyone on the General Intuition team to join.

paula wehmeyer

38,652 次观看 • 1 个月前

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

Stochastic and deterministic sampling strategies for diffusion models produce strikingly different trajectories, but both ultimately achieve the same aim. I had a great time presenting our work, Diffusion Explorer, this week at IEEE VIS in Vienna.

Alec Helbling

279,002 次观看 • 6 个月前

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇

CanvAi is live on Product Hunt 😸 today 🎉 A canvas-based image generation tool built on tldraw. Sketch, composite, and iterate using the latest image models. Check it out 👇

Charlie Clark

13,412 次观看 • 5 个月前

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.

Ask Claude to document and describe the main flows in your app and output in a single page html + json data file. Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.

Dave Jeffery

482,379 次观看 • 26 天前

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.

DID YOU KNOW: An official sequel for Monolith's 'Claw' was in the works called Claw 3D (or Claw 2). It was drafted in 3D as an early prototype for the LithTech engine, with footage shown in an engine trailer. With being sold off, the original game has never been seen.

Lost Media Busters

12,910 次观看 • 11 个月前

I found a much cleaner way to browse the newly released UFO files. The site is called Show Me UFOs, and it is significantly easier to use than the Department of War 🇺🇸 release page. You can sort the files by document type, agency, date, and even explore them through an interactive map. For anyone trying to actually dig through this material instead of just skim headlines, this is a genuinely useful tool. Link is in the attached comment. #ufox #ufotwitter

I found a much cleaner way to browse the newly released UFO files. The site is called Show Me UFOs, and it is significantly easier to use than the Department of War 🇺🇸 release page. You can sort the files by document type, agency, date, and even explore them through an interactive map. For anyone trying to actually dig through this material instead of just skim headlines, this is a genuinely useful tool. Link is in the attached comment. #ufox #ufotwitter

Tom Thompson🛸 (ＣＯＲＴＥＸＺＥＲＯ)

31,024 次观看 • 28 天前

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘

For the Hoseok girlies who've been down bad and putting in the work for so long—this era is your reward. Soak it in. You’ve earned it. 😘

Hoseokiepedia

19,234 次观看 • 1 年前