POM's banner
POM's profile picture

POM

@peterom15,737 subscribers

Open Source AI Art hypeman w/ @banodoco | Tools for artistic expression w/ @Reigh_Art | 👫🐕 w/ @hannahsubmarine. Aspiring 2nd Renaissance Man.

Shorts

The progress of the Animatediff community over the past 10 months has been miraculous - see attached! Now, closed startups like Krea are taking the fruits of all this effort - so I'd like to tell the story of how we got here & what people who believe in open source can do.

The progress of the Animatediff community over the past 10 months has been miraculous - see attached! Now, closed startups like Krea are taking the fruits of all this effort - so I'd like to tell the story of how we got here & what people who believe in open source can do.

166,604 Aufrufe

The meme-power of video-editing with Wan will be a force to be reckoned with Excellent examples by Illuminati Reptilien:

The meme-power of video-editing with Wan will be a force to be reckoned with Excellent examples by Illuminati Reptilien:

52,973 Aufrufe

Woke up today to see spacepxl trained a model on top of Wan to achieve SOTA performance in deblurring w/ an approach that will generalise to all kinds of video control tasks - upscaling, canny, video inpainting, etc. I love the open source AI art/banodoco community so much

Woke up today to see spacepxl trained a model on top of Wan to achieve SOTA performance in deblurring w/ an approach that will generalise to all kinds of video control tasks - upscaling, canny, video inpainting, etc. I love the open source AI art/banodoco community so much

45,609 Aufrufe

I believe that StoryDiffusion has the potential to be Animatediff's complex motion sister-model! While AD is amazing for granular control, micro-motion and all kinds of abstract motion, it fails at complex realistic motion - walking, human movements, cars, etc. StoryDiffusion seems very promising for this + also has characteristics that will likely make the community very receptive to it and likely to extend its capabilities: 2) Appealing base-model results - likely to get the community excited - feels like significantly better realistic motion than AD 2) Modular - their approach is built with a number of components that can be combined and taken apart - it works by generating consistent images, then animating them together - each of these stages can likely be upgraded, used and influenced in different ways. 3) Flexible - they demonstrate a bunch of different conditioning options 4) Likely easy on RAM - it's based on SD 1.5 + authors mention precautions to reduce RAM consumption 5) Built to plug into the existing ecosystem - e.g. the fact that it works with the SD1.5 ecosystem will give it a huge advantage! While it's very early to say - e.g. the video model hasn't even been released yet! - it does seem very promising. With 9 months of SD1.5/Animatediff-esque progress improving every element of it, I can see an an extremely extended version of this beating Sora + running for a fraction of the compute resources on a consumer GPU. Together with Animatediff to drive the micro-motions and abstract stuff, it could produce be extraordinary/otherworldly/insane/beautiful stuff. This is the first open video model I've been excited about since Animatediff - though cautiously optimistic! Link here:

I believe that StoryDiffusion has the potential to be Animatediff's complex motion sister-model! While AD is amazing for granular control, micro-motion and all kinds of abstract motion, it fails at complex realistic motion - walking, human movements, cars, etc. StoryDiffusion seems very promising for this + also has characteristics that will likely make the community very receptive to it and likely to extend its capabilities: 2) Appealing base-model results - likely to get the community excited - feels like significantly better realistic motion than AD 2) Modular - their approach is built with a number of components that can be combined and taken apart - it works by generating consistent images, then animating them together - each of these stages can likely be upgraded, used and influenced in different ways. 3) Flexible - they demonstrate a bunch of different conditioning options 4) Likely easy on RAM - it's based on SD 1.5 + authors mention precautions to reduce RAM consumption 5) Built to plug into the existing ecosystem - e.g. the fact that it works with the SD1.5 ecosystem will give it a huge advantage! While it's very early to say - e.g. the video model hasn't even been released yet! - it does seem very promising. With 9 months of SD1.5/Animatediff-esque progress improving every element of it, I can see an an extremely extended version of this beating Sora + running for a fraction of the compute resources on a consumer GPU. Together with Animatediff to drive the micro-motions and abstract stuff, it could produce be extraordinary/otherworldly/insane/beautiful stuff. This is the first open video model I've been excited about since Animatediff - though cautiously optimistic! Link here:

22,141 Aufrufe

Videos

Keine weiteren Inhalte verfügbar