AI Toolkit now officially supports training Lightricks LTX-2 LoRAs.

I will work on getting some docs and a tutorial put together today/tomorrow, but basically for the dataset, just make sure it is trimmed to the proper length for 24 fps. Frame count is (seconds * 24) + 1. So 5 second videos is 121 frames. 5 * 24 + 1 = 121

If you appreciate my work or find it useful, please consider contributing.

These are the important settings to train on 5 second videos on a 5090 with 64 GB of CPU ram.

@Lightricks

@Lightricks Amazing, thanks for your hard work with all the models !!!

@Lightricks I'm just about to install LTX-2 into my new rig with a 5090. So far I'm terrible at all this but I want to lean how to train it. Where do I begin?

@Lightricks Thank you for your work!

@Lightricks Would be great if AI Toolkit could work with comfyui models and not diffusers. It is painful to download all these shards.

@Lightricks Pure power! Amazing job! 🔥

@Lightricks Can we train on 30 fps ?

@Lightricks Currently it is hardcoded to 24fps in order to sync the audio. Also, temporal compression is a factor of 8, so anything other than 8, 16, 24, 32, etc is not possible.

@Lightricks I see

@Lightricks Fantastic , started my first lora with 300 pictures , gpu load is around 13% on a Blackwell 6000 (7.5s/Iter) , other models are at 100% . Any idea ?

@Lightricks Cache latents (pull latest first) and text embeddings, turn off offloading.

@Lightricks thank you 🙏

@Lightricks YES

@Lightricks And VRAM considerations for a RTX 6000? Or do i not need to worry about it. Also can you do character lora's with images only?

@Lightricks Well done.👍 how long ?

@Lightricks nice

Hello, I have a question. When I train with images, the results are very good. However, when I train with videos, I found a problem. The audio fits faster, but the video part hasn't been fully fitted yet. If I continue training, LoRa will force it to read according to the audio text, completely ignoring the prompt words. But if the number of steps is not enough, the audio can be controlled, but the training of video, such as face recognition, hasn't been fully fitted. Currently, when the number of steps is not particularly high, I can only reduce the weight of LoRa, but the effect is still not very good because the model weights of audio and video cannot be separated. What are your good suggestions

@Lightricks You could try training with both an image dataset and a video dataset at the same time.

@Lightricks thanks,let me try

@Lightricks "Ostris Ai Toolclit" My brain is playing tricks on me.

@Lightricks Legend!

@Lightricks Holy shit

@Lightricks Can I train Lora on rtx 5080 + 64 GB RAM?

@Lightricks I am getting a dataset error. It says there are no images, but I have a dataset of 14 videos with captions in the dataset. Any idea? : AssertionErrorAssertionError: : no images found in /app/ai-toolkit/datasets/loratestno images found in /app/ai-toolkit/datasets/loratest

@Lightricks On the dataset, you have to designate number of frames. If it is 1, it looks for images, if it is more than 1, it looks for videos. I assume you have it set to 1.

@Lightricks yep. that was 100% it! Sorry to take your time with such a newb question. Sending $20 your way via PayPal when I cash out from Patreon this evening. I truly appreciate your hard work!

@Lightricks Thank you. No issue at all, I need to surface that error, better, you are not the first today to run into it.