Загрузка видео...

Не удалось загрузить видео

На главную

Everybody speak about consistent character for video generation as Minimax, Kling etc... But nothing looking super stable. And so I trained for the first time a LoRA model on Hunyuan video model (open source) on a real person. It take approximatly 15h of training on a A100 GPU with...

OdinLovis's profile picture

Lovis Odin

8,570 subscribers

93,560 просмотров • 1 год назад •via X (Twitter)

ОбразованиеНаука и технологииИгры
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Lovis
Lovis1 год назад

Only train on 20 epoch for now.. impatient to see what it give on 100

Фото профиля Freepik
Freepik1 год назад

🚀 Introducing Freepik AI video generator. Everything you need to create high-quality, physically accurate videos in one place. 🤩

Фото профиля Lovis
Lovis1 год назад

For a lot of people asking : I am training on 50 vidéos of 2 second ( 50 frames ) , 448 px height by 576 width . You Can find some content online but I am using this library for training

Фото профиля Moozi
Moozi1 год назад

This is great! So are you using the Lora in a text2video workflow or video2video?

Фото профиля Lovis
Lovis1 год назад

For now only text 2 video, but next step is video to video with a real moving character cutting across different scene , using a volumetric capture.

Фото профиля Michele Rocchi ⚛️🤖
Michele Rocchi ⚛️🤖1 год назад

do you mind if I post this on LinkedIn? I'll obviously cite you as the author, I'm not stealing anything. if you have LinkedIn, tell me your name and I'll tag you (the post will be in Italian)

Фото профиля Van
Van1 год назад

Question!! How much gpu vram ur computer has for this video, which laptop is better PC or Mac, & should I get a Nvidia 4090 or 3090 for comfyui video/image generator???

Фото профиля 𓆏
𓆏1 год назад

@AlfredAlfer77 create your own Will

Фото профиля Andre.W
Andre.W1 год назад

I think it's unnecessary to train with videos when it comes to a person, pictures are completely sufficient.

Фото профиля Ago-ra@AI
Ago-ra@AI1 год назад

Wooooow

Фото профиля ElAndrew73Gamer
ElAndrew73Gamer1 год назад

Can you show the flow of Comfy? I'm using it to create images with Flux, but I want to try it with videos. Thanks.

Похожие видео

I trained this LTX LTX 2.3 LoRA of George Costanza at home on my 5090 in about a day with AI Toolkit. I generated this 30 second video with ComfyUI on my 5090 in 6 minutes. Open source is, always has been, and always will be, the future of generative AI. (SOUND ON)
0:30

I trained this LTX LTX 2.3 LoRA of George Costanza at home on my 5090 in about a day with AI Toolkit. I generated this 30 second video with ComfyUI on my 5090 in 6 minutes. Open source is, always has been, and always will be, the future of generative AI. (SOUND ON)

Ostris

394,597 просмотров • 4 месяцев назад

Virtual try-on is the future of retail. Soon, we won’t even remember a time when we shopped without it. With Doji I created a fashion campaign with my favorite new products with myself as the model, an AI generated digital twin, and brought it to life additional AI video tools. Open to see more experiments and use cases for this technology:
0:50

Virtual try-on is the future of retail. Soon, we won’t even remember a time when we shopped without it. With Doji I created a fashion campaign with my favorite new products with myself as the model, an AI generated digital twin, and brought it to life additional AI video tools. Open to see more experiments and use cases for this technology:

Paige Piskin XR

52,666 просмотров • 1 год назад

This is crazy! With LTX 2 in ComfyUI i was able to keep my original camera movement + it did a full lip-sync on the singing haha. All running locally. I made the original video back in 2021 when Metahumans first came out and we were working in Unreal Engine 4. This music video was some kind of result from my first experiments with Livelink and Control Rig at that time. Now the new AI improved version is not perfect and i didn’t train any lora for character consistency yet. I also didn’t have the original video file anymore so I downloaded the Youtube version as input haha. However the camera result + facial expressions are really promising! I wonder how it would handle more complex and fast shots though.
1:38

This is crazy! With LTX 2 in ComfyUI i was able to keep my original camera movement + it did a full lip-sync on the singing haha. All running locally. I made the original video back in 2021 when Metahumans first came out and we were working in Unreal Engine 4. This music video was some kind of result from my first experiments with Livelink and Control Rig at that time. Now the new AI improved version is not perfect and i didn’t train any lora for character consistency yet. I also didn’t have the original video file anymore so I downloaded the Youtube version as input haha. However the camera result + facial expressions are really promising! I wonder how it would handle more complex and fast shots though.

Jerome | InsaneUnreal

41,375 просмотров • 6 месяцев назад

Big day for open source. MiniMax Music 3 is undeniably a state of the art open weights Music Generation model and a real alternative to Suno. My favorite part about open weights: the best is yet to come. Once the community starts playing with the model and training LoRAs, this model only gets better and allows for more control.
1:59

Big day for open source. MiniMax Music 3 is undeniably a state of the art open weights Music Generation model and a real alternative to Suno. My favorite part about open weights: the best is yet to come. Once the community starts playing with the model and training LoRAs, this model only gets better and allows for more control.

rob - comfyui

298,258 просмотров • 7 дней назад

🚨 AI just crossed into reality. A single human eye reflects an entire AI-generated world — futuristic cities, robots, and digital landscapes forming in real time. We are no longer imagining the future. We are actively creating it. Made with AI video generation (Kling model) on Yapper “This is not the future. It’s now.”
0:53

🚨 AI just crossed into reality. A single human eye reflects an entire AI-generated world — futuristic cities, robots, and digital landscapes forming in real time. We are no longer imagining the future. We are actively creating it. Made with AI video generation (Kling model) on Yapper “This is not the future. It’s now.”

Mimu | AI Tools & News

72,897 просмотров • 4 месяцев назад

We have a new open source video model. These clips were all generated with LTX-2 on the creator's local machine 🤯 It can make clips up to 20 seconds at 4K resolution. And it speaks! (created by u/yanokusnir)
0:47

We have a new open source video model. These clips were all generated with LTX-2 on the creator's local machine 🤯 It can make clips up to 20 seconds at 4K resolution. And it speaks! (created by u/yanokusnir)

Justine Moore

618,536 просмотров • 7 месяцев назад

AI is out of control 🤣 Wan 2.1 14B model is the best open-source video model I have ever used. Been trying it on Novita AI Playground, and it's fast and the quality is good. - Better than SORA and close to Google Veo-2. - Other models like DeepSeek R1, V3, Hunyuan Video and more available.. - API access too. More details 👇
0:35

AI is out of control 🤣 Wan 2.1 14B model is the best open-source video model I have ever used. Been trying it on Novita AI Playground, and it's fast and the quality is good. - Better than SORA and close to Google Veo-2. - Other models like DeepSeek R1, V3, Hunyuan Video and more available.. - API access too. More details 👇

AshutoshShrivastava

74,350 просмотров • 1 год назад

This is Pika 2.1 🤯 This is serendipitous as Pika was one of the first Gen AI video tools I ever used. The first time I generated a video on Discord, my life changed forever. And now they have released a game-changing new model with fantastic fidelity—native 1080p. We have arrived at the future of storytelling. And I'm glad they are part of our journey. Cheers.
0:48

This is Pika 2.1 🤯 This is serendipitous as Pika was one of the first Gen AI video tools I ever used. The first time I generated a video on Discord, my life changed forever. And now they have released a game-changing new model with fantastic fidelity—native 1080p. We have arrived at the future of storytelling. And I'm glad they are part of our journey. Cheers.

Dave Clark

22,341 просмотров • 1 год назад

A behind-the-scenes look at my process for creating this AI water video: On the left, I recorded myself performing all the actions and demonstrated some of my compositing techniques using masking and shape layers. On the right, you can see the final result after being generated in ComfyUI.
0:23

A behind-the-scenes look at my process for creating this AI water video: On the left, I recorded myself performing all the actions and demonstrated some of my compositing techniques using masking and shape layers. On the right, you can see the final result after being generated in ComfyUI.

enigmatic_e

145,346 просмотров • 2 лет назад

This is amazing research result! Xun Huang and team trained a bidirectional teacher model for motion control, and used Self-Forcing to make it autoregressive. Xun recently gave a great talk at the Real Time Video AI summit about this work, and a few other projects around video world models. Here is the recording.
29:48

This is amazing research result! Xun Huang and team trained a bidirectional teacher model for motion control, and used Self-Forcing to make it autoregressive. Xun recently gave a great talk at the Real Time Video AI summit about this work, and a few other projects around video world models. Here is the recording.

Eric is Daydreaming

52,796 просмотров • 9 месяцев назад

💄AI Video Model Comparison: Text to Video 10 Models included: • Google Veo2 • Adobe Firefly Video • Hunyuan Video • Pika 2.1 • Wan-2.1 • Runway Gen-3 • Kling AI 1.6 • Luma Ray2 • Hailuo MiniMax • Sora (not on grid, but in the full video) I used the same prompt on each model x 4 generations and chose my favorite results below. ⭕ Prompt: A person stands in a bright, airy room, bathed in soft natural light streaming through a large window adorned with sheer white curtains. As they gaze into a shimmering mirror, their hand, elegantly manicured with a deep crimson polish, lifts a glossy lip applicator towards their lips. Their outfit, a flowing pastel blouse paired with chic high-waisted trousers, exudes effortless style, complemented by delicate gold jewelry that sparkles subtly in the light. Applying lipstick, lip gloss, or any makeup seems to be extremely difficult for AI video models. It was hard to get a usable result for many of these. This prompt was generated on Nim with the Video Describe feature, and it worked better than my original prompt. Let me know what you think of the results!
1:10

💄AI Video Model Comparison: Text to Video 10 Models included: • Google Veo2 • Adobe Firefly Video • Hunyuan Video • Pika 2.1 • Wan-2.1 • Runway Gen-3 • Kling AI 1.6 • Luma Ray2 • Hailuo MiniMax • Sora (not on grid, but in the full video) I used the same prompt on each model x 4 generations and chose my favorite results below. ⭕ Prompt: A person stands in a bright, airy room, bathed in soft natural light streaming through a large window adorned with sheer white curtains. As they gaze into a shimmering mirror, their hand, elegantly manicured with a deep crimson polish, lifts a glossy lip applicator towards their lips. Their outfit, a flowing pastel blouse paired with chic high-waisted trousers, exudes effortless style, complemented by delicate gold jewelry that sparkles subtly in the light. Applying lipstick, lip gloss, or any makeup seems to be extremely difficult for AI video models. It was hard to get a usable result for many of these. This prompt was generated on Nim with the Video Describe feature, and it worked better than my original prompt. Let me know what you think of the results!

Heather Cooper

61,177 просмотров • 1 год назад

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models
0:30

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 просмотров • 1 год назад

Alright guys, wow. Qwen 3.8 27B is basically Qwen 4. What we have here is one of the best one-shot shark survival games I've made with any model. I ran this same prompt through fable, and the biggest thing all models struggle with in this test is the top-down view of the sharks. Fable even had all of the fins reversed and details jumbled on the first turn. Before now, it was really a 2 to 8 turn test to get something final regardless of model. Qwen 3.8 27B just nailed this in one output, no harness at all, just purely the model thoroughly thinking before outputting the final result. On every asset used, different shark variants, everything, it all looks incredibly tight. The boat movement, shark speed and engagement, everything is so much better than I expected, and I was expecting a lot. Imagine what 8 turns would do in a harness? This confirms to me that while Qwen 3.8 27B thinks a lot, for the first time in a local model on the base weights, IT MIGHT ACTUALLY BE WORTH IT This is a Frontier Lab model result, and it ran on my 5090 at 60tps in Q5_K_M. I am just in complete astoundment at what Qwen has accomplished here, and I am incredibly grateful that this model is open source. The speed will also increase drastically with optimisations. This is just an incredible day. What a blessing. More to come!
4:28

Alright guys, wow. Qwen 3.8 27B is basically Qwen 4. What we have here is one of the best one-shot shark survival games I've made with any model. I ran this same prompt through fable, and the biggest thing all models struggle with in this test is the top-down view of the sharks. Fable even had all of the fins reversed and details jumbled on the first turn. Before now, it was really a 2 to 8 turn test to get something final regardless of model. Qwen 3.8 27B just nailed this in one output, no harness at all, just purely the model thoroughly thinking before outputting the final result. On every asset used, different shark variants, everything, it all looks incredibly tight. The boat movement, shark speed and engagement, everything is so much better than I expected, and I was expecting a lot. Imagine what 8 turns would do in a harness? This confirms to me that while Qwen 3.8 27B thinks a lot, for the first time in a local model on the base weights, IT MIGHT ACTUALLY BE WORTH IT This is a Frontier Lab model result, and it ran on my 5090 at 60tps in Q5_K_M. I am just in complete astoundment at what Qwen has accomplished here, and I am incredibly grateful that this model is open source. The speed will also increase drastically with optimisations. This is just an incredible day. What a blessing. More to come!

Kyle Hessling

24,917 просмотров • 6 дней назад

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.
0:51

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,128 просмотров • 1 год назад

ELON: GROK 5 WILL UNDERSTAND REAL TIME VIDEO AND CREATE ITS OWN TOOLS “The quality of the data we are training on with Grok 5 is extraordinary. It is inherently multimodal with text, pictures, video, and audio. It is going to be much better at tool use and in fact at creating tools to be more effective at answering questions and understanding the world. Its vision will be extremely good. It will have real time video, which is fundamentally important." Source: Baron Capital
2:10

ELON: GROK 5 WILL UNDERSTAND REAL TIME VIDEO AND CREATE ITS OWN TOOLS “The quality of the data we are training on with Grok 5 is extraordinary. It is inherently multimodal with text, pictures, video, and audio. It is going to be much better at tool use and in fact at creating tools to be more effective at answering questions and understanding the world. Its vision will be extremely good. It will have real time video, which is fundamentally important." Source: Baron Capital

Mario Nawfal

148,015 просмотров • 9 месяцев назад

Introducing Lemon Slice Live: the first real-time audio-video AI model Anyone can video chat with any character by just uploading a single photo. Every pixel on the screen is rendered at 25fps in real-time by a transformer model. Here’s how we built it + a live demo! 👇
0:23

Introducing Lemon Slice Live: the first real-time audio-video AI model Anyone can video chat with any character by just uploading a single photo. Every pixel on the screen is rendered at 25fps in real-time by a transformer model. Here’s how we built it + a live demo! 👇

LemonSlice

171,007 просмотров • 1 год назад