This is getting ridiculous. You need to try this AI filmmaking workflow. My Blender reference is embarrassingly simple. You can drive a precise multi-character shot with nothing but basic shapes. 1. Generate a start frame in Midjourney 2. Block out the scene in Blender with boxes, animate the camera... 3. Feed both to Seedance This feels like a real unlock. Your 3D reference doesn't need to look good.show more

Pro tip: Give the boxes hex colors and then tell seedance that each character corresponds with their hex color. This works with one plate shot, but once you have the characters as ingredients you need to color code the boxes :)

Nice yeah this is a great idea. I actually did reference the colors from the viewport in my latest shot I posted today!

the boxes outperforming actual mannequins is the part that breaks my brain. feels like the model wants ambiguity to fill in rather than something it has to respect

Yeah for sure. The model has trouble differentiating between direction in the prompt (telling the characters to move) and a static mannequin (telling the characters to stay still). Simple boxes avoids conflicting direction.

Which platform are you using for seedance? In my experience its not enough to use Seedance you have to use the correct Ui like Artlist, Higgsfield, LUMA for certain things. Would love to know what you are using please.

I'm using @magnific !

@Cole_A_Higgins @magnific I'm also using magnific but I couldn't figure out how to load a video clip.

@Cole_A_Higgins @magnific Magnific calls it “references” :) there’s a spot to upload right below the place you can select a start and end frame. And keep in mind the model that works with references is separate from the model that works with start and end frames - you cannot have both.

@Cole_A_Higgins @magnific Great, man! Thanks for your response. I appreciate how generous you are with your time and your knowledge!

@Cole_A_Higgins @magnific 💪✨

Actually, this hybrif technique is a way more convenient compromise than just prompting. This avoids the prompting nightmare. Great stuff.

Totally agree. Prompting alone is not the way to work creatively!

I’m loving this example. I was worried I’d have to work too hard in blender, defeating the purpose of AI. How did you prompt for the blocks to be recognized as the characters?

Honestly it just worked. The boxes are placed exactly to match the characters in the start frame, and seedance just intuits whats what. I think understanding the camera movement and relation in 3D space is more important than "this block is this character" You can actually see the move isn't quite as accurate as my previous tests - it's close but deviates a little in the middle. Probably because my blockout was less accurate.

Do you think this is possible with other models, not just Seedance?

Try it and report back!

okay that’s it. im downloading blender

join the blender cult

Can you maintain character voices across different shots and scenes?

Yeah you can upload sample audio reference for the characters and it does a pretty good job matching. But I think the best workflow would be to record the dialogue in advance with a real human and upload that.

that's interesting! Did you get it right in the first go?

Usually takes a few generations to nail the take but it’s mostly because of small tweaks to the performances and stuff. It’s pretty great at matching the camera reference first try.

Damn, the AI revived Mario Almada

😂

Can you continue the scene, from a different camera angel or another character joining in or one of them leaving etc?

Yeah sometimes I pull start frames for subsequent shots from a Seedance master. I also use Nano Banana Pro and GPT-2 to make other angles.

this is fucking cool man

This is genuinely incredible. I think this is the inflection point where ai stops being as sloppy and starts being more controllable.

I agree. We’ve definitely crossed some threshold. We need as much control as possible

Sick workflow. Maybe I can combine that with a virtual camera to do quicker camera control references. Really cool for people doing large AI-based productions.

Like the 3D camera mapped to your phone with motion control, yeah? Excellent idea.

I love this. Reduce it down to the minimum effective input I always found image models did better with composition reference as an example

Yeah interestingly using boxes also prevents Seedance from getting confused about the characters being static. It's more likely to move them around than when I was using actual mannequins.

@brandonwilson so the simpler the better? if so that’s perfect for me lol.

@brandonwilson Seems that way! But it also means giving up more control to the video model. I'm going to keep testing - my goal is to maximize control while keeping the input resources "cheap"

Wow

This is so encouraging!

How much it costs and what do you have to use ?

Blender is free and open source! Seedance can be a bit pricey depending on your shot duration, resolution, etc. @mitte_ai is a great platform for the AI part of the process.

just prompt it bro

Not enough control with text prompts!

skill issue

😂

Any reason for Midjourney specifically

Best aesthetics of any image gen model by far. I can't get anywhere close to Midjourney for cinematic looking stuff with Nano Banana, ChatGPT, even Krea.

While this is the way forward (v2v references), I want to see prompts because Seedance is pretty terrible with prompt adherence. This is typically why glazing posts never show prompts.

Here is my exact prompt for this one:

Yeah I saw that, thanks. How would you rate the adherence? It seems to have left out a lot of the character actions in the end there.

You got this just with a starting image and this blender animation? How detailed was your prompt?

I posted my exact prompt here :)

Love the way people are figuring this stuff out. I have Blender integrated into Claude. Thinking of asking it to do the basic block design you show with a characters reference frame, then just go in and do the camera move. Could shave time off perhaps.

very nice shot!

@JSFILMZ0412 @Ludedwolf see

I really need to be testing more , never in my life would I have thought that goddamn blocks would perform as well as mannequins

It’s been like this since you could run video into AI. Now it’s just more refined on the AI side of the spectrum.

Wow

brilliant interworking

Wow!!

This is definitely the future. Is it hard to learn blender tho?

Blender can feel pretty intimidating but I'm not doing anything too crazy in this example! Just basic shapes and keyframing the camera.

Awesome work regardless brother. Would love to see how you face bigger clips, or a project that demands consistency. You have like a yt channel or something like that?

Not yet! But soon :)

Great workflow

Very cool. How do you drive the camera in the Blender reference to begin with (ie control where it looks, how fast it moves)?

It's manual keyframe animation!

None of it looks good mate.

Wow that’s so cool! 🔥

Thank you!