Ok, this is absurd. You can choreograph a complex action scene in Blender with basic shapes, then let Seedance make it real. You need to try this AI filmmaking workflow: 1. Generate a start frame in Midjourney 2. Block out the action in Blender 3. Feed both to Seedance... My Blender reference was just rough timing, camera shake, and spatial choreography, and Seedance tracked the speed, motion, and action way better than I expected. This is the difference between describing a shot and directing one.show more

this largely describes how I see the direction of many creative fields going

100%. The era for generalists and creative directors.

This is incredible. How does Seedance know who is who? Do you designate some type of color marker for the shapes so it knows X shape/color = X character, etc.?

Thank you! Seedance is actually pretty good at just intuiting the space based on the start frame, but I also mention the colors of the boxes in my prompt. I posted my full prompt here:

Awesome, thanks!

Clever way of adding constraints to video generation to make it less sloppy.

+1 for creative constraints!

I know, right? > "Art lives from constraints and dies from freedom." - Leonardo da Vinci You might be interested in an article we wrote about this recently.

This is the way, VFX is truly getting disrupted

The good kind of disruption I feel. The future is bright!

Reid, this is all really cool - I've been messing a bit a Blender/Seedance work flow as well. I've been trying to find good ways to create the additional frames I need, you need good starter frames per shot and this tends to work really well. I had been doing more complicated blender mockups though and it's awesome to see how much easier I can probably make it. Thanks for sharing.

Yeah I actually find the less detail the better sometimes, so Seedance doesn't get confused. The blockout is really just for spatial reference.

@magneticskiff what's the actual prompt so that seedance "doesn't get creative" and offsets from the original camera, timing and framing?

@magneticskiff Posted my prompt here:

I wonder if we could use Claude opus 4.8 to write hyperframes scene reference using threejs and use it as reference. I know the whole point of using blender is for more control. Great job btw It opens up so much possibilités like using geometry nodes to make dr strange like vfx.

Probably yeah! I'm comfortable in Blender so I don't mind doing it the manual way but yeah you could prob automate it

your reference images must have been insane. i love the old western film style

Thanks! Yeah a bulk of the work here is spent on the start frames, and I’ve spent dozens of hours at this point curating moodboards and style refs for Midjourney

Do you know you can do all of this inside Blender? You just add a scene strip to the video editor and convert it to video either locally with the 3dreal lora or via fal + seedance/3dreal ltx. Pallaidium:

which prompt to use?

Here's my prompt for this one: Use [img1] as the clean start frame. Preserve the train, man, lighting, color, film grain, and composition from the start frame. Use [vid1] as a motion reference for camera choreography and timing. Match the timing and path of the camera move as closely as possible, including the shake, the pan right, the pan back, and the final push-in. The train is already at full speed on frame 1. Because the camera is attached to the side of the train, the train remains relatively stable while the environment, tracks, dirt, dust, and telegraph poles streak rapidly from right to left across the frame. The world rushes past at high speed. The railroad tracks, dirt, grass, dust, and nearby ground streak rapidly from right to left with strong horizontal motion blur. Railroad ties and foreground details whip past in a fast strobing blur. Distant mountains move much more slowly, creating strong parallax. Action: The man leaning out of the doorway ducks as a bullet strikes the side of the train, kicking up sparks and dust. He raises his pistol and fires several shots while the camera pans right. As the camera pans right, reveal a wild west sheriff on horseback in the same relative placement as the green box in [vid1]. The sheriff is galloping hard alongside the train. The landscape is still racing past. The sheriff gets shot and tumbles off the horse away from camera. The horse continues running. The camera then pans back to the man on the train and pushes forward toward him, keeping the same high-speed background streaking and heavy wind the entire time. Do not slow the train down. Do not make the train drift gently. Do not make the landscape move at walking speed. Do not make the shot feel like a slow scenic ride. Keep the train violently fast throughout. No music.

@Darriux3 Did you provide any images for character consistency, environment consistency? TIA.

@Darriux3 Not for these tests, since they’re just single shots and Seedance can handle consistency within the shot. But you’d need to if you were doing many shots of the same character.

@PhilipMPowell @Darriux3 What is "img1" referring to? The blender output is the video, got that part.

I always include a start frame in the generation, along with the blender camera reference. That way the look of the shot is locked in advance. I want complete control over the look of every shot, and I want the video model to make as little decisions as possible on its own. I actually build the blockout in blender to match the start frame, instead of the other way around, since Midjourney isn’t good yet at precision instructions.

@PhilipMPowell @Darriux3 Can we see the frame?

@PhilipMPowell @Darriux3 Oh sorry I posted the frame from one of my other videos just now haha. here is the train car's start frame

Amazing! Can you share how you modeled this in blender and what prompt you used?

Yeah! Nothing too crazy, just basic shapes in blender. Everything is just cubes. I used fSpy to match the camera's focal length from the start frame and then just roughly dropped in the cubes to match the shot. And here's my full prompt for this one:

Thanks man!!

You’re on a mfing roll

I’ve got a lot more in the oven 😎

the 'directing vs describing a shot' line is the whole shift in one sentence. blockout as spatial reference is clever

Truly game changing

wow, wow, wow! 🔥

What's your video prompt?

Posted the full prompt here:

Haha trying to make the most out of blender. Did you get this result at the first try though?

Admittedly no, this one took a whopping 14 generations to get it perfect, with me tweaking the prompt. The main issue was that the train didn't look like it was moving fast enough. Eventually I realized that instead of prompting for a fast train, describing the environment as rushing by, motion blur, etc worked better. Got there eventually! You can see my full prompt for this one here:

Is dope

tyty

Keep sharing plz and ty

This is exciting stuff!!!! Never heard of blender before, this is incredible!

World-class free and open source 3D software! Join the club :)

You had us at FREE!!!!

is this real

Yup!

So cool. As a beginner is this easy to do in blender? Any tip on how to start?

Blender can feel pretty intimidating for beginners but it's definitely doable! I would start with the basics and try to understand how the 3D viewport works and how to move shapes around. Then you need to understand how keyframes work to animation the camera. That's really all you need for this workflow.

I'm on it bro. I believe methods like these will distance AI films from AI slop.

Interesting 🤩👍

Will u share Youtube tutorial ?

Maybe if enough people want one :)

you should :)) I'm sure there will be a lot of people watching on YouTube.

Have been loving these posts! Curious what your timeline has been for each of these from midjourney framing through final lock?

Depends on the complexity of the shot but the total is probably something like 5 hours. I spend a decent chunk of time on the start frame, and I've done a lot of upfront work curating my moodboards and style references that do a lot of heavy lifting there. Then the blockout and camera animation in Blender can be simple or difficult depending on the shot, anywhere from like 30mins to 3 hours. Generating in Seedance usually requires multiple tries, tweaking the prompt, etc. Then I compile, add music and usually some more sound design.

Sick! Thank you for the breakdown🙏

Man I would love to show this as an example of how to properly use AI as an artist in my talks. Is that ok?

Sure thing :)

Wow seedance can do that ?

Indeed it can 😎

Blocking it in Blender first is the sane move. The AI gets way better when you stop asking it to invent physics from vibes.

This is crazy

thanks man

Wow! We can make a western in the comfort of our own home without ever filming in Arizona? Dang....!

amazing

@wdhartley Thank you!

Can you do this with something shot in a VR experience?

I don’t see why not!

I know what I’m doing this weekend

Send me what you make! I’m super interested in the VR space. Virtual filmmaking in VR sounds fascinating.

I’ve been working on this and thinking about it a lot! I basically see VR as a stage, but we need full control of the cameras and audio… then we can rip out and replace the visuals, map and characters. Here’s a thread I did to show some prototypes

Honestly mate, your job is amazing.But been stolen to Chinese APPs to make profit. So ashamed.

Very Impressive.

Nice

@Crypto_G8s 🤠

What was the Prompt to follow the action?

Posted my full prompt here:

I had even admired this earlier but didn't make the connection!

Hah all good man - there’s a lot of noise right now! 🤠 I appreciate the kind words