The previs workflow is still leveling up! My AI filmmaking process is starting to feel less like wrangling AI and more like animation. Still just cubes in Blender, but now I’m moving the character through space and doing a handheld camera follow. Same basic process: 1. Generate a start... frame in Midjourney 2. Block out the scene in Blender, animate the camera 3. Feed both to Seedance Seedance nailed the gritty handheld feel and followed the character movement around the car way better than expected. You can get way more ambitious with your directing than I thought.show more

My Blender blockout is ridiculously simple. It's just cubes and a mannequin. You don't need to master 3D to do this stuff. Just learn the basics. This time I hooked up my phone to Blender as a virtual camera to record the move. I'll post more about this later. It's insanely fun.

I'm using @mitte_ai to generate in Seedance 2.0. It's one of the cleanest platforms I've worked with. Feels like a real production tool. I had to generate 16 takes to perfect the shot, iterating on the prompt. In the end, I actually spliced together two of my favorite takes for the final. Can you find the cut?

Everyone always asks for my Seedance prompt, so here it is! No secrets 😎 Use [image1] as the clean start frame. Don't change anything about the frame. Use [video1] as a camera motion reference. Follow the camera movement in the reference exactly, including the handheld motion following the girl walking around the car and then the pan left. A young girl stands at a payphone in the South Western United States. She hears trucks approaching from the distance and quickly glances towards the sound, and says "Shit." She drops the phone and quickly scurries around to the driver's side of the car. The camera follows behind her with shaky handheld. The camera whip pans to the left, and we see the side of a dusty highway winding into the desert, with mountains in the far distance. Two pickup trucks approach and pull off from the road driving towards us, skidding and kicking up dust from the ground. No music.

The virtual camera thing really helps sell this as more realistic. Well done, Reid!

Thanks Christopher!

No problem. How hard is that to set up in blender btw? Is it just a plugin?

VirtuCamera? Super easy. Yeah it’s a (paid) app for your phone and then on the blender side it’s an addon you install to make the link. Takes like 2 seconds to set up.

Awesome, will check it out. Thank you for the heads up!

いいですね！ ハンドヘルドカメラは、手付けでアニメーションつけて再現してるんですか？それとも、何かツールをつかって、カメラのモーションをキャプチャーしてるのでしょうか？

For this one I actually used VirtuCamera to record the camera animation with my phone, so the handheld motion is all real! But when I'm animating the camera with keyframes in Blender (like the train car scene) I add camera shake with Ian Hubert's Shakify addon.

VirtuCameraですね！ありがとうございます！

Yes! VirtuCamera2 - developer is "The Weird Byte"

はわっ！ありがとう、探してみます🙇🙇🙇🙇🙇

Thanks for this

No problem. More on the way!

Did you keyframe the handheld movement? Motion track?

For this shot I used VirtuCamera to connect my phone to blender and record the move! But usually I just keyframe the camera smoothly, and add handheld camera shake on top as a constraint layer.

Awesome, thanks for sharing I’ll try that out.

This is 95% there! The only thing I’d add to the prompt is to maintain strict spatial and environmental consistency. In this case, the phone booth in the beginning wasn’t facing the car, but when the camera turned around it switch the orientation of the booth. Besides that, great work!!!!!!

Yeah it’s definitely not perfect! Part of the problem too is when the blockout doesn’t spatially match exactly, which is often the case since I’m just approximating it. It’s pretty good at hitting the parks in 2D space but in 3D it’s a little more unpredictable.

Seedance is excellent; I believe it’s the best, but I don’t understand who their target audience is given the price.

they're targeting studios. they don't care about retail

I get that Studio have the budget, but people are still uncomfortable with AI in mainstream cinema. Just like it was with coding back then. It takes time for trust to build. I think they might need a broader acceptance first if they aim to target these studios

The studios are already working with AI. Google just invested 75 million into A24!

This is great but the only issue is that, using a image as a start frame in reference modes kills the quality as opposed to using it as a native start frame

100%. The fidelity is way better when you use the dedicated start frame version of the model. But since you can't upload any reference materials that way it's kind of a no go for me. I assume this will improve in the future

Bytedance actually is fixing that in 2.5 they are adding a 3d reference mode aswell you can upload with it the only way right now it looks good is if you separate all elements but you wont get that start frame look and the motion will be all over the place

This could also be a great use case for VR. We once made a VR universe where you could build anything around you with simple shapes.

Yeah virtual filmmaking in VR sounds super interesting to me

How'd you get the handheld shake in Blender?

For this shot I hooked up my phone to blender with VirtuCamera and recorded the move with my phone! So all the handheld shake is real. But usually I add handheld camera shake with Ian Hubert's shakify addon.

Ah I didn't know that was an option on Blender. Wow!

It's a third party addon but yes!

Using basic cubes in Blender just to block camera movement before feeding it to Seedance is such a smart low effort workflow

Haha thanks! Rather than thinking of it as low effort, I’m looking for ways to spend more time on my shots, and exert even more control haha. I don’t want to use AI so I can spend less effort - I want to use it as a multiplier to make more/better work with the same amount of effort :)

that mindset is what separates good results from great ones 🎬

Thanks for showing the process! I feel like I need to see what happens next. Which means you got the emotion of the scene right.

That’s the goal :) appreciate you Gabriel

How many time to create this blender action?

I probably spent like 1.5 hours on the blender portion

Amazing. Thanks to reply.

Are you able to have character consistency throughout several scenes aswell? Any tips?

Yeah! Seedance 2.0 allows you to upload up to 12 multi-modal references, so you can make a character sheet and upload that. Also when I'm building a scene I make all my start frames manually, so I ensure they're cohesive with each other before I even get to the video model.

So cool! Feels like puppeteering and filming with mobile phone would we even quicker and easier?

That's exactly what I did for this one! It's super fun.

Pretty cool. Love seeing updates to your workflow.

New experiments coming every day 🤝

Hey, bro, I must say that you really helped me a lot.

@LSYCJ888 Glad to hear it!

I mean that's seriously impressive because so much is left out. Did you mean for there to be two identical trucks at the end? Do you also provide references for your actor, truck style, environment, etc?

Thank you! I gave Seedance a start frame, so the environment and the woman were set by that. In this case I didn’t upload a reference for the trucks, I just put a description in the prompt. Normally I’d upload object reference sheets too though if I needed consistency between multiple shots

Very cool workflow.

Animate the camera?

Yup! With keyframes

That' is absolutely mind boggling, love the spatial reference only approach. Not overbuilt. can do this in after effects as well. :) Gonna have some fun.

Send me what you end up making!

I'm really drawn to Red Doll, Grey Checkerboard s01e01

there's a lot of lore to be explored

Amazing how just a tiny bit of human soul kills that slop vibe!

we're pro ai and anti slop here!

This is very interesting. I thought the previs had to inform the AI of emotion-driven body language. However you're prompting it, the AI seems to be doing quite well humanizing the character's movement. Thanks for sharing, Reid!

Yeah totally! I mainly use the previs for the camera movement and spatial relationships, and then prompt the performance.

Why though? In all seriousness, I've yet to hear from anyone why this is a good thing.

Why is what a good thing specifically? Doing a 3D blockout? Because it gives you total control over the timing, composition, and cinematography of the scene. Or do you mean why AI at all? Broader discussion than just one post but I would say this tech is going to lower the barrier of entry to more creators that otherwise wouldn’t be able to produce the resources necessary to make films, which is an extremely expensive medium. More people being able to realize their vision, more stories that otherwise would never have been told. The medium is a means to an end and I’m excited about a future where more people can make things.

I primarily mean AI. I read many justifications about its use in pre-visualization. But how much fidelity is required before principal photography? Are you not simply parodying a computer generation? And if the goal is to replace cinematography, lighting, actors, costumes, hair & makeup, set decoration, physical locations, et al., is this really a future you would wish to inhabit? The absence of communal artistry? To your point about democratization. Consider I'm old enough to have lived through the transition from film to digital, which was a landmark, and filled with the promises of bringing cinematic capabilities and tech to the masses. But how many people actually make movies on their iPhone? Has it truly benefitted or enhanced the art of storytelling? Or made it truly accessible to the next generation of storytellers? I dunno. Call me old. And it's exciting, believe me, I get that. But it seems like Big Tech makes these promises every decade, and each time, they seem to enrich only those at the top.

broooo! magic out here!

🪄✨

Which platform are you using seedance on?

I made this one with @mitte_ai - highly recommend

This feels Hollywood level

Oh wow haha thank you

Good job 👏🏼

Thank you!

Great test Reid!

Thank you!

nice

🤝

This is the way.

100%

For anyone who doesn't know, getting an #AI to have a person go to a car and open the car door correctly is not a trivial task. I've had all sorts of bizarre anomalies and wild interpretations of getting a character into and out of a car (using locally-hosted AI models).

Definitely! The previs makes thing a lot easier for sure

I hate this

ok 👍

that's an interesting workflow we are planning to include

Sweet looking forward to seeing what you make

Nothing wrong with animation Animation Directors make editing decisions down to the frame .. And literally discuss shots in frame count Pinpoint control is what a decent Director requires for ultimate psychological manipulation

Totally agreed