正在加载视频...
视频加载失败
Introducing Swing Choreo, our new model for human motion. 🤸 Describe a performance one beat at a time and Choreo performs all of it: each action in order, hands and faces, several people at once, moving where you tell them to.
14,599 次观看 • 1 个月前 •via X (Twitter)
42 条评论

Super interesting tool. Been messing with blender mcp these days, and getting characters to move properly has been a big challenge. cc @athreyac4

@athreyac4 You might find this useful too

@athreyac4 Thanks, will take a look

Game designers need to get on this ASAP, this is insane!

Progress!!!

Can the animation files be edited in other animation tools after creation too?

yes definitely! fbx, glb, mb, bvh, usd - whatever you'd like (even .qpos if you like robotics)

We've spared no expense to get whole sequences from one prompt. Most text-to-motion models blur a multi-part prompt into a single action. Choreo performs each part in order, with transitions that carry weight and timing. "Football player running, then slams football on the ground and celebrates."

Hands and faces. Choreo generates finger articulation and facial motion with the body, so a prompt can ask for a hand shape, a gesture, or a laugh. Here, two characters play rock, paper, scissors.

More than one person. Several characters at once, each with their own action and aware of the others. They face each other when they talk and stay in time when they dance.

Direction in space. Tell characters where to go, in the world or relative to the camera. Left means camera left. "Run to the left and then to the right."

In blind comparisons, professional animators preferred Choreo over Swing Sketch, our previous model, 88% of the time. The gap is widest where prompts ask the most of the model: several actions, several people, detailed hands.

How Choreo compares with our previous model, open research models, and commercial tools. The one thing it gives up is speed: Choreo takes minutes per generation, not seconds.

That time goes into quality. More compute buys better motion, and the Swing models sit on the frontier at every time budget we measured. Most of the gains came from data selection and post-training, which improved prompt adherence and motion quality together.

Choreo comes out of a loop we've spent the past year building: capture motion, train, evaluate with animators, and send what the model gets wrong back into the data. Each pass produces a better model and tells us what to capture next.

Motion generation doesn't have a good public benchmark yet, so we built one: sequences, hands, faces, multiple people, direction, and physical plausibility. On the preliminary version, Choreo scores 61/100. The next best system scores 38. Paper and code are coming soon.

Swing Choreo is available today in Performance Generator. For fast blocking, Swing Sketch still returns results in seconds. Try it: Full write-up:

This is surely some kind of magic, andrew?? :) congrats on the new model!

congrats for the release… motions are looking smooth

Thank you! Hats off (ha) for your work in roblox - that stuff is hard

ahh thanks man! means a lot

my project needs this, how can i use it with api?

I now have a project where I need this

whoa! well we should get lunch and you should tell me about it

Yeah, let's catch up. I'll dm you

Please let me know how to try this. Thanks

(free trial)

baller

we agree

I’ve been looking for something like since this morning, great serendipity ! Will send feedback

can it do a moonwalk? this is really important for my research(on how to dance better)

멋지네요 3D 맥스에서 바이패드에도 사용 가능할까요?

Out is FBX, GLTF, or BVH - so yes!

Is this stuff is free? For try??

Text-to-motion for the win! Great write-up on the blog, plenty of useful details. congrats :)

Why this and not NVIDIA Kimodo? Do not get offended I am just trying to understand the differences. I use Kimodo which is free and runs on my cheap CPU.

kimodo is awesome! a few things 1. better feet, finger motion 2. facial generation 3. multi person generations and interactions (from one prompt) 4. better style (kimodo only knows like 10 styles) 5. better prompt adherence

Thanks for your reply I will really give it a look. Congrats for putting this out there I just got to know it.

Of course!! And feel free to DM if you have more questions. I live and breathe this field and so am happy to help if I can

$20 a month for 3 mins of motion for ONE character hmmmmmm

ridiculously expensive: $20 per 3min of animation. it might be okay if every second of that is high quality, but anything generative is going to miss the mark on the first attempts

yeah I hear ya, we're working to serve it cheaper - but gpus aren't cheap 🫣
相关视频
This was crazy choreo for them to have only performed it all live once.
mniN1c0le⁷
147,610 次观看 • 1 个月前


