正在加载视频...

视频加载失败

Introducing Swing Choreo, our new model for human motion. 🤸 Describe a performance one beat at a time and Choreo performs all of it: each action in order, hands and faces, several people at once, moving where you tell them to.

14,599 次观看 • 1 个月前 •via X (Twitter)

42 条评论

Shrijith Venktramana 的头像
Shrijith Venktramana1 个月前

Super interesting tool. Been messing with blender mcp these days, and getting characters to move properly has been a big challenge. cc @athreyac4

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

@athreyac4 You might find this useful too

Shrijith Venktramana 的头像
Shrijith Venktramana1 个月前

@athreyac4 Thanks, will take a look

Lincoln Berbert 的头像
Lincoln Berbert1 个月前

Game designers need to get on this ASAP, this is insane!

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Progress!!!

Lincoln Berbert 的头像
Lincoln Berbert1 个月前

Can the animation files be edited in other animation tools after creation too?

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

yes definitely! fbx, glb, mb, bvh, usd - whatever you'd like (even .qpos if you like robotics)

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

We've spared no expense to get whole sequences from one prompt. Most text-to-motion models blur a multi-part prompt into a single action. Choreo performs each part in order, with transitions that carry weight and timing. "Football player running, then slams football on the ground and celebrates."

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Hands and faces. Choreo generates finger articulation and facial motion with the body, so a prompt can ask for a hand shape, a gesture, or a laugh. Here, two characters play rock, paper, scissors.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

More than one person. Several characters at once, each with their own action and aware of the others. They face each other when they talk and stay in time when they dance.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Direction in space. Tell characters where to go, in the world or relative to the camera. Left means camera left. "Run to the left and then to the right."

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

In blind comparisons, professional animators preferred Choreo over Swing Sketch, our previous model, 88% of the time. The gap is widest where prompts ask the most of the model: several actions, several people, detailed hands.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

How Choreo compares with our previous model, open research models, and commercial tools. The one thing it gives up is speed: Choreo takes minutes per generation, not seconds.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

That time goes into quality. More compute buys better motion, and the Swing models sit on the frontier at every time budget we measured. Most of the gains came from data selection and post-training, which improved prompt adherence and motion quality together.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Choreo comes out of a loop we've spent the past year building: capture motion, train, evaluate with animators, and send what the model gets wrong back into the data. Each pass produces a better model and tells us what to capture next.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Motion generation doesn't have a good public benchmark yet, so we built one: sequences, hands, faces, multiple people, direction, and physical plausibility. On the preliminary version, Choreo scores 61/100. The next best system scores 38. Paper and code are coming soon.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Swing Choreo is available today in Performance Generator. For fast blocking, Swing Sketch still returns results in seconds. Try it: Full write-up:

Toby VirtualFilmer 的头像
Toby VirtualFilmer1 个月前

This is surely some kind of magic, andrew?? :) congrats on the new model!

Boring Always Bored 的头像
Boring Always Bored1 个月前

congrats for the release… motions are looking smooth

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Thank you! Hats off (ha) for your work in roblox - that stuff is hard

Boring Always Bored 的头像
Boring Always Bored1 个月前

ahh thanks man! means a lot

Averse 的头像
Averse1 个月前

my project needs this, how can i use it with api?

🧗‍♂️ Matt Holt 的头像
🧗‍♂️ Matt Holt1 个月前

I now have a project where I need this

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

whoa! well we should get lunch and you should tell me about it

🧗‍♂️ Matt Holt 的头像
🧗‍♂️ Matt Holt1 个月前

Yeah, let's catch up. I'll dm you

Devang 的头像
Devang1 个月前

Please let me know how to try this. Thanks

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

(free trial)

montagovian 的头像
montagovian1 个月前

baller

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

we agree

Jordan 的头像
Jordan1 个月前

I’ve been looking for something like since this morning, great serendipity ! Will send feedback

TabTab 的头像
TabTab1 个月前

can it do a moonwalk? this is really important for my research(on how to dance better)

김란지 的头像
김란지1 个月前

멋지네요 3D 맥스에서 바이패드에도 사용 가능할까요?

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Out is FBX, GLTF, or BVH - so yes!

rockmet 的头像
rockmet1 个月前

Is this stuff is free? For try??

artur 的头像
artur1 个月前

Text-to-motion for the win! Great write-up on the blog, plenty of useful details. congrats :)

Luis 的头像
Luis1 个月前

Why this and not NVIDIA Kimodo? Do not get offended I am just trying to understand the differences. I use Kimodo which is free and runs on my cheap CPU.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

kimodo is awesome! a few things 1. better feet, finger motion 2. facial generation 3. multi person generations and interactions (from one prompt) 4. better style (kimodo only knows like 10 styles) 5. better prompt adherence

Luis 的头像
Luis1 个月前

Thanks for your reply I will really give it a look. Congrats for putting this out there I just got to know it.

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

Of course!! And feel free to DM if you have more questions. I live and breathe this field and so am happy to help if I can

NOISE ☻ 的头像
NOISE ☻1 个月前

$20 a month for 3 mins of motion for ONE character hmmmmmm

James 的头像
James1 个月前

ridiculously expensive: $20 per 3min of animation. it might be okay if every second of that is high quality, but anything generative is going to miss the mark on the first attempts

Andrew Carr 🤸 的头像
Andrew Carr 🤸1 个月前

yeah I hear ya, we're working to serve it cheaper - but gpus aren't cheap 🫣

相关视频