Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Interested in upgrading your Motion Matching system? Check out our #SIGGRAPH2024 work where we introduce Codebook Matching to train neural character controllers for synthesizing lifelike embodied animations in #Games and #VR! Project Page:

Sebastian Starke

3,310 subscribers

18,645 views • 2 years ago •via X (Twitter)

Science & Technology #SIGGRAPH2024 #Games #VR

Anya Rossi• Live Now

Private livecam show

4 Comments

Paul Starke2 years ago

Paper can be found here

Zhiyang (Frank) Dou2 years ago

Cool work!

Michael Gold2 years ago

Awesome work. Congrats!

Andrew Carr (e/🤸)2 years ago

Very cool work!

Related Videos

The code, dataset and demos for our #SIGGRAPH2024 work on generating embodied avatar movements with Codebook Matching are now available! Everything can be found on the project page, including a precompiled binary that you can play around with!

Sebastian Starke

18,379 views • 1 year ago

Remember we promised you a game animation sample at GDC 2024? It’s here! Get up and running with Motion Matching in this project that includes hundreds of free AAA-quality animations 🏃‍♂️

Remember we promised you a game animation sample at GDC 2024? It’s here! Get up and running with Motion Matching in this project that includes hundreds of free AAA-quality animations 🏃‍♂️

Unreal Engine

1,132,427 views • 2 years ago

Motion Matching based locomotion system in #UE5 (Gameplay ready & Fully Replicated)

Motion Matching based locomotion system in #UE5 (Gameplay ready & Fully Replicated)

Kelechi Apakama 🔜 Develop Brighton | 🛠 Stormrite

28,151 views • 2 years ago

Fortnite has improved the animations on ALL platforms! "Motion Matching and Procedural Layering are features that result in improved animations for things like transitioning from walking to running, changing directions, and using a weapon."

Fortnite has improved the animations on ALL platforms! "Motion Matching and Procedural Layering are features that result in improved animations for things like transitioning from walking to running, changing directions, and using a weapon."

Shiina

1,988,699 views • 2 years ago

Synthesia research releases HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints project page: paper page:

Synthesia research releases HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints project page: paper page:

AK

160,702 views • 3 years ago

David Pusher released RDR System, a C++ project that integrates ALS and GASP with Motion Matching, along with seamless switching between third- and first-person perspectives. Get it for free:

David Pusher released RDR System, a C++ project that integrates ALS and GASP with Motion Matching, along with seamless switching between third- and first-person perspectives. Get it for free:

80 LEVEL

31,794 views • 1 year ago

$Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.$

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

AK

93,694 views • 3 years ago

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

Ajay Mandlekar

35,575 views • 1 year ago

My upcoming camera plugin works out of the box on the new motion-matching sample project! Instanced modifier stacks have no issue reading the required data, and the runtime blending and modification work perfectly.

My upcoming camera plugin works out of the box on the new motion-matching sample project! Instanced modifier stacks have no issue reading the required data, and the runtime blending and modification work perfectly.

Ev Schwab

50,875 views • 2 years ago

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!

Felix Heide

25,314 views • 7 months ago

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

61,889 views • 7 months ago

BUILD GAMES. WOW. My unfiltered thoughts so far and what’s next. Please shill your project in the comments for people to check out:

BUILD GAMES. WOW. My unfiltered thoughts so far and what’s next. Please shill your project in the comments for people to check out:

🦅

11,731 views • 2 months ago

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

Bowen Li

19,274 views • 1 year ago

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Zhiyang (Frank) Dou

14,610 views • 11 months ago

🚨 Pairs & Perils has a release date: 20th January! Like the memory-matching game, you combine matching cards to attack the enemies, collect your loot, and try to beat the boss. #TrailerTuesday Check out the free demo for a limited time!

🚨 Pairs & Perils has a release date: 20th January! Like the memory-matching game, you combine matching cards to attack the enemies, collect your loot, and try to beat the boss. #TrailerTuesday Check out the free demo for a limited time!

Rogue Duck Interactive

29,296 views • 1 year ago

A player-driven, branching narrative and tactile gameplay in I AM RIPPER ensures that the latest project from White Paper Games puts players into a gritty, immersive experience. Find out how the team used Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create nail-biting thriller I AM RIPPER:

A player-driven, branching narrative and tactile gameplay in I AM RIPPER ensures that the latest project from White Paper Games puts players into a gritty, immersive experience. Find out how the team used Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create nail-biting thriller I AM RIPPER:

Unreal Engine

10,101 views • 7 months ago

Last week, we talked to the team at White Paper Games about how they utilized Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create a gritty, immersive, nail-biting thriller in I AM RIPPER. Explore the process behind the player-driven, branching narrative and how it game together in our article:

Last week, we talked to the team at White Paper Games about how they utilized Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create a gritty, immersive, nail-biting thriller in I AM RIPPER. Explore the process behind the player-driven, branching narrative and how it game together in our article:

Unreal Engine

10,266 views • 7 months ago

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:

Tencent Hy

327,984 views • 5 months ago