Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Interested in upgrading your Motion Matching system? Check out our #SIGGRAPH2024 work where we introduce *Codebook Matching* to train neural character controllers for synthesizing lifelike embodied animations in #Games and #VR! Project Page:

blacksquirrel__'s profile picture

Sebastian Starke

3,301 subscribers

18,659 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie#SIGGRAPH2024#Games#VR
LIVE

Anya Rossi• Live Now

Private livecam show

4 Kommentare

Profilbild von Paul Starke
Paul Starkevor 2 Jahren

Paper can be found here

Profilbild von Zhiyang (Frank) Dou
Zhiyang (Frank) Douvor 2 Jahren

Cool work!

Profilbild von Michael Gold
Michael Goldvor 2 Jahren

Awesome work. Congrats!

Profilbild von Andrew Carr (e/🤸)
Andrew Carr (e/🤸)vor 2 Jahren

Very cool work!

Ähnliche Videos

The code, dataset and demos for our #SIGGRAPH2024 work on generating embodied avatar movements with Codebook Matching are now available! Everything can be found on the project page, including a precompiled binary that you can play around with!
0:40

The code, dataset and demos for our #SIGGRAPH2024 work on generating embodied avatar movements with Codebook Matching are now available! Everything can be found on the project page, including a precompiled binary that you can play around with!

Sebastian Starke

18,386 Aufrufe • vor 2 Jahren

Remember we promised you a game animation sample at GDC 2024? It’s here! Get up and running with Motion Matching in this project that includes hundreds of free AAA-quality animations 🏃‍♂️
0:29

Remember we promised you a game animation sample at GDC 2024? It’s here! Get up and running with Motion Matching in this project that includes hundreds of free AAA-quality animations 🏃‍♂️

Unreal Engine

1,132,707 Aufrufe • vor 2 Jahren

Fortnite has improved the animations on ALL platforms! "Motion Matching and Procedural Layering are features that result in improved animations for things like transitioning from walking to running, changing directions, and using a weapon."
0:36

Fortnite has improved the animations on ALL platforms! "Motion Matching and Procedural Layering are features that result in improved animations for things like transitioning from walking to running, changing directions, and using a weapon."

Shiina

1,988,805 Aufrufe • vor 2 Jahren

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.
5:56

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

AK

93,729 Aufrufe • vor 3 Jahren

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/
0:59

Tired of endlessly teleoperating your robot in order to train it? Introducing SkillMimicGen, a data generation system that automatically scales robot imitation learning by synthesizing demos through integrating motion planning and demo adaptation. 1/

Ajay Mandlekar

35,575 Aufrufe • vor 1 Jahr

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!
0:35

Excited to share our #NeurIPS2025 work on learning motion hierarchies! We introduce a general hierarchical graph learning method that learns structured, interpretable motion directly from data, no prior structure or assumptions needed!!! Project and Paper: Amazing work led by William Koch, Cheng Zheng, and Baiang Li ! See us in San Diego for #NeurIPS2025!

Felix Heide

25,314 Aufrufe • vor 8 Monaten

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥
1:32

How do you give a humanoid the general motion capability? Not just single motions, but all motion? Introducing SONIC, our new work on supersizing motion tracking for natural humanoid control. We argue that motion tracking is the scalable foundation task for humanoids. So we "supersized" it: 9k+ GPU hours and 100M+ motion frames. But tracking alone is not enough; we show how to make a useful control system out of it: - Universal Kinematic Planner: Enables game-like gamepad control and high-level teleoperation, just like controlling a character in a game. - VR Full-Body Teleop: Direct, real-time whole-body control by a human wearing a VR headset. - VR Keypoint Teleop: Control the upper body (hands/head) while our planner handles robust locomotion automatically. - VLA Integration: We connect this motion tracker to autonomous Visual-Language-Action (VLA) models for autonomous task execution! We use a Universal Token Space to UNIFY this command space, turning our robust tracker into a general-purpose, programmable humanoid brain. This is the generalist "System 1" for humanoids. 🚀 Project: #Humanoids #Robotics #AI #FoundationModels #NVIDIAResearch 🧠🔥

Zhengyi “Zen” Luo

62,040 Aufrufe • vor 8 Monaten

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:
0:39

"Generalization means being able to solve problems that the system hasn't been prepared for." Our latest work in #RSS2025 can automatically invent neural networks as state abstractions, which help robots generalize. Check it out here:

Bowen Li

19,399 Aufrufe • vor 1 Jahr

BUILD GAMES. WOW. My unfiltered thoughts so far and what’s next. Please shill your project in the comments for people to check out:
6:03

BUILD GAMES. WOW. My unfiltered thoughts so far and what’s next. Please shill your project in the comments for people to check out:

🦅

11,736 Aufrufe • vor 3 Monaten

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI
3:58

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Zhiyang (Frank) Dou

14,610 Aufrufe • vor 1 Jahr

A player-driven, branching narrative and tactile gameplay in I AM RIPPER ensures that the latest project from White Paper Games puts players into a gritty, immersive experience. Find out how the team used Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create nail-biting thriller I AM RIPPER:
0:33

A player-driven, branching narrative and tactile gameplay in I AM RIPPER ensures that the latest project from White Paper Games puts players into a gritty, immersive experience. Find out how the team used Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create nail-biting thriller I AM RIPPER:

Unreal Engine

10,113 Aufrufe • vor 8 Monaten

Last week, we talked to the team at White Paper Games about how they utilized Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create a gritty, immersive, nail-biting thriller in I AM RIPPER. Explore the process behind the player-driven, branching narrative and how it game together in our article:
0:38

Last week, we talked to the team at White Paper Games about how they utilized Unreal Engine 5’s Motion Matching, Control Rig and Sequencer to create a gritty, immersive, nail-biting thriller in I AM RIPPER. Explore the process behind the player-driven, branching narrative and how it game together in our article:

Unreal Engine

10,290 Aufrufe • vor 8 Monaten

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:
1:15

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:

Tencent Hy

328,493 Aufrufe • vor 6 Monaten

Well, the cat’s out of the bag. My opponent went on Fox News and said what we all knew — he’s not interested in fixing our broken immigration system. He’s more interested in scoring political points and playing political games. He’s doing it while wasting your taxpayer money.
0:28

Well, the cat’s out of the bag. My opponent went on Fox News and said what we all knew — he’s not interested in fixing our broken immigration system. He’s more interested in scoring political points and playing political games. He’s doing it while wasting your taxpayer money.

DPA Chair Marcus Jones

27,006 Aufrufe • vor 2 Jahren

Today we are releasing Character-1.5 and Stylize. Available now at This update introduces stylization, allowing you to transform into a character while maintaining your identity. Character-1.5 brings sharper visual output, more lifelike animations with natural blinking and head movements, and improved handling of uploaded images. It's a major leap forward for our foundation model before we release Character-2.
0:26

Today we are releasing Character-1.5 and Stylize. Available now at This update introduces stylization, allowing you to transform into a character while maintaining your identity. Character-1.5 brings sharper visual output, more lifelike animations with natural blinking and head movements, and improved handling of uploaded images. It's a major leap forward for our foundation model before we release Character-2.

Hedra

75,465 Aufrufe • vor 1 Jahr

If you could work in any part of the DD shop, where would you be: design, machining, or assembly? Now’s your chance to make it real. Check out our open positions on our careers page, and join the DD Team!
0:25

If you could work in any part of the DD shop, where would you be: design, machining, or assembly? Now’s your chance to make it real. Check out our open positions on our careers page, and join the DD Team!

Daniel Defense

27,991 Aufrufe • vor 1 Jahr

Physics-based Motion Retargeting from Sparse Inputs paper page: Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.
7:12

Physics-based Motion Retargeting from Sparse Inputs paper page: Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.

AK

106,527 Aufrufe • vor 3 Jahren

Wouldn't it be great if we could train robots without any teleoperation! In our latest paper, we train robots to mimic a human video of the task by simply matching the object features using RL. We only need one video and under an hour of robot training.
0:22

Wouldn't it be great if we could train robots without any teleoperation! In our latest paper, we train robots to mimic a human video of the task by simply matching the object features using RL. We only need one video and under an hour of robot training.

Lerrel Pinto

46,221 Aufrufe • vor 1 Jahr