Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Story-to-Motion: Synthesizing Infinite and Controllable Character Animation from Long Text Outperforms previous SotA motion synthesis methods across the board proj: abs:

arankomatsuzaki's profile picture

Aran Komatsuzaki

177,913 subscribers

38,401 görüntüleme • 2 yıl önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

6 Yorum

Pseudonym 🦅 profil fotoğrafı
Pseudonym 🦅2 yıl önce

Looks like we have all the pieces now to have a LLM assemble a 3d universe. Motion, physics, models, textures, voice, and endless content all from natural language. Surely someone is working on this? If not someone mail me large crates of cash 😂

kache profil fotoğrafı
kache2 yıl önce

weits

Chef Wang profil fotoğrafı
Chef Wang2 yıl önce

Astonishing

Emma Catnip profil fotoğrafı
Emma Catnip2 yıl önce

✨👀✨

Sourav Kumar Bose profil fotoğrafı
Sourav Kumar Bose2 yıl önce

🤯 AI-generated animated films are almost here!

Adrian Perdjon profil fotoğrafı
Adrian Perdjon1 yıl önce

RigPlay dataset might be perfect for your development. Learn more at [email protected]

Benzer Videolar

Loopy Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency paper page: With the introduction of diffusion-based video generation techniques, audio-conditioned human video generation has recently achieved significant breakthroughs in both the naturalness of motion and the synthesis of portrait details. Due to the limited control of audio signals in driving human motion, existing methods often add auxiliary spatial signals to stabilize movements, which may compromise the naturalness and freedom of motion. In this paper, we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference. Extensive experiments show that Loopy outperforms recent audio-driven portrait diffusion models, delivering more lifelike and high-quality results across various scenarios.
0:45

Loopy Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency paper page: With the introduction of diffusion-based video generation techniques, audio-conditioned human video generation has recently achieved significant breakthroughs in both the naturalness of motion and the synthesis of portrait details. Due to the limited control of audio signals in driving human motion, existing methods often add auxiliary spatial signals to stabilize movements, which may compromise the naturalness and freedom of motion. In this paper, we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference. Extensive experiments show that Loopy outperforms recent audio-driven portrait diffusion models, delivering more lifelike and high-quality results across various scenarios.

AK

128,803 görüntüleme • 1 yıl önce

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses
0:48

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 görüntüleme • 3 yıl önce

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.
5:56

Example-based Motion Synthesis via Generative Motion Matching paper page: present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

AK

93,694 görüntüleme • 3 yıl önce

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.
6:34

📢📢📢 Excited to share our new work *Autonomous Character-Scene Interaction Synthesis from Text Instruction* (Siggraph Asia 24). It presents a unified model for flexible scene-conditioned motion generation given text, scene, trajectory conditions. The results with smooth interaction look very impressive! 📰Paper: Project: Code and data will be released soon.

Siyuan Huang

11,340 görüntüleme • 1 yıl önce

Got five papers accepted by #ECCV2024 European Conference on Computer Vision #ECCV2026 ! Huge thanks to all my collaborators! 😃 See you in Milan 🇮🇹 Summary of Selected Works (I made a fast-forward for them 😄) - [Shape Generation] Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models, ECCV 2024. - [Efficient Motion Generation] EMDM: Efficient Motion Diffusion Model for Fast, High-Quality Human Motion Generation, ECCV 2024. - [Controllable Motion Generation] TLControl: Trajectory and Language Control for Human Motion Synthesis, ECCV 2024. - [Avatar Generation] Disentangled Clothed Avatar Generation from Text Descriptions, ECCV 2024. Project Page: Surf-D: EMDM: TLControl: SOSMPL:
3:49

Got five papers accepted by #ECCV2024 European Conference on Computer Vision #ECCV2026 ! Huge thanks to all my collaborators! 😃 See you in Milan 🇮🇹 Summary of Selected Works (I made a fast-forward for them 😄) - [Shape Generation] Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models, ECCV 2024. - [Efficient Motion Generation] EMDM: Efficient Motion Diffusion Model for Fast, High-Quality Human Motion Generation, ECCV 2024. - [Controllable Motion Generation] TLControl: Trajectory and Language Control for Human Motion Synthesis, ECCV 2024. - [Avatar Generation] Disentangled Clothed Avatar Generation from Text Descriptions, ECCV 2024. Project Page: Surf-D: EMDM: TLControl: SOSMPL:

Zhiyang (Frank) Dou

18,182 görüntüleme • 2 yıl önce

Introducing Motion Tokens. Data-driven animation. Every property controlled at runtime. Update text, colors, gradients and transforms — all from code.
0:47

Introducing Motion Tokens. Data-driven animation. Every property controlled at runtime. Update text, colors, gradients and transforms — all from code.

LottieFiles

60,401 görüntüleme • 4 ay önce

FrankenMotion: Part-level human motion generation; - enables compositional synthesis by combining atomic body-part movements; - DiT + FrankenAgent LLM. Indie game devs/animators can make complex character animations via text without manual keyframing.
1:17

FrankenMotion: Part-level human motion generation; - enables compositional synthesis by combining atomic body-part movements; - DiT + FrankenAgent LLM. Indie game devs/animators can make complex character animations via text without manual keyframing.

Wildminder

33,890 görüntüleme • 5 ay önce

Animate-X Universal Character Image Animation with Enhanced Motion Representation
0:36

Animate-X Universal Character Image Animation with Enhanced Motion Representation

AK

101,151 görüntüleme • 1 yıl önce

animators are not needed anymore this 3D AI motion capture plugin can convert character movement from real video to 3D data and.. you can apply the motion to any 3D character.. link in comments
0:35

animators are not needed anymore this 3D AI motion capture plugin can convert character movement from real video to 3D data and.. you can apply the motion to any 3D character.. link in comments

el.cine

64,514 görüntüleme • 7 ay önce

Next is "Eunha" death motion and motion editing. Please write down the character you want to see! #ryona #リョナ
2:12

Next is "Eunha" death motion and motion editing. Please write down the character you want to see! #ryona #リョナ

Ferret

12,698 görüntüleme • 1 yıl önce

Next-level motion from @HeyGen_Official I got to play with the new AI Motion from HeyGen, and it blew my mind! You generate an image and describe the motion; then, you can generate long videos from that avatar. Sound effects included! 1. Tales from The Heartland
1:11

Next-level motion from @HeyGen_Official I got to play with the new AI Motion from HeyGen, and it blew my mind! You generate an image and describe the motion; then, you can generate long videos from that avatar. Sound effects included! 1. Tales from The Heartland

Alex Patrascu

57,728 görüntüleme • 1 yıl önce

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:
1:15

✨We are excited to open-source Tencent HY-Motion 1.0, a billion-parameter text-to-motion model built on the Diffusion Transformer (DiT) architecture and flow matching. Tencent HY-Motion 1.0 empowers developers and individual creators alike by transforming natural language into high-fidelity, fluid, and diverse 3D character animations, delivering exceptional instruction-following capabilities across a broad range of categories. The generated 3D animation assets can be seamlessly integrated into typical 3D animation pipelines.🎮🎥 Highlights: 🔹Billion-Scale DiT: Successfully scaled flow-matching DiT to 1B+ parameters, setting a new ceiling for instruction-following capability and generated motion quality. 🔹Full-Stage Training Strategy: The industry’s first motion generation model featuring a complete Pre-training → SFT → RL loop to optimize physical plausibility and semantic accuracy. 🔹Comprehensive Category Coverage: Features 200+ motion categories across 6 major classes—the most comprehensive in the industry, curated via a meticulous data pipeline. 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical report:

Tencent Hy

327,984 görüntüleme • 6 ay önce

Warudo Features - Motion Recorder & Motion Player 🌠The simple and universal motion-recording function allows you to play your own actions and dances at any time!🎇 🌟Play multiple Character Motions at the same time 💃Motion data can be applied to any character model ✂️ Freely trim Motion to a suitable timing 💖You can even export the recorded motion to Unity Animation (.anim) for various editing and use!😋 🌻For more information, watch this tutorial!🌻 白夜 Tira 夢宮ヨユ ⚡ SLiMe VTuber / Warudo 🎵BGM: ASMRZ(TANAKA, NEEDMORECASH) - 晚安小姐(prod. Gwana) & フォンコール #Warudo #VTuber #3D #LiveStream #3DLive #Unity #mocap #motion
2:18

Warudo Features - Motion Recorder & Motion Player 🌠The simple and universal motion-recording function allows you to play your own actions and dances at any time!🎇 🌟Play multiple Character Motions at the same time 💃Motion data can be applied to any character model ✂️ Freely trim Motion to a suitable timing 💖You can even export the recorded motion to Unity Animation (.anim) for various editing and use!😋 🌻For more information, watch this tutorial!🌻 白夜 Tira 夢宮ヨユ ⚡ SLiMe VTuber / Warudo 🎵BGM: ASMRZ(TANAKA, NEEDMORECASH) - 晚安小姐(prod. Gwana) & フォンコール #Warudo #VTuber #3D #LiveStream #3DLive #Unity #mocap #motion

Warudo @ Hakuya Labs

48,361 görüntüleme • 1 yıl önce

Interested in upgrading your Motion Matching system? Check out our #SIGGRAPH2024 work where we introduce *Codebook Matching* to train neural character controllers for synthesizing lifelike embodied animations in #Games and #VR! Project Page:
0:20

Interested in upgrading your Motion Matching system? Check out our #SIGGRAPH2024 work where we introduce *Codebook Matching* to train neural character controllers for synthesizing lifelike embodied animations in #Games and #VR! Project Page:

Sebastian Starke

18,645 görüntüleme • 2 yıl önce

I modified "Story Framework" (from Fab) to include MetaHuman face and body animations to NPCs! The face animation is Audio Driven Anim and the body animation is from Narrative Motion. I'm going to make a short narrative game using this framework and put all of the MOCAP source animation online.
0:28

I modified "Story Framework" (from Fab) to include MetaHuman face and body animations to NPCs! The face animation is Audio Driven Anim and the body animation is from Narrative Motion. I'm going to make a short narrative game using this framework and put all of the MOCAP source animation online.

Matt Workman

21,137 görüntüleme • 9 ay önce

INFINITE ZAMASU BATTLE MOTION!!!!!!!!!!!!!!!!
0:36

INFINITE ZAMASU BATTLE MOTION!!!!!!!!!!!!!!!!

DaTruthDT

151,971 görüntüleme • 3 ay önce

Wan-Move Motion-controllable Video Generation via Latent Trajectory Guidance
2:00

Wan-Move Motion-controllable Video Generation via Latent Trajectory Guidance

AK

14,853 görüntüleme • 6 ay önce