Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing Ctrl-VI, a video sampling method allowing for a flexible set of user controls—ranging from coarse but easy-to-specify text prompts to precise camera/object trajectories. (1/n)

zhang_yunzhi's profile picture

Yunzhi Zhang

2,553 subscribers

42,663 Aufrufe • vor 8 Monaten •via X (Twitter)

KunstBildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.
0:24

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.

MrNeRF

38,619 Aufrufe • vor 1 Jahr

Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation paper page: Recent advances in generative modeling have led to promising progress on synthesizing 3D human motion from text, with methods that can generate character animations from short prompts and specified durations. However, using a single text prompt as input lacks the fine-grained control needed by animators, such as composing multiple actions and defining precise durations for parts of the motion. To address this, we introduce the new problem of timeline control for text-driven motion synthesis, which provides an intuitive, yet fine-grained, input interface for users. Instead of a single prompt, users can specify a multi-track timeline of multiple prompts organized in temporal intervals that may overlap. This enables specifying the exact timings of each action and composing multiple actions in sequence or at overlapping intervals. To generate composite animations from a multi-track timeline, we propose a new test-time denoising method. This method can be integrated with any pre-trained motion diffusion model to synthesize realistic motions that accurately reflect the timeline. At every step of denoising, our method processes each timeline interval (text prompt) individually, subsequently aggregating the predictions with consideration for the specific body parts engaged in each action. Experimental comparisons and ablations validate that our method produces realistic motions that respect the semantics and timing of given text prompts.
0:21

Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation paper page: Recent advances in generative modeling have led to promising progress on synthesizing 3D human motion from text, with methods that can generate character animations from short prompts and specified durations. However, using a single text prompt as input lacks the fine-grained control needed by animators, such as composing multiple actions and defining precise durations for parts of the motion. To address this, we introduce the new problem of timeline control for text-driven motion synthesis, which provides an intuitive, yet fine-grained, input interface for users. Instead of a single prompt, users can specify a multi-track timeline of multiple prompts organized in temporal intervals that may overlap. This enables specifying the exact timings of each action and composing multiple actions in sequence or at overlapping intervals. To generate composite animations from a multi-track timeline, we propose a new test-time denoising method. This method can be integrated with any pre-trained motion diffusion model to synthesize realistic motions that accurately reflect the timeline. At every step of denoising, our method processes each timeline interval (text prompt) individually, subsequently aggregating the predictions with consideration for the specific body parts engaged in each action. Experimental comparisons and ablations validate that our method produces realistic motions that respect the semantics and timing of given text prompts.

AK

126,548 Aufrufe • vor 2 Jahren

A viral video has surfaced showing a man using a metal object to take cashew nuts from stored bags, sparking debate online. Many have questioned whether he was conducting a legitimate sampling process and if the method used was appropriate.
0:43

A viral video has surfaced showing a man using a metal object to take cashew nuts from stored bags, sparking debate online. Many have questioned whether he was conducting a legitimate sampling process and if the method used was appropriate.

THE STATE NEWS

104,019 Aufrufe • vor 2 Monaten

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)
0:54

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)

Poe

66,708 Aufrufe • vor 1 Jahr

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:
0:46

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,363 Aufrufe • vor 1 Jahr

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇
0:48

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,312 Aufrufe • vor 1 Jahr

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:
1:00

Introducing HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction. We built a multi-camera system and a semi-automatic method for annotating the shape and pose of hands and objects Project page:

Yu Xiang

57,228 Aufrufe • vor 1 Jahr

New Studio camera controls are now in beta, introducing a new Locking functionality for camera speed as well as new modes of camera movement.
0:26

New Studio camera controls are now in beta, introducing a new Locking functionality for camera speed as well as new modes of camera movement.

Bloxy News

11,589 Aufrufe • vor 1 Jahr

🎨 KLING AI 1.6 PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI 1.6 model text to video. You can use them as ideas or base prompts. All details in the thread , bookmark for later👇 1/18
1:30

🎨 KLING AI 1.6 PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI 1.6 model text to video. You can use them as ideas or base prompts. All details in the thread , bookmark for later👇 1/18

LudovicCreator

24,736 Aufrufe • vor 1 Jahr

enigmatic_e shows how our models fill a gap in his creative workflow, making it easy to stage characters within scenes and direct precise camera movements. 8/n
1:55

enigmatic_e shows how our models fill a gap in his creative workflow, making it easy to stage characters within scenes and direct precise camera movements. 8/n

World Labs

35,044 Aufrufe • vor 1 Jahr

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"
0:53

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"

Fabien Benetou

222,914 Aufrufe • vor 3 Jahren

🎨 KLING AI 1.6 POV PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI text to video. You can use them as ideas or base prompts. Sound on : Freepik All details in the thread , bookmark for later👇 1/25
2:05

🎨 KLING AI 1.6 POV PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI text to video. You can use them as ideas or base prompts. Sound on : Freepik All details in the thread , bookmark for later👇 1/25

LudovicCreator

58,020 Aufrufe • vor 1 Jahr

Introducing FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes! Our system merges a library of skills with foundation models for efficient robotic feeding, tailored to user preferences. 🔗 📃 To Appear at RSS ’24 1/N
0:41

Introducing FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes! Our system merges a library of skills with foundation models for efficient robotic feeding, tailored to user preferences. 🔗 📃 To Appear at RSS ’24 1/N

Priya Sundaresan

23,080 Aufrufe • vor 1 Jahr

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 Like other Lyria RT demos, you can mix together text prompts and other controls to steer the model in real-time. But now with a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with. 💾 Get it here:
0:20

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 Like other Lyria RT demos, you can mix together text prompts and other controls to steer the model in real-time. But now with a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with. 💾 Get it here:

Jesse Engel

59,838 Aufrufe • vor 11 Monaten

Introducing MULTI-FRAMES — a major step forward in controllable video generation. Define up to 10 keyframes and generate a coherent one-take sequence up to ~54 seconds. Transitions can be set from 1–6 seconds, with prompts to control how each scene evolves between frames. Check our showcase video below👇 #DreaminaMultiFrames
0:58

Introducing MULTI-FRAMES — a major step forward in controllable video generation. Define up to 10 keyframes and generate a coherent one-take sequence up to ~54 seconds. Transitions can be set from 1–6 seconds, with prompts to control how each scene evolves between frames. Check our showcase video below👇 #DreaminaMultiFrames

Dreamina AI

139,742 Aufrufe • vor 7 Monaten

Now, camera work isn’t limited by physics. Introducing Higgsfield Mix - combine multiple motion controls in a single shot, including moves that aren’t possible with real cameras. Also dropping 10 new motion controls built for speed, tension, and cinematic impact. 🧩 1/n
0:53

Now, camera work isn’t limited by physics. Introducing Higgsfield Mix - combine multiple motion controls in a single shot, including moves that aren’t possible with real cameras. Also dropping 10 new motion controls built for speed, tension, and cinematic impact. 🧩 1/n

Higgsfield AI 🧩

650,634 Aufrufe • vor 1 Jahr

Introducing Luma Camera Controls! This is what I meant when I said we need a better prompting experience. See how smooth and easy it is! Kudos to Luma product team 👏
0:26

Introducing Luma Camera Controls! This is what I meant when I said we need a better prompting experience. See how smooth and easy it is! Kudos to Luma product team 👏

Dogan Ural

55,854 Aufrufe • vor 1 Jahr