Загрузка видео...

Не удалось загрузить видео

На главную

Introducing Ctrl-VI, a video sampling method allowing for a flexible set of user controls—ranging from coarse but easy-to-specify text prompts to precise camera/object trajectories. (1/n)

zhang_yunzhi's profile picture

Yunzhi Zhang

2,629 subscribers

42,830 просмотров • 9 месяцев назад •via X (Twitter)

ИскусствоОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.
0:24

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.

MrNeRF

38,746 просмотров • 1 год назад

A viral video has surfaced showing a man using a metal object to take cashew nuts from stored bags, sparking debate online. Many have questioned whether he was conducting a legitimate sampling process and if the method used was appropriate.
0:43

A viral video has surfaced showing a man using a metal object to take cashew nuts from stored bags, sparking debate online. Many have questioned whether he was conducting a legitimate sampling process and if the method used was appropriate.

THE STATE NEWS

104,022 просмотров • 3 месяцев назад

Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation paper page: Recent advances in generative modeling have led to promising progress on synthesizing 3D human motion from text, with methods that can generate character animations from short prompts and specified durations. However, using a single text prompt as input lacks the fine-grained control needed by animators, such as composing multiple actions and defining precise durations for parts of the motion. To address this, we introduce the new problem of timeline control for text-driven motion synthesis, which provides an intuitive, yet fine-grained, input interface for users. Instead of a single prompt, users can specify a multi-track timeline of multiple prompts organized in temporal intervals that may overlap. This enables specifying the exact timings of each action and composing multiple actions in sequence or at overlapping intervals. To generate composite animations from a multi-track timeline, we propose a new test-time denoising method. This method can be integrated with any pre-trained motion diffusion model to synthesize realistic motions that accurately reflect the timeline. At every step of denoising, our method processes each timeline interval (text prompt) individually, subsequently aggregating the predictions with consideration for the specific body parts engaged in each action. Experimental comparisons and ablations validate that our method produces realistic motions that respect the semantics and timing of given text prompts.
0:21

Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation paper page: Recent advances in generative modeling have led to promising progress on synthesizing 3D human motion from text, with methods that can generate character animations from short prompts and specified durations. However, using a single text prompt as input lacks the fine-grained control needed by animators, such as composing multiple actions and defining precise durations for parts of the motion. To address this, we introduce the new problem of timeline control for text-driven motion synthesis, which provides an intuitive, yet fine-grained, input interface for users. Instead of a single prompt, users can specify a multi-track timeline of multiple prompts organized in temporal intervals that may overlap. This enables specifying the exact timings of each action and composing multiple actions in sequence or at overlapping intervals. To generate composite animations from a multi-track timeline, we propose a new test-time denoising method. This method can be integrated with any pre-trained motion diffusion model to synthesize realistic motions that accurately reflect the timeline. At every step of denoising, our method processes each timeline interval (text prompt) individually, subsequently aggregating the predictions with consideration for the specific body parts engaged in each action. Experimental comparisons and ablations validate that our method produces realistic motions that respect the semantics and timing of given text prompts.

AK

126,585 просмотров • 2 лет назад

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)
0:54

Introducing Poe Apps: a new, easy way to create and use visual interfaces into any combination of the 100+ text, image, video, and audio models on Poe. (1/5)

Poe

66,813 просмотров • 1 год назад

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:
0:46

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,467 просмотров • 1 год назад

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇
0:48

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,366 просмотров • 1 год назад

🎨 KLING AI 1.6 PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI 1.6 model text to video. You can use them as ideas or base prompts. All details in the thread , bookmark for later👇 1/18
1:30

🎨 KLING AI 1.6 PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI 1.6 model text to video. You can use them as ideas or base prompts. All details in the thread , bookmark for later👇 1/18

LudovicCreator

24,736 просмотров • 1 год назад

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"
0:53

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"

Fabien Benetou

222,914 просмотров • 3 лет назад

🎨 KLING AI 1.6 POV PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI text to video. You can use them as ideas or base prompts. Sound on : Freepik All details in the thread , bookmark for later👇 1/25
2:05

🎨 KLING AI 1.6 POV PROMPTS THREAD 🎨 Here for you a new thread of prompts for Kling AI text to video. You can use them as ideas or base prompts. Sound on : Freepik All details in the thread , bookmark for later👇 1/25

LudovicCreator

58,020 просмотров • 1 год назад

Introducing FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes! Our system merges a library of skills with foundation models for efficient robotic feeding, tailored to user preferences. 🔗 📃 To Appear at RSS ’24 1/N
0:41

Introducing FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes! Our system merges a library of skills with foundation models for efficient robotic feeding, tailored to user preferences. 🔗 📃 To Appear at RSS ’24 1/N

Priya Sundaresan

23,080 просмотров • 2 лет назад

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 Like other Lyria RT demos, you can mix together text prompts and other controls to steer the model in real-time. But now with a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with. 💾 Get it here:
0:20

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 Like other Lyria RT demos, you can mix together text prompts and other controls to steer the model in real-time. But now with a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with. 💾 Get it here:

Jesse Engel

59,879 просмотров • 1 год назад

Introducing MULTI-FRAMES — a major step forward in controllable video generation. Define up to 10 keyframes and generate a coherent one-take sequence up to ~54 seconds. Transitions can be set from 1–6 seconds, with prompts to control how each scene evolves between frames. Check our showcase video below👇 #DreaminaMultiFrames
0:58

Introducing MULTI-FRAMES — a major step forward in controllable video generation. Define up to 10 keyframes and generate a coherent one-take sequence up to ~54 seconds. Transitions can be set from 1–6 seconds, with prompts to control how each scene evolves between frames. Check our showcase video below👇 #DreaminaMultiFrames

Dreamina AI

139,742 просмотров • 8 месяцев назад

Google announces PALP Prompt Aligned Personalization of Text-to-Image Models paper page: Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.
1:16

Google announces PALP Prompt Aligned Personalization of Text-to-Image Models paper page: Content creators often aim to create personalized images using personal subjects that go beyond the capabilities of conventional text-to-image models. Additionally, they may want the resulting image to encompass a specific location, style, ambiance, and more. Existing personalization methods may compromise personalization ability or the alignment to complex textual prompts. This trade-off can impede the fulfillment of user prompts and subject fidelity. We propose a new approach focusing on personalization methods for a single prompt to address this issue. We term our approach prompt-aligned personalization. While this may seem restrictive, our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts, which may pose a challenge for current techniques. In particular, our method keeps the personalized model aligned with a target prompt using an additional score distillation sampling term. We demonstrate the versatility of our method in multi- and single-shot settings and further show that it can compose multiple subjects or use inspiration from reference images, such as artworks. We compare our approach quantitatively and qualitatively with existing baselines and state-of-the-art techniques.

AK

90,199 просмотров • 2 лет назад

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.
0:24

DiffSplat Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation DiffSplat is a generative framework to synthesize 3D Gaussian Splats from text prompts & single-view images in ⚡️ 1~2 seconds. It is fine-tuned directly from a pretrained text-to-image diffusion model.

AK

38,416 просмотров • 1 год назад

[1/N] We present a plug-and-play mechanism to controllably steer inference of any diffusion/flow model towards a sharper or flatter sampling distribution, resulting in improvements across domains e.g. text-to-image (10% FID reduction), protein generation (improved designability).
0:51

[1/N] We present a plug-and-play mechanism to controllably steer inference of any diffusion/flow model towards a sharper or flatter sampling distribution, resulting in improvements across domains e.g. text-to-image (10% FID reduction), protein generation (improved designability).

Shubham Tulsiani

82,999 просмотров • 10 месяцев назад

We are just scratching the surface of precise control over AI video generation. MotionStream unlocks real-time video with interactive motion controls. You can interactively generate video based on motion inputs (like drawn trajectories, camera movements, or motion transfer). 29fps generation w/ 0.4 second latency on a single H100, oh and infinite length streaming. So much more intuitive than text prompts. Almost feels like performance capture - painting motion paths and watching AI generated video follow those paths instantly. Amazing work by Xun Huang et al. Feels like we're on the cusp of a new type of creation engine. And let me tell you - it ain't all nodes :-)
0:32

We are just scratching the surface of precise control over AI video generation. MotionStream unlocks real-time video with interactive motion controls. You can interactively generate video based on motion inputs (like drawn trajectories, camera movements, or motion transfer). 29fps generation w/ 0.4 second latency on a single H100, oh and infinite length streaming. So much more intuitive than text prompts. Almost feels like performance capture - painting motion paths and watching AI generated video follow those paths instantly. Amazing work by Xun Huang et al. Feels like we're on the cusp of a new type of creation engine. And let me tell you - it ain't all nodes :-)

Bilawal Sidhu

46,079 просмотров • 8 месяцев назад

1/7: 📢 Big release! Introducing Astra 2 🚀 Our next-gen creative model with a whole new level of controls, including PROMPTS! Try it NOW in Astra app See what it's best for in this short video and 🧵👇
1:11

1/7: 📢 Big release! Introducing Astra 2 🚀 Our next-gen creative model with a whole new level of controls, including PROMPTS! Try it NOW in Astra app See what it's best for in this short video and 🧵👇

Topaz Labs

2,625,848 просмотров • 3 месяцев назад