Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Stability AI just dropped Stable Virtual Camera on Hugging Face a generalist diffusion model designed to address the exciting challenge of Novel View Synthesis (NVS). With just one or a few images, it allows you to create a smooth trajectory video from any viewpoint you desire.

AK

514,947 subscribers

133,647 views • 1 year ago •via X (Twitter)

Education Science & Technology News & Politics

Anya Rossi• Live Now

Private livecam show

11 Comments

AK1 year ago

app:

AK1 year ago

model:

FRANK E ELKINS1 year ago

Confused about AI? Get clarity today with Book VI - "The Rational Being!" Understand the benefits & risks, empower your future now by learning what AI really is and how it really works! Also check out the Free Weekly Newsletter "How Things Work: A Brief History of Reality"

Garfield_Agents_😽_Kaisar1 year ago

That sounds amazing! With those cool videos, why not let your WiFi earn some of that crypto action too? @SparkChainAI pays you in SOL for unused internet bandwidth. Time to get earning while creating!

Mike Mozart 🎨 Artist MiMo 🐺1 year ago

@kamui_qai have you seen this yet?

Universa1 year ago

The release of Stable Virtual Camera on Hugging Face is an exciting development in AI research. This technology has the potential to drive innovation in various fields, from entertainment to education.

Wayne 🌊1 year ago

damn! But this will take years to make it to AppleVisionPro 😑or meta quest! Overhang is real

Abhishek Yadav1 year ago

mind blowing work 🔥

Puzzle Paws1 year ago

Stability AI's latest on Hugging Face for NVS? Cool toy, but let's see if it actually delivers!

Arpit Sharma1 year ago

Can’t wait to see what the community creates with it!

Rafat Siddiqui1 year ago

What’s your take on Generative 3D vs manually making models? Can gen3D truly replace or are programmatic frameworks like openUSD going to be better ?

Related Videos

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.

Stable Virtual Camera: Generative View Synthesis with Diffusion Models Hint: Check the project website. It is awesome! Contributions: 1. A training strategy for jointly modeling large viewpoint changes and temporal smoothness. 2. A two-pass procedural sampling method for smooth video generation along arbitrarily long camera trajectories. 3. A comprehensive benchmark that evaluates NVS methods across different datasets and settings. 4. An open-source release of model weights to support future research.

MrNeRF

38,746 views • 1 year ago

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Decart

1,011,981 views • 1 year ago

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Stable Diffusion 3.5 from Stability AI is now LIVE on Civitai! Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency. Generate with SD 3.5 Live on Civitai here👇🏽 All images used to create this video were generated with SD 3.5 by our community 🫶🏽

Civitai

31,428 views • 1 year ago

and Wan2.2 again. MoCam - novel view synthesis. Lets you change the camera trajectory of a photo/video. - Video Re-camera - 3D Photo Conversion - Cinematic Control - Precise orbital/pan/zoom control - Corrects geometric artifacts

and Wan2.2 again. MoCam - novel view synthesis. Lets you change the camera trajectory of a photo/video. - Video Re-camera - 3D Photo Conversion - Cinematic Control - Precise orbital/pan/zoom control - Corrects geometric artifacts

Wildminder

14,069 views • 3 months ago

NVIDIA just released AnyFlow on Hugging Face The first any-step video diffusion model that generates high-quality text-to-video with any inference budget - 4 steps or 50, quality scales smoothly without degradation.

NVIDIA just released AnyFlow on Hugging Face The first any-step video diffusion model that generates high-quality text-to-video with any inference budget - 4 steps or 50, quality scales smoothly without degradation.

DailyPapers

41,699 views • 3 months ago

Hauke Hilberg and I received early access to Stable Video from Stability AI We were curious to evaluate the platform's handling of movements. Here's what we created. No upscales, no effects added - just pure Stable Video footage, enhanced with Vivaldi's 'Storm'. Some observations: - You can either upload images or create them directly on the platform. - If you want your clip to have movements, consider using images of runners, cars, or well: dancers. - The quality of the movements is impressive. Thank you so much for giving us the opportunity to make our ideas come to life, Stability AI

Hauke Hilberg and I received early access to Stable Video from Stability AI We were curious to evaluate the platform's handling of movements. Here's what we created. No upscales, no effects added - just pure Stable Video footage, enhanced with Vivaldi's 'Storm'. Some observations: - You can either upload images or create them directly on the platform. - If you want your clip to have movements, consider using images of runners, cars, or well: dancers. - The quality of the movements is impressive. Thank you so much for giving us the opportunity to make our ideas come to life, Stability AI

Martin Haerlin

121,679 views • 2 years ago

🎬 Create a video from an image. With HPVideo’s premium models, you can turn static images into smooth, cinematic motion. From image to motion — in just a few steps. 👉 #HPVideo #AIVideo #ImageToVideo #GenerativeAI #CreatorTools

🎬 Create a video from an image. With HPVideo’s premium models, you can turn static images into smooth, cinematic motion. From image to motion — in just a few steps. 👉 #HPVideo #AIVideo #ImageToVideo #GenerativeAI #CreatorTools

HPVideo

31,357 views • 7 months ago

Check out CAT3D! Image(s)-to-3D in 1 minute! Given any number of real or generated images, CAT3D uses a multi-view diffusion prior to create consistent novel views. These views are used to reconstruct a 3D scene using NeRF/3DGS.

Check out CAT3D! Image(s)-to-3D in 1 minute! Given any number of real or generated images, CAT3D uses a multi-view diffusion prior to create consistent novel views. These views are used to reconstruct a 3D scene using NeRF/3DGS.

Philipp Henzler

12,747 views • 2 years ago

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

Stability AI

36,021 views • 1 year ago

Tried generating images with LLMs before? You're usually stuck with one model, one style, one result at a time. ASI:One breaks that.⚡ Now you can call multiple image generation agents from the same prompt, in the same window. Pick the best. Compare styles. Iterate faster. All powered by agents on Agentverse. Agents used: Stability AI based model on Hugging Face: BlackForestLabsAI - Unofficial on Hugging Face: OpenAI Dall-E: Watch now👇

Tried generating images with LLMs before? You're usually stuck with one model, one style, one result at a time. ASI:One breaks that.⚡ Now you can call multiple image generation agents from the same prompt, in the same window. Pick the best. Compare styles. Iterate faster. All powered by agents on Agentverse. Agents used: Stability AI based model on Hugging Face: BlackForestLabsAI - Unofficial on Hugging Face: OpenAI Dall-E: Watch now👇

Fetch.ai

29,075 views • 1 year ago

AI just fired the video editor 3D Motion just dropped the fastest way to turn any image into a cinematic video. With depth mapping, smooth camera moves, and movie-like lighting, it makes static photos look like Hollywood shots. Tutorial below + Prompts:

AI just fired the video editor 3D Motion just dropped the fastest way to turn any image into a cinematic video. With depth mapping, smooth camera moves, and movie-like lighting, it makes static photos look like Hollywood shots. Tutorial below + Prompts:

Robin Delta

137,796 views • 11 months ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 views • 3 years ago

Introducing; Mii Creator! A website that allows you to make your Mii! It doesn't stop there! It also allows you to create a full-body render of it with a free camera and change facial expressions! (and download a 3D model of your Mii!) Try it here:

Introducing; Mii Creator! A website that allows you to make your Mii! It doesn't stop there! It also allows you to create a full-body render of it with a free camera and change facial expressions! (and download a 3D model of your Mii!) Try it here:

FlipZ

918,098 views • 1 year ago

So Gemini can now generate complete presentations?! You can transform any document into an editable slideshow with images, charts, text, etc. - Upload a file or a GDocs document - Ask Gemini to create a presentation - Wait just a few minutes Done. And you can either download it or export it to Google Slides to edit it.

So Gemini can now generate complete presentations?! You can transform any document into an editable slideshow with images, charts, text, etc. - Upload a file or a GDocs document - Ask Gemini to create a presentation - Wait just a few minutes Done. And you can either download it or export it to Google Slides to edit it.

Paul Couvert

47,774 views • 9 months ago

You asked for it — PicPrompter now allows you to generate initial images from just a prompt! From there, you can ask AI to edit the image however you see fit. AI-first Photoshop is happening. What features do you want to see next? Try it:

You asked for it — PicPrompter now allows you to generate initial images from just a prompt! From there, you can ask AI to edit the image however you see fit. AI-first Photoshop is happening. What features do you want to see next? Try it:

Matt Shumer

43,528 views • 1 year ago

LiveCC just dropped on Hugging Face Learning Video LLM with Streaming Speech Transcription at Scale video LLM capable of real-time commentary, trained with a novel video-ASR streaming method, SOTA on both streaming and offline benchmarks.

LiveCC just dropped on Hugging Face Learning Video LLM with Streaming Speech Transcription at Scale video LLM capable of real-time commentary, trained with a novel video-ASR streaming method, SOTA on both streaming and offline benchmarks.

AK

45,967 views • 1 year ago

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,496 views • 2 years ago

"The music video tells a story of what life would be like if we were to change things or make an insignificant choice. The power of AI and stable diffusion lies in its ability to transport Peter to any location, dress in any fashion and experience dimensions beyond the real world." - Junie Lau Junie's Official Video for Peter's 'The Court' (Dark-Side Mix) is out now! Stability AI Stable Diffusion #diffusetogether

"The music video tells a story of what life would be like if we were to change things or make an insignificant choice. The power of AI and stable diffusion lies in its ability to transport Peter to any location, dress in any fashion and experience dimensions beyond the real world." - Junie Lau Junie's Official Video for Peter's 'The Court' (Dark-Side Mix) is out now! Stability AI Stable Diffusion #diffusetogether

Peter Gabriel

74,251 views • 2 years ago