正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

ControlNet is a neural network structure to control diffusion models by adding extra conditions, processing any of photo, text, or logo and getting hundreds of design options in just a couple of clicks

Massimo

4,437,680 subscribers

213,965 次观看 • 3 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

9 条评论

dudeguy 的头像

dudeguy3 年前

okay but what the fuck is that picture you picked

Jelena Perfiljeva 的头像

Jelena Perfiljeva3 年前

Always love your tweets but can we please not promote that picture? How do you think it makes many women feel to see this?

Chiguy 的头像

Chiguy3 年前

This is scary because older people, and most people really, will get to the point we will never know what’s real and what’s not anymore. All the doctored videos and pics people make that are fake to rile up everyone.

Liquirizia 的头像

Liquirizia3 年前

This is the entrance of Witherun lol

Deviant Weather Observer 的头像

Deviant Weather Observer3 年前

😁

Jimmy Wine 的头像

Jimmy Wine3 年前

Whiterun

Blaze 的头像

Blaze3 年前

graphic design schools and studios in 10 years

🇺🇦Sonia! 的头像

🇺🇦Sonia!3 年前

Damn, it's an unsubscribe for me. Promoting AI based on theft is cringe at this point.

brena se ha ido a blusky 的头像

brena se ha ido a blusky3 年前

i love the stuff you guys post but pls stop posting Ais related content

相关视频

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵

Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases. Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools. 🧵

Thomas Kipf

98,036 次观看 • 2 年前

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,231 次观看 • 3 年前

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,681 次观看 • 3 年前

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 次观看 • 3 年前

Exploring animated intros with text/logo #warpfusion Stable Diffusion #midjourney #ai #stablediffusion #controlnet #animation #aiar

Mrboofy

180,012 次观看 • 3 年前

A lot of people think that diffusion LLMs == BERT. But masked dLLMs are just one form of text diffusion, popularized by Subham Sahoo and his group at Cornell. There's much more to explore in text diffusion land! Check out our conversation (link below)!

A lot of people think that diffusion LLMs == BERT. But masked dLLMs are just one form of text diffusion, popularized by Subham Sahoo and his group at Cornell. There's much more to explore in text diffusion land! Check out our conversation (link below)!

Julia Turc

26,124 次观看 • 6 个月前

"In the future, you won't need any software at all. The neural network will completely replace it." - stephen balaban, co-founder of Lambda "Here's the idea: instead of generating a program, just go to ChatGPT and say, 'Hi, please behave like a calculator or a spreadsheet.'" "'Generate an ASCII user interface for me. And I want you to essentially just respond, implement the logic of that program in your mind.'" "This is what I call neural software. Normal software is really brittle. 'Neural software' can't have bugs, only misunderstandings." "Large language models will be the software you interact with." From Stephen's appearance on the show in May.

"In the future, you won't need any software at all. The neural network will completely replace it." - stephen balaban, co-founder of Lambda "Here's the idea: instead of generating a program, just go to ChatGPT and say, 'Hi, please behave like a calculator or a spreadsheet.'" "'Generate an ASCII user interface for me. And I want you to essentially just respond, implement the logic of that program in your mind.'" "This is what I call neural software. Normal software is really brittle. 'Neural software' can't have bugs, only misunderstandings." "Large language models will be the software you interact with." From Stephen's appearance on the show in May.

TBPN

28,665 次观看 • 11 个月前

Google just released Muse: a text to image model achieving state of the art performance. It is significantly more efficient than diffusion models. Muse also allows for inpainting, outpainting, and mask-free editing without finetuning or inverting the model.

Google just released Muse: a text to image model achieving state of the art performance. It is significantly more efficient than diffusion models. Muse also allows for inpainting, outpainting, and mask-free editing without finetuning or inverting the model.

bleedingedge.ai

183,766 次观看 • 3 年前

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

AK

117,091 次观看 • 2 年前

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies *perception-planning-control*, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:

🏃Motion Tokenizer Bridging Semantic and Kinematic Conditions🏃‍♀️ #MoTok is a diffusion-based tokenizer that unifies perception-planning-control, combining the strengths of continuous diffusion & discrete tokens - Project: - Code:

Ziwei Liu

19,259 次观看 • 7 个月前

Stable Diffusion is now capable of creating photo realistic full 3D models from single images. The amount of ways it could be used in video games and the metaverse blows my mind. AI is getting closer and closer to futuristic sci-fi movies!

Stable Diffusion is now capable of creating photo realistic full 3D models from single images. The amount of ways it could be used in video games and the metaverse blows my mind. AI is getting closer and closer to futuristic sci-fi movies!

CONTROLLER TALK

224,122 次观看 • 3 年前

In this experiment you can bring your scribbles to life with just a prompt! 🎨 I've used Stable Diffusion, ControlNet, a 360° Lora and 8th Wall's Sky segmentation. #stablediffusion #controlnet #threejs #8thwall

In this experiment you can bring your scribbles to life with just a prompt! 🎨 I've used Stable Diffusion, ControlNet, a 360° Lora and 8th Wall's Sky segmentation. #stablediffusion #controlnet #threejs #8thwall

Stijn Spanhove

39,212 次观看 • 3 年前

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,729 次观看 • 3 年前

visualization of a simple neural network

visualization of a simple neural network

ℏεsam

79,716 次观看 • 1 年前

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

Dreamix: Video Diffusion Models are General Video Editors abs: project page: present diffusion-based method that is able to perform text-based motion and appearance editing of general videos

AK

398,254 次观看 • 3 年前

I spent the weekend hacking an AI room makeover using ControlNet & Stable Diffusion, restyling my parent's "drawing room." How soon until this workflow is standard for home décor & interior design? ControlNet is a glimpse into the AI-infused future of 3D rendering & AR.

I spent the weekend hacking an AI room makeover using ControlNet & Stable Diffusion, restyling my parent's "drawing room." How soon until this workflow is standard for home décor & interior design? ControlNet is a glimpse into the AI-infused future of 3D rendering & AR.

Bilawal Sidhu

473,319 次观看 • 3 年前

CATransformers is a carbon-driven neural architecture and system hardware co-design framework. Using CATransformers, we discover greener CLIP models that achieve an average of 9.1% reduction potential in total lifecycle carbon emissions while maintaining accuracy (or increasing accuracy) and latency. This research is the first to look into carbon-driven neural architecture and system hardware co-design. It is enabled by a first-of-its-kind architectural carbon modeling tool – ACT, which we developed at FAIR. Check out: Our paper ➡️ code repository ➡️ and the additional carbon design tools and research artifacts in Sustainable AI ➡️

CATransformers is a carbon-driven neural architecture and system hardware co-design framework. Using CATransformers, we discover greener CLIP models that achieve an average of 9.1% reduction potential in total lifecycle carbon emissions while maintaining accuracy (or increasing accuracy) and latency. This research is the first to look into carbon-driven neural architecture and system hardware co-design. It is enabled by a first-of-its-kind architectural carbon modeling tool – ACT, which we developed at FAIR. Check out: Our paper ➡️ code repository ➡️ and the additional carbon design tools and research artifacts in Sustainable AI ➡️

AI at Meta

32,738 次观看 • 1 年前