Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Amazon presents Diffuse to Choose Enriching Image Conditioned Inpainting in Latent Diffusion Models for Virtual Try-All paper page: Diffuse to Choose (DTC) allows users to virtually place any e-commerce item in any setting, ensuring detailed, semantically coherent blending with realistic lighting and shadows.

AK

502,061 subscribers

118,231 görüntüleme • 2 yıl önce •via X (Twitter)

Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

10 Yorum

Alex Volkov (Thursd/AI) profil fotoğrafı

Alex Volkov (Thursd/AI)2 yıl önce

No demo or code but coming soon. Website for the project:

Marco Bertini profil fotoğrafı

Marco Bertini2 yıl önce

If someone is interested in virtual try-on I'd suggest checking our (by @A_Baldrati and @omedivad) ACM MM'23 paper with related code on Github:

Amazon Science profil fotoğrafı

Amazon Science2 yıl önce

More about Diffuse-to-Choose below.

Vojta Letal profil fotoğrafı

Vojta Letal2 yıl önce

Instead of ugly renders, fake products will have convincing images now... Great... Thanks...

Jasper profil fotoğrafı

Jasper2 yıl önce

I’ve been building this at excited to compare this to my own models

Vanja 🦘 profil fotoğrafı

Vanja 🦘2 yıl önce

any comment if it incorporates sizing?

curator profil fotoğrafı

curator2 yıl önce

There's a similar paper from the Alibaba group titled: "Outfit Anyone" and they also have a demo here to try it: The results are great, especially the details of clothes, but need improvement in the consistency of different outfits. Sadly, no code yet!

Nicolai Klemke profil fotoğrafı

Nicolai Klemke2 yıl önce

@dannypostmaa this could have been you 😁

Mehmet Saygın Seyfioğlu profil fotoğrafı

Mehmet Saygın Seyfioğlu2 yıl önce

Thank you @_akhaliq, for covering our work! Please see the following thread for more information! Also, please consider following so you won't miss it when we share the codebase and the model 😅

thepeeps profil fotoğrafı

thepeeps2 yıl önce

Cher's wardrobe app is getting closer!

Benzer Videolar

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 görüntüleme • 3 yıl önce

Introducing group chat on Poe: a new way to collaborate with any AI and anyone you know, all in a single conversation. Now available to all users worldwide, groups can interact with any of the 200+ text, image, video, or audio models on Poe, plus any creator-made bots. (1/8)

Introducing group chat on Poe: a new way to collaborate with any AI and anyone you know, all in a single conversation. Now available to all users worldwide, groups can interact with any of the 200+ text, image, video, or audio models on Poe, plus any creator-made bots. (1/8)

Poe

21,934 görüntüleme • 8 ay önce

Add shadows to your drawing with Shading Assist in Clip Studio Paint! There are several presets to choose from so you can find the perfect shadows for any scene. Adjust them with the manipulator and use it as is or as a guide for your artwork!

Add shadows to your drawing with Shading Assist in Clip Studio Paint! There are several presets to choose from so you can find the perfect shadows for any scene. Adjust them with the manipulator and use it as is or as a guide for your artwork!

CLIP STUDIO PAINT

355,329 görüntüleme • 11 ay önce

Introducing Ideogram Character -- the first character consistency model that works with just one reference image. Now available to all users for free! Create consistent character images in any style, expression, scene, and lighting. Who wants video next? 🥺

Introducing Ideogram Character -- the first character consistency model that works with just one reference image. Now available to all users for free! Create consistent character images in any style, expression, scene, and lighting. Who wants video next? 🥺

Ideogram

1,713,468 görüntüleme • 1 yıl önce

The Arena Robinhood Launchpad is live. - Trade and launch through the app at - Trade and launch through our trading terminal at All tokens graduate to a Uniswap V4 pool. Choose the chain. Choose what you pair your token with. Swap any token for any Robinhood token in either the app or trading terminal without needing to bridge. Be careful, always do your own research before trading any tokens, and most importantly HAVE FUN!

The Arena Robinhood Launchpad is live. - Trade and launch through the app at - Trade and launch through our trading terminal at All tokens graduate to a Uniswap V4 pool. Choose the chain. Choose what you pair your token with. Swap any token for any Robinhood token in either the app or trading terminal without needing to bridge. Be careful, always do your own research before trading any tokens, and most importantly HAVE FUN!

The Arena ⚔️

71,765 görüntüleme • 17 gün önce

Detailed Video Guide AquaFlux How to complete new tasks AquaFlux in Pharos | Mainnet Arc testnet? 👇 - Claim all available tokens in the Faucet tab; - Go to Structure tab and choose any asset what you want (I chose Private Credit) > Enter amount > Wrap & Split; - Go to Earn tab and choose the same as in the previous step (Private Credit for me) > Enter amount > Stake. ✅ Repeat each action: 41 times Structure and 41 times Earn (1000 points in total) #PharosStoryteller

Detailed Video Guide AquaFlux How to complete new tasks AquaFlux in Pharos | Mainnet Arc testnet? 👇 - Claim all available tokens in the Faucet tab; - Go to Structure tab and choose any asset what you want (I chose Private Credit) > Enter amount > Wrap & Split; - Go to Earn tab and choose the same as in the previous step (Private Credit for me) > Enter amount > Stake. ✅ Repeat each action: 41 times Structure and 41 times Earn (1000 points in total) #PharosStoryteller

Evaneskate

15,190 görüntüleme • 8 ay önce

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 görüntüleme • 3 yıl önce

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution paper page: Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior. However, applying these models to video super-resolution remains challenging due to the high demands for output fidelity and temporal consistency, which is complicated by the inherent randomness in diffusion models. Our study introduces Upscale-A-Video, a text-guided latent diffusion framework for video upscaling. This framework ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences; globally, without training, a flow-guided recurrent latent propagation module is introduced to enhance overall video stability by propagating and fusing latent across the entire sequences. Thanks to the diffusion paradigm, our model also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation, enabling a trade-off between fidelity and quality. Extensive experiments show that Upscale-A-Video surpasses existing methods in both synthetic and real-world benchmarks, as well as in AI-generated videos, showcasing impressive visual realism and temporal consistency.

AK

32,849 görüntüleme • 2 yıl önce

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

AK

16,062 görüntüleme • 2 yıl önce

Google has added its flight simulator to the web version of Google Earth, making it available to all users worldwide. >The feature runs directly in any modern browser at with no downloads or installations required. >Users can open the simulator from the Tools menu after selecting Explore and pilot a virtual aircraft over detailed 3D terrain, buildings, and satellite imagery across the entire planet. >It uses simplified flight physics designed for casual exploration and works with standard mouse and keyboard controls.

Google has added its flight simulator to the web version of Google Earth, making it available to all users worldwide. >The feature runs directly in any modern browser at with no downloads or installations required. >Users can open the simulator from the Tools menu after selecting Explore and pilot a virtual aircraft over detailed 3D terrain, buildings, and satellite imagery across the entire planet. >It uses simplified flight physics designed for casual exploration and works with standard mouse and keyboard controls.

Pirat_Nation 🔴

100,095 görüntüleme • 1 ay önce

True values are often revealed in how we choose to communicate with the world. This video presents a striking contrast in expression and public discourse between two individuals. While one focuses on divisive rhetoric and territorial claims the other emphasizes gratitude and community blessings. Cultivating a respectful mindset is essential for any leader or influencer aiming to build a lasting and positive legacy.

True values are often revealed in how we choose to communicate with the world. This video presents a striking contrast in expression and public discourse between two individuals. While one focuses on divisive rhetoric and territorial claims the other emphasizes gratitude and community blessings. Cultivating a respectful mindset is essential for any leader or influencer aiming to build a lasting and positive legacy.

गुरुकुलम

95,791 görüntüleme • 3 ay önce

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,511 görüntüleme • 2 yıl önce

Loopy Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency paper page: With the introduction of diffusion-based video generation techniques, audio-conditioned human video generation has recently achieved significant breakthroughs in both the naturalness of motion and the synthesis of portrait details. Due to the limited control of audio signals in driving human motion, existing methods often add auxiliary spatial signals to stabilize movements, which may compromise the naturalness and freedom of motion. In this paper, we propose an end-to-end audio-only conditioned video diffusion model named Loopy. Specifically, we designed an inter- and intra-clip temporal module and an audio-to-latents module, enabling the model to leverage long-term motion information from the data to learn natural motion patterns and improving audio-portrait movement correlation. This method removes the need for manually specified spatial motion templates used in existing methods to constrain motion during inference. Extensive experiments show that Loopy outperforms recent audio-driven portrait diffusion models, delivering more lifelike and high-quality results across various scenarios.

AK

128,852 görüntüleme • 1 yıl önce

META automatically opted all users into its new deepfake AI system “Meta is facing backlash over a new AI feature just rolled out on Instagram” The app is called Muse Image. It allows users to make realistic deepfakes using other people's photos. Any adult with a public Instagram account was automatically included. There are ways to turn it off and opt out, if you don’t do this everything you post of yourself can be used for deepfakes - You go to your settings and search sharing and reuse - Then you scroll down to allow people to ‘create with and reuse your content’ and toggle both of those off

META automatically opted all users into its new deepfake AI system “Meta is facing backlash over a new AI feature just rolled out on Instagram” The app is called Muse Image. It allows users to make realistic deepfakes using other people's photos. Any adult with a public Instagram account was automatically included. There are ways to turn it off and opt out, if you don’t do this everything you post of yourself can be used for deepfakes - You go to your settings and search sharing and reuse - Then you scroll down to allow people to ‘create with and reuse your content’ and toggle both of those off

Wall Street Apes

49,848 görüntüleme • 1 ay önce

This is gonna blow your mind!🤯💥 I have been creating the "Item Customizer" mod in Hytale #Hytale #HytaleMod Allows players to fully customize any item in-game LIVE👀 Inside the game, no restart needed either. It generates a unique copy of the item, does not replace the original vanilla item files.🔥 In this UI: - Re-paint/Edit the item's texture with advanced tools. - Rename the item's name. - OP players can fully configurate all settings and behavior of the item. - OP players can also replace the original vanilla item instead. - OP players can change all damage/similar values, and even change the item's interactions!!! Planned: - For now can only customize non-placeable items, so I'll be adding an spray-painter to allow OP players paint any placeable item (blocks). - I'll be adding a new tab in the UI to edit the crafting recipe of the item. -I'll be adding a custom bench for this Item Customizer UI to open.

This is gonna blow your mind!🤯💥 I have been creating the "Item Customizer" mod in Hytale #Hytale #HytaleMod Allows players to fully customize any item in-game LIVE👀 Inside the game, no restart needed either. It generates a unique copy of the item, does not replace the original vanilla item files.🔥 In this UI: - Re-paint/Edit the item's texture with advanced tools. - Rename the item's name. - OP players can fully configurate all settings and behavior of the item. - OP players can also replace the original vanilla item instead. - OP players can change all damage/similar values, and even change the item's interactions!!! Planned: - For now can only customize non-placeable items, so I'll be adding an spray-painter to allow OP players paint any placeable item (blocks). - I'll be adding a new tab in the UI to edit the crafting recipe of the item. -I'll be adding a custom bench for this Item Customizer UI to open.

Starky / StarkyMods

19,089 görüntüleme • 2 ay önce

We're launching Computer Use in smolagents! 🥳 -> As vision models become more capable, they become able to power complex agentic workflows. Especially Qwen-VL models, that support built-in grounding, i.e. ability to locate any element in an image by its coordinates, thus to click any item on a screenshot. So we worked with Leandro von Werra and Miquel Farré to integrate Computer Use in smolagents. 🚀 Our Computer Use agent works wonders! I asked it how long the soldiers from Alexander had walked from their departure in Macedonia, all the way to India, when they decided they were too tired to continue. => It turns out, they had walked quite a bit!

We're launching Computer Use in smolagents! 🥳 -> As vision models become more capable, they become able to power complex agentic workflows. Especially Qwen-VL models, that support built-in grounding, i.e. ability to locate any element in an image by its coordinates, thus to click any item on a screenshot. So we worked with Leandro von Werra and Miquel Farré to integrate Computer Use in smolagents. 🚀 Our Computer Use agent works wonders! I asked it how long the soldiers from Alexander had walked from their departure in Macedonia, all the way to India, when they decided they were too tired to continue. => It turns out, they had walked quite a bit!

m_ric

43,850 görüntüleme • 1 yıl önce

SORA by Hand ✍️ OpenAI’s #SORA took over the Internet when it was announced earlier this year. The technology behind Sora is the Diffusion Transformer (DiT) developed by William Peebles and Shining Xie. How does DiT work? 𝗚𝗼𝗮𝗹: Generate a video conditioned by a text prompt and a series of diffusion steps [1] Given ↳ Video ↳ Prompt: "sora is sky" ↳ Diffusion step: t = 3 [2] Video → Patches ↳ Divide all pixels in all frames into 4 spacetime patches [3] Visual Encoder: Pixels 🟨 → Latent 🟩 ↳ Multiply the patches with weights and biases, followed by ReLU ↳ The result is a latent feature vector per patch ↳ The purpose is dimension reduction from 4 (2x2x1) to 2 (2x1). ↳ In the paper, the reduction is 196,608 (256x256x3)→ 4096 (32x32x4) [4] ⬛ Add Noise ↳ Sample a noise according to the diffusion time step t. Typically, the larger the t, the smaller the noise. ↳ Add the Sampled Noise to latent features to obtain Noised Latent. ↳ The goal is to purposely add noise to a video and ask the model to guess what that noise is. ↳ This is analogous to training a language model by purposely deleting a word in a sentence and ask the model to guess what the deleted word was. [5-7] 🟪 Conditioning by Adaptive Layer Norm [5] Encode Conditions ↳ Encode "sora is sky" into a text embedding vector [0,1,-1]. ↳ Encode t = 3 to as a binary vector [1,1]. ↳ Concatenate the two vectors in to a 5D column vector. [6] Estimate Scale/Shift ↳ Multiply the combined vector with weights and biases ↳ The goal is to estimate the scale [2,-1] and shift [-1,5]. ↳ Copy the result to (X) and (+) [7] Apply Scale/Sift ↳ Scale the noised latent by [2,-1] ↳ Shifted the scaled noised latent by [-1, 5] ↳ The result is "conditioned" noise latent. [8-10] Transformer [8] Self-Attention ↳ Feed the conditioned noised latent to Query-Key function to obtain a self-attention matrix ↳ Value is omitted for simplicity [9] Attention Pooling ↳ Multiply the conditioned noised latent with the self-attention matrix ↳ The result are attention weighted features [10] Pointwise Feed Forward Network ↳ Multiply the attention weighted features with weights and biases ↳ The result is the Predicted Noise 🏋️‍♂️ 𝗧𝗿𝗮𝗶𝗻 [11] ↳ Calculate MSE loss gradients by taking the different between the Predicted Noise and the Sampled Noise (ground truth). ↳ Use the loss gradients to kick off backpropagation to update all learnable parameters (red borders) ↳ Note the visual encoder and decoder's parameters are frozen (blue borders) 🎨 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲 (𝗦𝗮𝗺𝗽𝗹𝗲) [12] Denoise ↳ Subtract the predicted noise from the noised latent to obtain the noise-free latent [13] Visual Decoder: Latent 🟩 → Pixels 🟨 ↳ Multiply the patches with weights and biases, followed by ReLU [14] Patches → Video ↳ Rearrange patches into a sequence of video frames.

SORA by Hand ✍️ OpenAI’s #SORA took over the Internet when it was announced earlier this year. The technology behind Sora is the Diffusion Transformer (DiT) developed by William Peebles and Shining Xie. How does DiT work? 𝗚𝗼𝗮𝗹: Generate a video conditioned by a text prompt and a series of diffusion steps [1] Given ↳ Video ↳ Prompt: "sora is sky" ↳ Diffusion step: t = 3 [2] Video → Patches ↳ Divide all pixels in all frames into 4 spacetime patches [3] Visual Encoder: Pixels 🟨 → Latent 🟩 ↳ Multiply the patches with weights and biases, followed by ReLU ↳ The result is a latent feature vector per patch ↳ The purpose is dimension reduction from 4 (2x2x1) to 2 (2x1). ↳ In the paper, the reduction is 196,608 (256x256x3)→ 4096 (32x32x4) [4] ⬛ Add Noise ↳ Sample a noise according to the diffusion time step t. Typically, the larger the t, the smaller the noise. ↳ Add the Sampled Noise to latent features to obtain Noised Latent. ↳ The goal is to purposely add noise to a video and ask the model to guess what that noise is. ↳ This is analogous to training a language model by purposely deleting a word in a sentence and ask the model to guess what the deleted word was. [5-7] 🟪 Conditioning by Adaptive Layer Norm [5] Encode Conditions ↳ Encode "sora is sky" into a text embedding vector [0,1,-1]. ↳ Encode t = 3 to as a binary vector [1,1]. ↳ Concatenate the two vectors in to a 5D column vector. [6] Estimate Scale/Shift ↳ Multiply the combined vector with weights and biases ↳ The goal is to estimate the scale [2,-1] and shift [-1,5]. ↳ Copy the result to (X) and (+) [7] Apply Scale/Sift ↳ Scale the noised latent by [2,-1] ↳ Shifted the scaled noised latent by [-1, 5] ↳ The result is "conditioned" noise latent. [8-10] Transformer [8] Self-Attention ↳ Feed the conditioned noised latent to Query-Key function to obtain a self-attention matrix ↳ Value is omitted for simplicity [9] Attention Pooling ↳ Multiply the conditioned noised latent with the self-attention matrix ↳ The result are attention weighted features [10] Pointwise Feed Forward Network ↳ Multiply the attention weighted features with weights and biases ↳ The result is the Predicted Noise 🏋️‍♂️ 𝗧𝗿𝗮𝗶𝗻 [11] ↳ Calculate MSE loss gradients by taking the different between the Predicted Noise and the Sampled Noise (ground truth). ↳ Use the loss gradients to kick off backpropagation to update all learnable parameters (red borders) ↳ Note the visual encoder and decoder's parameters are frozen (blue borders) 🎨 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲 (𝗦𝗮𝗺𝗽𝗹𝗲) [12] Denoise ↳ Subtract the predicted noise from the noised latent to obtain the noise-free latent [13] Visual Decoder: Latent 🟩 → Pixels 🟨 ↳ Multiply the patches with weights and biases, followed by ReLU [14] Patches → Video ↳ Rearrange patches into a sequence of video frames.

Tom Yeh

238,286 görüntüleme • 2 yıl önce

Godzilla: Destroy All Monsters Melee Remastered Announced -Coming November 3 -The original 2002 Godzilla arena fighter, remastered. -12 Kaiju to choose from, each with a uniquely vicious moveset. -8 locations, including recreated real-world cities, Monster Island, and the Alien Mothership. -Gallery items to unlock. -Play with friends in local co-op and all-new online multiplayer. I-mproved unlock system gives you the choice to unlock monsters, locations, and gallery items in any order. Improved unlock system gives you the choice to unlock monsters, locations, and gallery items in any order. -Also includes additional single-player campaigns for each kaiju.

Godzilla: Destroy All Monsters Melee Remastered Announced -Coming November 3 -The original 2002 Godzilla arena fighter, remastered. -12 Kaiju to choose from, each with a uniquely vicious moveset. -8 locations, including recreated real-world cities, Monster Island, and the Alien Mothership. -Gallery items to unlock. -Play with friends in local co-op and all-new online multiplayer. I-mproved unlock system gives you the choice to unlock monsters, locations, and gallery items in any order. Improved unlock system gives you the choice to unlock monsters, locations, and gallery items in any order. -Also includes additional single-player campaigns for each kaiju.

Chito

53,881 görüntüleme • 2 ay önce

For people who don’t get the 2D gaming on virtual monitors in VR use case: HMDs have the potential to be the ultimate monitor for any content. Any aspect ratio, any resolution, any size, it’s all “native” in vr I get to play my game on a massive screen while I have spatial awareness, see my wife, take a sip of coffee, and have depth correct rendering content in my room. If and when I choose to, I can “turn off” my surrounds and go into a virtual environment. It’s MR by default, VR by choice Nobody has really taken this use case all that seriously and optimized the hardware/software around it. Now, it seems like Valve is. Also some context: I’m streaming SW Jedi Fallen Order at 4k 90fps my PC to my Vision Pro, and playing the audio thru my surround sound speakers. It’s the best of ALL worlds

For people who don’t get the 2D gaming on virtual monitors in VR use case: HMDs have the potential to be the ultimate monitor for any content. Any aspect ratio, any resolution, any size, it’s all “native” in vr I get to play my game on a massive screen while I have spatial awareness, see my wife, take a sip of coffee, and have depth correct rendering content in my room. If and when I choose to, I can “turn off” my surrounds and go into a virtual environment. It’s MR by default, VR by choice Nobody has really taken this use case all that seriously and optimized the hardware/software around it. Now, it seems like Valve is. Also some context: I’m streaming SW Jedi Fallen Order at 4k 90fps my PC to my Vision Pro, and playing the audio thru my surround sound speakers. It’s the best of ALL worlds

Azad Balabanian

616,738 görüntüleme • 9 ay önce