Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

The code of GSPN #CVPR2025 is released! We proposed a new sqrt(N) complexity attention mechanism, which enables efficient high resolution image generation. We can generate 8k images with 42x speed up compared to self-attention in StableDiffusionXL! Code: Paper:

Xiaolong Wang

21,955 subscribers

354,887 Aufrufe • vor 1 Jahr •via X (Twitter)

Gesundheit & Wellness Wissenschaft & Technologie Bildung #CVPR2025

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Rawlala

Rawlalavor 1 Jahr

Can we see the images 😁

Profilbild von Qodo

Qodovor 1 Jahr

Pro tip: turn review feedback into ready-to-commit code changes instantly with Qodo Merge's /implement command.

Profilbild von Worldrover

Worldrovervor 1 Jahr

Who did you steal it from?

Profilbild von Greg Huntoon

Greg Huntoonvor 1 Jahr

Me trying to follow along

Profilbild von TheNan

TheNanvor 1 Jahr

Why didn’t I think of that?🧐😁

Profilbild von WDWDVC🌴

WDWDVC🌴vor 1 Jahr

But they aren't Mickey shaped

Profilbild von Harold

Haroldvor 1 Jahr

Yes

Profilbild von Zin 🧀💊

Zin 🧀💊vor 1 Jahr

nice work

Profilbild von boisselle

boissellevor 1 Jahr

Interesting work. The CUDA algorithms are especially cool!

Profilbild von Ann Lewis

Ann Lewisvor 1 Jahr

His advice is very good. @GraysonQuant

Ähnliche Videos

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,285 Aufrufe • vor 1 Jahr

Kalman Filtering + Deep Attention Models = LOVE ❤️ #CoRL '23 paper: we identify a link between Kalman filters and neural attention mechanisms. Idea: the Kalman-gain is a form of attention! No need to tweak parameters. Check website and code: ASU Robotics

Kalman Filtering + Deep Attention Models = LOVE ❤️ #CoRL '23 paper: we identify a link between Kalman filters and neural attention mechanisms. Idea: the Kalman-gain is a form of attention! No need to tweak parameters. Check website and code: ASU Robotics

Heni Ben Amor

19,054 Aufrufe • vor 2 Jahren

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!

Felix Heide

19,117 Aufrufe • vor 2 Jahren

Introducing Omma Studio, a full suite of AI tools to generate 3d, video/image, and audio with SOTA models. Everything made in Studio can integrate directly with your Omma's chat code projects to deploy/publish to the web. Content can be generated in parallel alongside your code generation in Omma, too. Studio -> Code -> Deploy

Introducing Omma Studio, a full suite of AI tools to generate 3d, video/image, and audio with SOTA models. Everything made in Studio can integrate directly with your Omma's chat code projects to deploy/publish to the web. Content can be generated in parallel alongside your code generation in Omma, too. Studio -> Code -> Deploy

Omma AI

22,864 Aufrufe • vor 1 Monat

FlexAttention is a novel compiler-driven programming model that allows implementing the majority of attention variants in a few lines of idiomatic PyTorch code Boyuan Feng & Avik Chaudhuri show how many existing attention variants can be implemented via FlexAttention & that we achieve competitive performance compared to handwritten kernels. 📺 Watch the full video on YouTube:

FlexAttention is a novel compiler-driven programming model that allows implementing the majority of attention variants in a few lines of idiomatic PyTorch code Boyuan Feng & Avik Chaudhuri show how many existing attention variants can be implemented via FlexAttention & that we achieve competitive performance compared to handwritten kernels. 📺 Watch the full video on YouTube:

PyTorch

17,929 Aufrufe • vor 1 Jahr

Four months ago we released "Empire by Code" which laid out a thesis for the rise of USD Stablecoins. We have now released “Stablecoin Wars" which deals with the various factions vying for control of this new system. Both can be found at the link below.

Four months ago we released "Empire by Code" which laid out a thesis for the rise of USD Stablecoins. We have now released “Stablecoin Wars" which deals with the various factions vying for control of this new system. Both can be found at the link below.

Santiago Capital

117,744 Aufrufe • vor 5 Monaten

Everyone can now train high-dimensional muscle agents with RL, as we just released our code! Come join me at the #MyoChallenge workshop tomorrow at 10:00 in room 354 at #NeurIPS2023 to see my poster. Code is here!

Everyone can now train high-dimensional muscle agents with RL, as we just released our code! Come join me at the #MyoChallenge workshop tomorrow at 10:00 in room 354 at #NeurIPS2023 to see my poster. Code is here!

Pierre Schumacher

159,528 Aufrufe • vor 2 Jahren

I'm super pumped about this one—we partnered with @heytavus on a plugin to simplify the process of building realtime AI avatars. There's a crazy amount of complexity to realtime procedural video generation that we just handle. In two lines of code—add an avatar to any agent.

Shayne

45,090 Aufrufe • vor 1 Jahr

This is insane! I just used the new Claude Code Playground plugin to level up my Nano Banana Image generator skill. My skill has a self-improving loop, but with the playground skill, I can also pass precise annotations to nano banana as it improves the images. It's so good!

This is insane! I just used the new Claude Code Playground plugin to level up my Nano Banana Image generator skill. My skill has a self-improving loop, but with the playground skill, I can also pass precise annotations to nano banana as it improves the images. It's so good!

elvis

286,768 Aufrufe • vor 6 Monaten

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:

Stability AI

976,456 Aufrufe • vor 2 Jahren

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein

Matthias Niessner

21,891 Aufrufe • vor 1 Jahr

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors paper page: present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors. In the first stage, we optimize a neural radiance field to produce a coarse geometry. In the second stage, we adopt a memory-efficient differentiable mesh representation to yield a high-resolution mesh with a visually appealing texture. In both stages, the 3D content is learned through reference view supervision and novel views guided by a combination of 2D and 3D diffusion priors. We introduce a single trade-off parameter between the 2D and 3D priors to control exploration (more imaginative) and exploitation (more precise) of the generated geometry. Additionally, we employ textual inversion and monocular depth regularization to encourage consistent appearances across views and to prevent degenerate solutions, respectively. Magic123 demonstrates a significant improvement over previous image-to-3D techniques, as validated through extensive experiments on synthetic benchmarks and diverse real-world images.

AK

305,663 Aufrufe • vor 3 Jahren

📣 New research from GenAI at Meta, introducing Meta 3D Gen: A new system for end-to-end generation of 3D assets from text in <1min. Meta 3D Gen is a new combined AI system that can generate high-quality 3D assets, with both high-resolution textures and material maps end-to-end, producing results that are superior to existing solutions — at 3-10x the speed of existing work in this space. Details in the technical report ➡️

📣 New research from GenAI at Meta, introducing Meta 3D Gen: A new system for end-to-end generation of 3D assets from text in <1min. Meta 3D Gen is a new combined AI system that can generate high-quality 3D assets, with both high-resolution textures and material maps end-to-end, producing results that are superior to existing solutions — at 3-10x the speed of existing work in this space. Details in the technical report ➡️

AI at Meta

408,833 Aufrufe • vor 2 Jahren

T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation paper page: Despite the stunning ability to generate high-quality images by recent text-to-image models, current approaches often struggle to effectively compose objects with different attributes and relationships into a complex and coherent scene. We propose T2I-CompBench, a comprehensive benchmark for open-world compositional text-to-image generation, consisting of 6,000 compositional text prompts from 3 categories (attribute binding, object relationships, and complex compositions) and 6 sub-categories (color binding, shape binding, texture binding, spatial relationships, non-spatial relationships, and complex compositions). We further propose several evaluation metrics specifically designed to evaluate compositional text-to-image generation. We introduce a new approach, Generative mOdel fine-tuning with Reward-driven Sample selection (GORS), to boost the compositional text-to-image generation abilities of pretrained text-to-image models. Extensive experiments and evaluations are conducted to benchmark previous methods on T2I-CompBench, and to validate the effectiveness of our proposed evaluation metrics and GORS approach.

T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation paper page: Despite the stunning ability to generate high-quality images by recent text-to-image models, current approaches often struggle to effectively compose objects with different attributes and relationships into a complex and coherent scene. We propose T2I-CompBench, a comprehensive benchmark for open-world compositional text-to-image generation, consisting of 6,000 compositional text prompts from 3 categories (attribute binding, object relationships, and complex compositions) and 6 sub-categories (color binding, shape binding, texture binding, spatial relationships, non-spatial relationships, and complex compositions). We further propose several evaluation metrics specifically designed to evaluate compositional text-to-image generation. We introduce a new approach, Generative mOdel fine-tuning with Reward-driven Sample selection (GORS), to boost the compositional text-to-image generation abilities of pretrained text-to-image models. Extensive experiments and evaluations are conducted to benchmark previous methods on T2I-CompBench, and to validate the effectiveness of our proposed evaluation metrics and GORS approach.

AK

17,503 Aufrufe • vor 3 Jahren

AI is moving fast. And OpenAI's new image model proves it — knowing how to use AI images to your advantage is now crucial for designers. So we teamed up with Nick St. Pierre (The AI Image Master) to walkthrough one of the best AI image generation tools right now, Midjourney, and build a full website using only AI images, in 1 hour. Watch the full breakdown👇

AI is moving fast. And OpenAI's new image model proves it — knowing how to use AI images to your advantage is now crucial for designers. So we teamed up with Nick St. Pierre (The AI Image Master) to walkthrough one of the best AI image generation tools right now, Midjourney, and build a full website using only AI images, in 1 hour. Watch the full breakdown👇

Relume

14,898 Aufrufe • vor 1 Jahr

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

V3D Video Diffusion Models are Effective 3D Generators Automatic 3D generation has recently attracted widespread attention. Recent methods have greatly accelerated the generation speed, but usually produce less-detailed objects due to limited model capacity or 3D data. Motivated by recent advancements in video diffusion models, we introduce V3D, which leverages the world simulation capacity of pre-trained video diffusion models to facilitate 3D generation. To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator. Benefiting from this, the state-of-the-art video diffusion model could be fine-tuned to generate 360degree orbit frames surrounding an object given a single image. With our tailored reconstruction pipelines, we can generate high-quality meshes or 3D Gaussians within 3 minutes. Furthermore, our method can be extended to scene-level novel view synthesis, achieving precise control over the camera path with sparse input views. Extensive experiments demonstrate the superior performance of the proposed approach, especially in terms of generation quality and multi-view consistency

AK

31,997 Aufrufe • vor 2 Jahren

Extract accurate depth from your 3D gaussians! Fast, accurate, and easy to use. Code of RaDe-GS is released! project page: code: paper: No need to change your 3D GS representations, we propose a play-and-plug rasterizer to extract accurate depth and normal from 3D gaussians, leading to the possibility of high-quality geometry reconstruction. It's quite efficient and fast, achieving 0.69mm errors on DTU dataset in only 5 minutes. Very easy to adopt the rasterizer in your any 3dgs projects.

Extract accurate depth from your 3D gaussians! Fast, accurate, and easy to use. Code of RaDe-GS is released! project page: code: paper: No need to change your 3D GS representations, we propose a play-and-plug rasterizer to extract accurate depth and normal from 3D gaussians, leading to the possibility of high-quality geometry reconstruction. It's quite efficient and fast, achieving 0.69mm errors on DTU dataset in only 5 minutes. Very easy to adopt the rasterizer in your any 3dgs projects.

Xiao-Xiao Long

31,305 Aufrufe • vor 2 Jahren

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate? 🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch. 🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks. Code is fully released; check out our: Homepage: Paper link: Github:

Haoran Geng

95,549 Aufrufe • vor 1 Jahr