Загрузка видео...

Не удалось загрузить видео

На главную

NVIDIA AI Released DiffusionRenderer: An AI Model for Editable, Photorealistic 3D Scenes from a Single Video In a groundbreaking new paper, researchers at NVIDIA, University of Toronto, Vector Institute and the University of Illinois Urbana-Champaign have unveiled a framework that directly tackles this challenge. DiffusionRenderer represents a revolutionary leap...

Marktechpost's profile picture

Marktechpost AI Dev News ⚡

9,310 subscribers

104,741 просмотров • 1 год назад •via X (Twitter)

Наука и технологииНовости и политика
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

As announced in partnership with NVIDIA at CES, we’re excited to introduce Stable Point Aware 3D (SPAR3D), setting a new standard in 3D generation. Ideal for running on NVIDIA RTX AI PCs, SPAR3D enables real-time editing and complete structure generation of 3D objects from a single image in under a second. You can download the weights on Hugging Face and code on GitHub, or access the model through the Stability AI API. Learn more here: (1/3)
0:13

As announced in partnership with NVIDIA at CES, we’re excited to introduce Stable Point Aware 3D (SPAR3D), setting a new standard in 3D generation. Ideal for running on NVIDIA RTX AI PCs, SPAR3D enables real-time editing and complete structure generation of 3D objects from a single image in under a second. You can download the weights on Hugging Face and code on GitHub, or access the model through the Stability AI API. Learn more here: (1/3)

Stability AI

181,441 просмотров • 1 год назад

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,039 просмотров • 1 год назад

Selected as a best paper finalist at #CVPR2026: PixelDiT from NVIDIA Research In most image generation models, a pretrained autoencoder compresses the image before any diffusion happens, causing quality loss that accumulates across the entire pipeline. PixelDiT, or Pixel Diffusion Transformers, removes this step entirely. It's a single-stage model that learns the diffusion process directly in pixel space, end-to-end.
0:18

Selected as a best paper finalist at #CVPR2026: PixelDiT from NVIDIA Research In most image generation models, a pretrained autoencoder compresses the image before any diffusion happens, causing quality loss that accumulates across the entire pipeline. PixelDiT, or Pixel Diffusion Transformers, removes this step entirely. It's a single-stage model that learns the diffusion process directly in pixel space, end-to-end.

NVIDIA AI

27,833 просмотров • 1 месяц назад

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 просмотров • 1 год назад

Open models are at the center of one of the most important shifts in AI. At #NVIDIAGTC, NVIDIA founder and CEO Jensen Huang is bringing together leaders from some of the most exciting AI companies to discuss what comes next. Featuring leaders from: Ai2, Black Forest Labs, Cursor, LangChain, Mistral AI, OpenEvidence, Perplexity, Reflection AI, Thinking Machines, and University of Washington.
0:14

Open models are at the center of one of the most important shifts in AI. At #NVIDIAGTC, NVIDIA founder and CEO Jensen Huang is bringing together leaders from some of the most exciting AI companies to discuss what comes next. Featuring leaders from: Ai2, Black Forest Labs, Cursor, LangChain, Mistral AI, OpenEvidence, Perplexity, Reflection AI, Thinking Machines, and University of Washington.

NVIDIA

48,435 просмотров • 4 месяцев назад

Use AI to render 3D Workflows with Blender, Magnific, new 3D model from Stability and new powerful 3D tools form NVIDIA Let’s have a look.
0:16

Use AI to render 3D Workflows with Blender, Magnific, new 3D model from Stability and new powerful 3D tools form NVIDIA Let’s have a look.

Linus ✦ Ekenstam

156,107 просмотров • 2 лет назад

Today at NVIDIA #GTC, IonQ announced a strategic alliance and MOU with NVIDIA and KISTI to build a national quantum-HPC infrastructure for South Korea. Together, we're providing the foundational technology and AI models that will accelerate quantum computing globally. Read more here: #QuantumComputing #AI
0:14

Today at NVIDIA #GTC, IonQ announced a strategic alliance and MOU with NVIDIA and KISTI to build a national quantum-HPC infrastructure for South Korea. Together, we're providing the foundational technology and AI models that will accelerate quantum computing globally. Read more here: #QuantumComputing #AI

IonQ

90,976 просмотров • 3 месяцев назад

Announcing 📣 A first-of-its-kind partnership between Cisco and NVIDIA to bring #AI to the enterprise with the NVIDIA Spectrum-X #networking platform. Learn more:
0:13

Announcing 📣 A first-of-its-kind partnership between Cisco and NVIDIA to bring #AI to the enterprise with the NVIDIA Spectrum-X #networking platform. Learn more:

Cisco

30,797 просмотров • 1 год назад

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:
0:14

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:

Lior Alexander

158,553 просмотров • 3 лет назад

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:
0:13

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:

Bilawal Sidhu

44,822 просмотров • 2 лет назад

What if a network could deliver AI results closer to where data is created, instead of sending it to far‑off data centers? In collaboration with NVIDIA and Decart, we’re making that a reality by bringing GPU‑powered computing to the network edge — closer to homes and businesses — so AI applications respond in real-time. This is how we're laying the foundation for the next generation of AI‑driven services:
0:16

What if a network could deliver AI results closer to where data is created, instead of sending it to far‑off data centers? In collaboration with NVIDIA and Decart, we’re making that a reality by bringing GPU‑powered computing to the network edge — closer to homes and businesses — so AI applications respond in real-time. This is how we're laying the foundation for the next generation of AI‑driven services:

Comcast

15,350 просмотров • 3 месяцев назад

A quick test of using 3d drawing with 6DOF controllers as an "instructor" for a generative AI process. There's so many powerful and fun new ways to create just around the corner.. Here I'm using Dreams, Krea and 3daistudio. The 3d model at the end of the video was generated from the Dreams+Krea output in just around 15 seconds. Only the model on the left is a "true" 3d model. #ai #madeindreams
0:17

A quick test of using 3d drawing with 6DOF controllers as an "instructor" for a generative AI process. There's so many powerful and fun new ways to create just around the corner.. Here I'm using Dreams, Krea and 3daistudio. The 3d model at the end of the video was generated from the Dreams+Krea output in just around 15 seconds. Only the model on the left is a "true" 3d model. #ai #madeindreams

Martin Nebelong

132,823 просмотров • 2 лет назад

Don't miss the livestream of NVIDIA CEO Jensen Huang's keynote at #NVIDIAGTC Taipei. Hear Jensen unveil the breakthroughs in physical AI, scaling infrastructure, science, and more — and what's driving the next generation of AI. It all starts here. 📍 Monday, June 1 | 11 a.m. Taipei Time 📍 Sunday, May 31 | 8 p.m. Pacific Time 🎥 Tune in for one of the biggest moments of GTC:
0:15

Don't miss the livestream of NVIDIA CEO Jensen Huang's keynote at #NVIDIAGTC Taipei. Hear Jensen unveil the breakthroughs in physical AI, scaling infrastructure, science, and more — and what's driving the next generation of AI. It all starts here. 📍 Monday, June 1 | 11 a.m. Taipei Time 📍 Sunday, May 31 | 8 p.m. Pacific Time 🎥 Tune in for one of the biggest moments of GTC:

NVIDIA

66,265 просмотров • 1 месяц назад

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.
0:11

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.

Gipp 🦅

590,100 просмотров • 1 месяц назад

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality. Learn More →
0:19

Announcing NVIDIA DLSS 5, an AI-powered breakthrough in visual fidelity for games, coming this fall. DLSS 5 infuses pixels with photorealistic lighting and materials, bridging the gap between rendering and reality. Learn More →

Rock Solid

13,360 просмотров • 3 месяцев назад

We are teaming up with NVIDIA to throw a NIM hackathon NVIDIA AI NIMs are an easy way to run AI models anywhere Free GPUs, beer, and early access to NVIDIA's Llama3 NIM a week before it's released 🤙 Come learn from the brightest and hack on the most cutting edge AI software Speakers: Robert Nishihara -- CEO Anyscale Beyang -- CTO Sourcegraph Michael Balint -- NIM NVIDIA Joseph Nelson -- CEO Roboflow ishan -- AI Lead brev.dev Apply here ->
0:14

We are teaming up with NVIDIA to throw a NIM hackathon NVIDIA AI NIMs are an easy way to run AI models anywhere Free GPUs, beer, and early access to NVIDIA's Llama3 NIM a week before it's released 🤙 Come learn from the brightest and hack on the most cutting edge AI software Speakers: Robert Nishihara -- CEO Anyscale Beyang -- CTO Sourcegraph Michael Balint -- NIM NVIDIA Joseph Nelson -- CEO Roboflow ishan -- AI Lead brev.dev Apply here ->

Nader Khalil🍊

43,410 просмотров • 2 лет назад

What happens when the world’s most powerful computing meets the world's most creative enterprise? 🤝 Adobe CEO Shantanu Narayen and NVIDIA CEO Jensen Huang took the stage at Adobe Summit to showcase how their collaboration is building the future of agentic AI.
0:16

What happens when the world’s most powerful computing meets the world's most creative enterprise? 🤝 Adobe CEO Shantanu Narayen and NVIDIA CEO Jensen Huang took the stage at Adobe Summit to showcase how their collaboration is building the future of agentic AI.

NVIDIA

32,300 просмотров • 2 месяцев назад

From simulation to reality 🤖 Robotics creator Skyentific built a walking bipedal robot using a simulation-first approach to design, test, and iterating in virtual environments before deploying in the real world. Powered by the NVIDIA Isaac platform and NVIDIA Jetson for on-device AI and control. 📖 #NationalRoboticsWeek
0:12

From simulation to reality 🤖 Robotics creator Skyentific built a walking bipedal robot using a simulation-first approach to design, test, and iterating in virtual environments before deploying in the real world. Powered by the NVIDIA Isaac platform and NVIDIA Jetson for on-device AI and control. 📖 #NationalRoboticsWeek

NVIDIA Robotics

21,697 просмотров • 3 месяцев назад