Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Do 3D reconstruction transformers really need a billion parameters, or are most of those layers just doing the same thing over and over? Introducing Déjà View: a single transformer block, looped K times, that matches or beats models 8–10× its size with lower compute. 🧵

TobiasFischer11's profile picture

Tobias Fischer

1,027 subscribers

91,527 Aufrufe • vor 22 Tagen •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

What are we doing people, is it just me, or does the foolery need to stop. What’s the difference than going on a field with an actual base and work on the same thing? Are we creating robots or athletes?
0:18

What are we doing people, is it just me, or does the foolery need to stop. What’s the difference than going on a field with an actual base and work on the same thing? Are we creating robots or athletes?

Austin Ford Sr.

448,948 Aufrufe • vor 10 Monaten

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)
0:17

How can we use test-time compute for spatial understanding? 🤔 In InterPose, we propose to repeatedly sample generative video models to help two-view pose estimation and reconstruction, by leveraging the video models' keyframe interpolation abilities. A 🧵... (1/8)

Ricardo Martin-Brualla

21,685 Aufrufe • vor 1 Jahr

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 Aufrufe • vor 1 Jahr

Create a 3D model from a single image, set of images or a text prompt in < 1 minute 😮‍💨 This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or your favorite 2D image generator like Midjourney (just an image). How does this magic work? “This architecture is similar to video diffusion models, but with camera pose embeddings for each image instead of time embeddings. The generated views are passed into a robust 3D reconstruction pipeline to create the 3D representation (Zip-NeRF or 3DGS)”
0:18

Create a 3D model from a single image, set of images or a text prompt in < 1 minute 😮‍💨 This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or your favorite 2D image generator like Midjourney (just an image). How does this magic work? “This architecture is similar to video diffusion models, but with camera pose embeddings for each image instead of time embeddings. The generated views are passed into a robust 3D reconstruction pipeline to create the 3D representation (Zip-NeRF or 3DGS)”

Bilawal Sidhu

92,760 Aufrufe • vor 2 Jahren

🚀 Introducing GenLit – Reformulating Single-Image Relighting as Video Generation! We leverage video diffusion models to perform realistic near-field relighting from just a single image—No explicit 3D reconstruction or ray tracing required! No intermediate graphics buffers, directly in the pixel space! 📄 Dive into the paper: 🎥 Project page & demos: 🛠 Code coming soon! #GenerativeAI #ComputerVision #Relighting #DiffusionModels #Graphics 🧵 1/5
0:19

🚀 Introducing GenLit – Reformulating Single-Image Relighting as Video Generation! We leverage video diffusion models to perform realistic near-field relighting from just a single image—No explicit 3D reconstruction or ray tracing required! No intermediate graphics buffers, directly in the pixel space! 📄 Dive into the paper: 🎥 Project page & demos: 🛠 Code coming soon! #GenerativeAI #ComputerVision #Relighting #DiffusionModels #Graphics 🧵 1/5

Haven Feng @ CVPR

22,427 Aufrufe • vor 1 Jahr

Platysternon megacephalum (or big headed turtle) is a very odd-shaped turtle with a huge head and a long tail that are almost the same size as its body. [📹 47ruacanhphongthuybmt]
0:17

Platysternon megacephalum (or big headed turtle) is a very odd-shaped turtle with a huge head and a long tail that are almost the same size as its body. [📹 47ruacanhphongthuybmt]

Massimo

16,142,025 Aufrufe • vor 2 Jahren

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.
0:13

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,042 Aufrufe • vor 1 Jahr

The US "diet industry" makes over $70 billion a year... Yet Americans are fatter than ever. Why? They sell you useless fad diets and supplements that don't work. Here are the only 8 diet rules you need for fat loss (bookmark this): 🧵 1. Eat 150g+ of protein a day
0:11

The US "diet industry" makes over $70 billion a year... Yet Americans are fatter than ever. Why? They sell you useless fad diets and supplements that don't work. Here are the only 8 diet rules you need for fat loss (bookmark this): 🧵 1. Eat 150g+ of protein a day

Chace Chambers

156,764 Aufrufe • vor 6 Monaten

all the parameters (weights and biases) of the same multilayer perception over 200 epochs red/blue/yellow-three layers The top right knob in each layer being the weight
0:10

all the parameters (weights and biases) of the same multilayer perception over 200 epochs red/blue/yellow-three layers The top right knob in each layer being the weight

Kat ⊷ the Poet Engineer

15,851 Aufrufe • vor 1 Jahr

Conducting road safety audits, scheduling maintenance, or planning logistics routes used to require combing through imagery or, worse, driving miles of roads yourself. Not anymore. New AI-powered data layers in Google Earth are changing that. Pulling from billions of Google Street View images, these layers now spot infrastructure assets like stop signs, speed limit signs, and more to map infrastructure for you. By combining these assets into a single project and diving into Street View you can locate and validate assets on Google Earth in seconds. These layers are now available for Professional & Professional Advanced customers on web and Android, with coverage expanding over the coming weeks.
0:10

Conducting road safety audits, scheduling maintenance, or planning logistics routes used to require combing through imagery or, worse, driving miles of roads yourself. Not anymore. New AI-powered data layers in Google Earth are changing that. Pulling from billions of Google Street View images, these layers now spot infrastructure assets like stop signs, speed limit signs, and more to map infrastructure for you. By combining these assets into a single project and diving into Street View you can locate and validate assets on Google Earth in seconds. These layers are now available for Professional & Professional Advanced customers on web and Android, with coverage expanding over the coming weeks.

Google Earth

18,208 Aufrufe • vor 11 Tagen

The US "diet industry" makes over $70 billion a year... Yet Americans are fatter than ever. Why? They sell you useless fad diets and supplements that don´t work. Here are the only 8 diet rules you need for fat loss (bookmark this): 🧵 1. Eat 150g+ of protein a day
0:11

The US "diet industry" makes over $70 billion a year... Yet Americans are fatter than ever. Why? They sell you useless fad diets and supplements that don´t work. Here are the only 8 diet rules you need for fat loss (bookmark this): 🧵 1. Eat 150g+ of protein a day

Chace Chambers

642,492 Aufrufe • vor 9 Monaten

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers &amp; hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀
0:17

The rise and fall of popular transformer architectures 📈 One pro of having a lot of transformer architectures implemented in transformers &amp; hosted on the Hugging Face Hub is that one can see the evolution of popularity of different models 👀

Lysandre

31,730 Aufrufe • vor 2 Jahren

STEYER: "If you keep doing the same thing over and over and expecting a different outcome, that's insanity!" Steve Hilton: "Then don't vote Democrat!
0:12

STEYER: "If you keep doing the same thing over and over and expecting a different outcome, that's insanity!" Steve Hilton: "Then don't vote Democrat!

Daily Caller

101,457 Aufrufe • vor 1 Monat

🚨🚨 The natural beauty of Pashtuns, mashaAllah. No makeup, no surgery — just a brother and his sisters from a modest family. They do not need jewelry or cosmetic procedures; those whom God has blessed with beauty are never in need of anything artificial.
0:15

🚨🚨 The natural beauty of Pashtuns, mashaAllah. No makeup, no surgery — just a brother and his sisters from a modest family. They do not need jewelry or cosmetic procedures; those whom God has blessed with beauty are never in need of anything artificial.

GPX

2,389,518 Aufrufe • vor 6 Monaten

View of destructive storms hitting downtown Indianapolis from my third floor. Not sure if those blue flashes are transformers blowing or not #inwx
0:10

View of destructive storms hitting downtown Indianapolis from my third floor. Not sure if those blue flashes are transformers blowing or not #inwx

Cody Moore

105,737 Aufrufe • vor 1 Jahr

SHES SO SICK OF YALL ASKING HER TO DO THE SAME THING OVER AND OVER AGAIN LMFAOOO
0:14

SHES SO SICK OF YALL ASKING HER TO DO THE SAME THING OVER AND OVER AGAIN LMFAOOO

🐿️💨

279,985 Aufrufe • vor 2 Monaten

Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction Note: Check below for full video. Abstract (cited): "In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion, and 3D Gaussian representations, enabling accurate and efficient scene reconstruction. Our technique is particularly effective for high-quality scene reconstruction from large field-of-view (FOV) imagery taken with wide-angle lenses, allowing the scene to be modeled from a smaller number of images. We introduce a novel method for modeling complex lens distortions using a hybrid network that combines invertible residual networks with explicit grids. This design effectively regularizes the optimization process, achieving greater accuracy than conventional camera models. Additionally, we propose a cubemap-based resampling strategy to support large FOV images without sacrificing resolution or introducing distortion artifacts. Our method is compatible with the fast rasterization of Gaussian Splatting, adaptable to a wide variety of camera lens distortions, and demonstrates state-of-the-art performance on both synthetic and real-world datasets."
0:16

Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction Note: Check below for full video. Abstract (cited): "In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion, and 3D Gaussian representations, enabling accurate and efficient scene reconstruction. Our technique is particularly effective for high-quality scene reconstruction from large field-of-view (FOV) imagery taken with wide-angle lenses, allowing the scene to be modeled from a smaller number of images. We introduce a novel method for modeling complex lens distortions using a hybrid network that combines invertible residual networks with explicit grids. This design effectively regularizes the optimization process, achieving greater accuracy than conventional camera models. Additionally, we propose a cubemap-based resampling strategy to support large FOV images without sacrificing resolution or introducing distortion artifacts. Our method is compatible with the fast rasterization of Gaussian Splatting, adaptable to a wide variety of camera lens distortions, and demonstrates state-of-the-art performance on both synthetic and real-world datasets."

MrNeRF

17,206 Aufrufe • vor 1 Jahr

Playing around with a "tunnel gun" that can open a tunnel in front of you that can be looked into or moved through. This type of thing is really simple in my engine - just spawn an SDF capsule subtraction that is locked to the player's view.
0:17

Playing around with a "tunnel gun" that can open a tunnel in front of you that can be looked into or moved through. This type of thing is really simple in my engine - just spawn an SDF capsule subtraction that is locked to the player's view.

Mike Turitzin

281,995 Aufrufe • vor 8 Monaten

CNN Legal Analyst Elie Honig: "Pam Bondi is, without a question, qualified to be Attorney General. She's been a prosecutor for 20 years in Florida. For 8 of those, she was the Attorney General of the state. That's a very big, very complicated job‚ and that level of experience is on par with, or better than most United States Attorneys General that we've seen over the past 50 years or so."
0:19

CNN Legal Analyst Elie Honig: "Pam Bondi is, without a question, qualified to be Attorney General. She's been a prosecutor for 20 years in Florida. For 8 of those, she was the Attorney General of the state. That's a very big, very complicated job‚ and that level of experience is on par with, or better than most United States Attorneys General that we've seen over the past 50 years or so."

Trump War Room

456,072 Aufrufe • vor 1 Jahr

New update drop 🎶 Blaster's beats are taking over! Download the latest 2.2 update and join the party! #Transformers #Gaming #Blaster
0:17

New update drop 🎶 Blaster's beats are taking over! Download the latest 2.2 update and join the party! #Transformers #Gaming #Blaster

TRANSFORMERS: Tactical Arena

40,533 Aufrufe • vor 1 Jahr