Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Beyond 3D representations, we must rethink appearance too—textures specifically. UV-maps, the de-facto standard, have inherent flaws: seams, distortions, memory waste, and non-uniform resolution. Can we do better? With Simone Foti, we propose UV3-TeD: 👇

Tolga Birdal

3,457 subscribers

22,794 views • 1 year ago •via X (Twitter)

Education Arts Science & Technology

Anya Rossi• Live Now

Private livecam show

6 Comments

Tolga Birdal1 year ago

Yet, restricting the texture representation onto the object surface is fundamentally different from generating colored point clouds. We introduce attention-enhanced geodesic heat diffusion along with denoising diffusions (DDPM) to accurately represent and generate textures.👇

Tolga Birdal1 year ago

To handle large collections of potentially non-orientable, non-manifold, disconnected and topologically intricate surfaces, we propose a mixed point-cloud & mesh Laplacian that combines geometric and topological cues to diffuse heat in the presence of complex structures. 👇

Tolga Birdal1 year ago

Eventually, we present a complete framework for texture representation and (geometric and positionally conditioned) generation that integrates point / color sampling, spectral shape description and ray-based rendering, which does not resort to UV-mapping. 👇

Tolga Birdal1 year ago

Even advanced point cloud texturing models like Point-UV Diffusion still rely on UV-maps, somehow. Our UV3-TeD captures features across distant regions, improving upon these models to produce more diverse and semantically meaningful textures on meshes with arbitrary topologies.👇

Tolga Birdal1 year ago

I feel proud to share our first and very pleasurable collaboration, led by @simo_foti. Code is now available: The project page is under: #CVPR #CVML #AI #Graphics

HUDI2 years ago

Guess who’s not a fan of data ownership? 😅 When #Zuckerberg gives zero stars, you know the @datamaskwallet app is keeping your data safe from the big tech giants' grasp. 🚫📊 #DataPrivacy #BigTechNemesis art and creative by @Matteo_Frog

Related Videos

Too many people have accepted Donald Trump’s presidency as "normal." Too many believe there is nothing we can do, and that we simply need to outlast his term. Let me be clear: We cannot endure another 32 months with this mendacious, narcissistic, rapist, want to be dictator who is selling us out to the highest bidder. Not only can we stand up and fight back—we must. Join our campaign so we can take this fight to Congress to impeach, convict, and remove Trump and his cabinet.

Too many people have accepted Donald Trump’s presidency as "normal." Too many believe there is nothing we can do, and that we simply need to outlast his term. Let me be clear: We cannot endure another 32 months with this mendacious, narcissistic, rapist, want to be dictator who is selling us out to the highest bidder. Not only can we stand up and fight back—we must. Join our campaign so we can take this fight to Congress to impeach, convict, and remove Trump and his cabinet.

George Conway ⚖️🇺🇸

328,375 views • 3 months ago

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,514 views • 2 years ago

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.

AK

140,992 views • 2 years ago

As a nation we cannot accept gun violence as a normal part of American life. Far too many families and communities have been left grieving because of repeated but preventable tragedies. I refuse to accept the status quo. Congress must act – ban assault weapons, strengthen background checks and red flag laws, and stop gun trafficking across state lines. We can, and must, do better for our kids and our communities.

As a nation we cannot accept gun violence as a normal part of American life. Far too many families and communities have been left grieving because of repeated but preventable tragedies. I refuse to accept the status quo. Congress must act – ban assault weapons, strengthen background checks and red flag laws, and stop gun trafficking across state lines. We can, and must, do better for our kids and our communities.

Rep. Tim Kennedy

17,160 views • 8 months ago

SpecNeRF: Gaussian Directional Encoding for Specular Reflections paper page: Neural radiance fields have achieved remarkable performance in modeling the appearance of 3D scenes. However, existing approaches still struggle with the view-dependent appearance of glossy surfaces, especially under complex lighting of indoor environments. Unlike existing methods, which typically assume distant lighting like an environment map, we propose a learnable Gaussian directional encoding to better model the view-dependent effects under near-field lighting conditions. Importantly, our new directional encoding captures the spatially-varying nature of near-field lighting and emulates the behavior of prefiltered environment maps. As a result, it enables the efficient evaluation of preconvolved specular color at any 3D location with varying roughness coefficients. We further introduce a data-driven geometry prior that helps alleviate the shape radiance ambiguity in reflection modeling. We show that our Gaussian directional encoding and geometry prior significantly improve the modeling of challenging specular reflections in neural radiance fields, which helps decompose appearance into more physically meaningful components.

SpecNeRF: Gaussian Directional Encoding for Specular Reflections paper page: Neural radiance fields have achieved remarkable performance in modeling the appearance of 3D scenes. However, existing approaches still struggle with the view-dependent appearance of glossy surfaces, especially under complex lighting of indoor environments. Unlike existing methods, which typically assume distant lighting like an environment map, we propose a learnable Gaussian directional encoding to better model the view-dependent effects under near-field lighting conditions. Importantly, our new directional encoding captures the spatially-varying nature of near-field lighting and emulates the behavior of prefiltered environment maps. As a result, it enables the efficient evaluation of preconvolved specular color at any 3D location with varying roughness coefficients. We further introduce a data-driven geometry prior that helps alleviate the shape radiance ambiguity in reflection modeling. We show that our Gaussian directional encoding and geometry prior significantly improve the modeling of challenging specular reflections in neural radiance fields, which helps decompose appearance into more physically meaningful components.

AK

33,933 views • 2 years ago

World Labs CEO Dr. Fei-Fei Li says AI must change how we teach and evaluate students: "AI must change learning. AI must change K-16 learning." "The most precious resource of our entire world is human capital." "When we have gotten the technology that can answer standardized tests... when AI can do better than an average human, it's not about humans are bad. It's about we need to change the education system." "We need to change how we evaluate. We need to change the way we empower teachers to educate the next generation of students where they can use these tools, be empowered, and do things that we can never imagine." "All of the kids today should not be scared of AI. They should feel the human agency to lead AI, to use AI in the right way, and to use AI to make the impact that they want to make for the world." Fei-Fei Li at Bloomberg Tech live with Emily Chang

World Labs CEO Dr. Fei-Fei Li says AI must change how we teach and evaluate students: "AI must change learning. AI must change K-16 learning." "The most precious resource of our entire world is human capital." "When we have gotten the technology that can answer standardized tests... when AI can do better than an average human, it's not about humans are bad. It's about we need to change the education system." "We need to change how we evaluate. We need to change the way we empower teachers to educate the next generation of students where they can use these tools, be empowered, and do things that we can never imagine." "All of the kids today should not be scared of AI. They should feel the human agency to lead AI, to use AI in the right way, and to use AI to make the impact that they want to make for the world." Fei-Fei Li at Bloomberg Tech live with Emily Chang

a16z

81,773 views • 2 months ago

Louisiana's governor suspending an active election goes far beyond redrawing maps. “If you think this is going to end with redistricting...you are not paying attention to what Donald Trump and the Republicans are getting us ready for come this fall,” says Marc E. Elias. Here's what we can do 👇

Louisiana's governor suspending an active election goes far beyond redrawing maps. “If you think this is going to end with redistricting...you are not paying attention to what Donald Trump and the Republicans are getting us ready for come this fall,” says Marc E. Elias. Here's what we can do 👇

Democracy Docket

14,826 views • 3 months ago

to my beloved jimin... everyday i am thankful that i am able to live in the same period as yu jimin... u shaped so many of our lives and i can only hope that we have made urs better too stay happy and healthy! BEYOND UP WITH KARINA #Chapter25WithKarina

to my beloved jimin... everyday i am thankful that i am able to live in the same period as yu jimin... u shaped so many of our lives and i can only hope that we have made urs better too stay happy and healthy! BEYOND UP WITH KARINA #Chapter25WithKarina

head rinabin fan

23,514 views • 1 year ago

“I’ll be very surprised if we can get him…” 🤔 Mark Goldbridge says even without CL football, Chelsea and Liverpool have a better chance of signing Alexis Mac Allister in the summer, due to their better record with doing deals. Do you think #mufc can sign him? 👇

“I’ll be very surprised if we can get him…” 🤔 Mark Goldbridge says even without CL football, Chelsea and Liverpool have a better chance of signing Alexis Mac Allister in the summer, due to their better record with doing deals. Do you think #mufc can sign him? 👇

The United Stand

53,590 views • 3 years ago

🚨MUST SEE: This is the scene where a soldier has a total meltdown in a parking lot with his wife. He strips his uniform off, throwing it to the ground and pushes her. Rumor is that he has been relieved of duty following this video. We have to do better to get our brave men and women who serve the help they need, both active and retired. We owe them that much.

🚨MUST SEE: This is the scene where a soldier has a total meltdown in a parking lot with his wife. He strips his uniform off, throwing it to the ground and pushes her. Rumor is that he has been relieved of duty following this video. We have to do better to get our brave men and women who serve the help they need, both active and retired. We owe them that much.

AmericanPapaBear™

66,751 views • 4 months ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 views • 3 years ago

Armenian Parliament Speaker Alen Simonyan: "Armenians should build relations with Azerbaijanis, Armenian society should move away from a ‘planet where only Armenians live." "When something about your neighbor is bothering you - be sure that the same issue is bothering them too. When you say that so many of my people have been killed, they also have lost many. There are mothers, too; they love their children the same way, whether the mother is Azerbaijani, Georgian, Armenian, or Russian. We need to move away from this “planet of Armenians.” There is a planet where only Armenians live. And I believe we are a wise people, and we can understand that. And the French and the English, they fought for 100 years. They called each other “frog-eaters” and “pigs.” They hated each other. They are now allies, they build the economy together, and they jointly set the rules. And what are we supposed to do? What is the choice? And doesn’t a person living in Azerbaijan see the Armenian flag? Doesn’t it bother him? Didn’t we kill anyone there? Was the war all one-sided? And why can they, but we can’t? #Armenians and #Azerbaijanis, We must live together, we must trade, and we must talk. We have a lot in common in everyday life and in traditions." Alen Simonyan said in an interview with Radio Van

Armenian Parliament Speaker Alen Simonyan: "Armenians should build relations with Azerbaijanis, Armenian society should move away from a ‘planet where only Armenians live." "When something about your neighbor is bothering you - be sure that the same issue is bothering them too. When you say that so many of my people have been killed, they also have lost many. There are mothers, too; they love their children the same way, whether the mother is Azerbaijani, Georgian, Armenian, or Russian. We need to move away from this “planet of Armenians.” There is a planet where only Armenians live. And I believe we are a wise people, and we can understand that. And the French and the English, they fought for 100 years. They called each other “frog-eaters” and “pigs.” They hated each other. They are now allies, they build the economy together, and they jointly set the rules. And what are we supposed to do? What is the choice? And doesn’t a person living in Azerbaijan see the Armenian flag? Doesn’t it bother him? Didn’t we kill anyone there? Was the war all one-sided? And why can they, but we can’t? #Armenians and #Azerbaijanis, We must live together, we must trade, and we must talk. We have a lot in common in everyday life and in traditions." Alen Simonyan said in an interview with Radio Van

The Azeri Times

34,006 views • 2 months ago

Armenian Parliament Speaker Alen Simonyan: "Armenians should build relations with Azerbaijanis, Armenian society should move away from a ‘planet where only Armenians live." "When something about your neighbor is bothering you - be sure that the same issue is bothering them too. When you say that so many of my people have been killed, they also have lost many. There are mothers, too; they love their children the same way, whether the mother is Azerbaijani, Georgian, Armenian, or Russian. We need to move away from this “planet of Armenians.” There is a planet where only Armenians live. And I believe we are a wise people, and we can understand that. And the French and the English, they fought for 100 years. They called each other “frog-eaters” and “pigs.” They hated each other. They are now allies, they build the economy together, and they jointly set the rules. And what are we supposed to do? What is the choice? And doesn’t a person living in Azerbaijan see the Armenian flag? Doesn’t it bother him? Didn’t we kill anyone there? Was the war all one-sided? And why can they, but we can’t? #Armenians and #Azerbaijanis, We must live together, we must trade, and we must talk. We have a lot in common in everyday life and in traditions." Alen Simonyan said in an interview with Radio Van

Armenian Parliament Speaker Alen Simonyan: "Armenians should build relations with Azerbaijanis, Armenian society should move away from a ‘planet where only Armenians live." "When something about your neighbor is bothering you - be sure that the same issue is bothering them too. When you say that so many of my people have been killed, they also have lost many. There are mothers, too; they love their children the same way, whether the mother is Azerbaijani, Georgian, Armenian, or Russian. We need to move away from this “planet of Armenians.” There is a planet where only Armenians live. And I believe we are a wise people, and we can understand that. And the French and the English, they fought for 100 years. They called each other “frog-eaters” and “pigs.” They hated each other. They are now allies, they build the economy together, and they jointly set the rules. And what are we supposed to do? What is the choice? And doesn’t a person living in Azerbaijan see the Armenian flag? Doesn’t it bother him? Didn’t we kill anyone there? Was the war all one-sided? And why can they, but we can’t? #Armenians and #Azerbaijanis, We must live together, we must trade, and we must talk. We have a lot in common in everyday life and in traditions." Alen Simonyan said in an interview with Radio Van

JAMnews

29,713 views • 2 months ago

Islam is not compatible with the West because they do not believe stoning infidels (the non-believers) to death is extreme. They do not believe separating women from men in a mosque is extreme. In order to practice Islam, you must be what the West deems "extreme." But to them, it is not. They tell us. We should believe them. Islam is a culture designed to oppress and conquer. The sooner we wake up to that, the better chance we have at survival.

Islam is not compatible with the West because they do not believe stoning infidels (the non-believers) to death is extreme. They do not believe separating women from men in a mosque is extreme. In order to practice Islam, you must be what the West deems "extreme." But to them, it is not. They tell us. We should believe them. Islam is a culture designed to oppress and conquer. The sooner we wake up to that, the better chance we have at survival.

Avery Daye

12,017 views • 9 months ago

Ann embodied something fundamental about this House: the passion and freedom to make our arguments robustly and without fear, and the unique responsibility that we have to the people we represent. The relationship between a Member of Parliament and their constituents is unlike almost anything else. We meet people face to face. We hold surgeries. We go into their homes. If someone is too ill to come and see us, we go and see them. We do so because personal contact with the people we represent is at the heart of our democracy. We must not allow the epitaph of Ann’s terrible murder to be that we withdraw from that responsibility or lose the very thing that makes this place and our democracy so special. But nor can we ignore the increasingly visceral and violent language that is allowed to spread online. The Government, and particularly the Home Secretary, must make it absolutely clear to the social media companies that they have a responsibility to do far more to tackle personal abuse, threats and violent rhetoric. Some of what has been said online following Ann’s murder has been utterly shocking, and we cannot simply pretend that this culture of hatred has no consequences. Ann is not the only Member of Parliament to have been murdered. Jo Cox was murdered. My good friend David Amess was murdered. And the plaques around this Chamber remind us that terrorists have murdered many Members of this House. We must be bold and we must be clear. We will not accept being shut down. We will not accept being prevented from carrying out our duties. And we will not accept being silenced. Because that, I believe, is what Ann would have wanted.

Ann embodied something fundamental about this House: the passion and freedom to make our arguments robustly and without fear, and the unique responsibility that we have to the people we represent. The relationship between a Member of Parliament and their constituents is unlike almost anything else. We meet people face to face. We hold surgeries. We go into their homes. If someone is too ill to come and see us, we go and see them. We do so because personal contact with the people we represent is at the heart of our democracy. We must not allow the epitaph of Ann’s terrible murder to be that we withdraw from that responsibility or lose the very thing that makes this place and our democracy so special. But nor can we ignore the increasingly visceral and violent language that is allowed to spread online. The Government, and particularly the Home Secretary, must make it absolutely clear to the social media companies that they have a responsibility to do far more to tackle personal abuse, threats and violent rhetoric. Some of what has been said online following Ann’s murder has been utterly shocking, and we cannot simply pretend that this culture of hatred has no consequences. Ann is not the only Member of Parliament to have been murdered. Jo Cox was murdered. My good friend David Amess was murdered. And the plaques around this Chamber remind us that terrorists have murdered many Members of this House. We must be bold and we must be clear. We will not accept being shut down. We will not accept being prevented from carrying out our duties. And we will not accept being silenced. Because that, I believe, is what Ann would have wanted.

Iain Duncan Smith MP Chingford & Woodford Green

20,896 views • 1 month ago

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 views • 3 years ago