Loading video...

Video Failed to Load

Go Home

From 2D to 3D segmentation: theory and applications of u-Segment3D by Felix Zhou et al.

RetoPaul's profile picture

Reto Fiolka

7,086 subscribers

17,594 views • 2 years ago •via X (Twitter)

EducationScience & TechnologyNews & Politics
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Segment Any 3D Gaussians paper page: Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA.
0:18

Segment Any 3D Gaussians paper page: Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA.

AK

69,542 views • 2 years ago

SpatialTracker Tracking Any 2D Pixels in 3D Space Recovering dense and long-range pixel motion in videos is a challenging problem. Part of the difficulty arises from the 3D-to-2D projection process, leading to occlusions and discontinuities in the 2D motion domain.
0:11

SpatialTracker Tracking Any 2D Pixels in 3D Space Recovering dense and long-range pixel motion in videos is a challenging problem. Part of the difficulty arises from the 3D-to-2D projection process, leading to occlusions and discontinuities in the 2D motion domain.

AK

77,162 views • 2 years ago

the transition from 2d to 3d cutscene was done really well
0:12

the transition from 2d to 3d cutscene was done really well

Marko

88,566 views • 5 months ago

was working on a small thingy of my own but i really don't think i'd make anything of it, especially anytime soon. it's like an open world game, played in both 2d and 3d, where you can switch from 2d people to 3d people and help each other out
0:17

was working on a small thingy of my own but i really don't think i'd make anything of it, especially anytime soon. it's like an open world game, played in both 2d and 3d, where you can switch from 2d people to 3d people and help each other out

Cassy

54,613 views • 1 year ago

They’ve really mastered the art of mixing 3d and 2d
0:12

They’ve really mastered the art of mixing 3d and 2d

Agnis

501,440 views • 1 month ago

New study from Youfang Zhou, Xianfeng Wang, et al. in the DengLab @tulanemedicine show that induced #polyploidy acts as an intrinsic cellular stress, reprogramming epithelial cells to promote motility and phagocytic behaviors.
0:10

New study from Youfang Zhou, Xianfeng Wang, et al. in the DengLab @tulanemedicine show that induced #polyploidy acts as an intrinsic cellular stress, reprogramming epithelial cells to promote motility and phagocytic behaviors.

Journal of Cell Biology

11,166 views • 3 months ago

My 3d/ 2d module at Gobelins...I decided to animate Sherlock Hound and Judy Hopps Audio is from doctor who #2danimation
0:14

My 3d/ 2d module at Gobelins...I decided to animate Sherlock Hound and Judy Hopps Audio is from doctor who #2danimation

Waackery

272,471 views • 4 months ago

YOU CAN NOW REMASTER ANY 2D GAME INTO A 3D WORLD Pokémon Ruby defined a generation of top-down handheld gaming. 2D is fast and solo-friendly: tiles, sprites, simple animations. 3D has always been brutal modeling, rigging, lighting, LODs, and months of specialized work that kills most small teams. Mint shatters that One screenshot → instant coherent 3D worlds No more waiting for studios to invest a 2D game becomes 3D today.
0:13

YOU CAN NOW REMASTER ANY 2D GAME INTO A 3D WORLD Pokémon Ruby defined a generation of top-down handheld gaming. 2D is fast and solo-friendly: tiles, sprites, simple animations. 3D has always been brutal modeling, rigging, lighting, LODs, and months of specialized work that kills most small teams. Mint shatters that One screenshot → instant coherent 3D worlds No more waiting for studios to invest a 2D game becomes 3D today.

mint

16,515 views • 2 months ago

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️
0:15

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️

AI at Meta

3,570,808 views • 3 years ago

This is 3D animation powered by AI. No suits or markers needed. This 2D video was captured with GoPro cameras and converted to 3D motion data using Move Multi-Cam. #MadeWithMove
0:15

This is 3D animation powered by AI. No suits or markers needed. This 2D video was captured with GoPro cameras and converted to 3D motion data using Move Multi-Cam. #MadeWithMove

Move AI

182,792 views • 2 years ago

🤗Another breakdown from an older project made for Lands of Noara . It was one of my first 2D VFX on a shot and a great project to direct and work on. I also made the motion design. Animated by the fantastic Makadam #blender #animation #3D #p2design
0:11

🤗Another breakdown from an older project made for Lands of Noara . It was one of my first 2D VFX on a shot and a great project to direct and work on. I also made the motion design. Animated by the fantastic Makadam #blender #animation #3D #p2design

Pierrick Picaut

42,631 views • 1 year ago

"Levi vs Willy" by Alp .K The idea of the exercise was to integrate 2D animation (Levi) with 3D animation (Willy) over a dialogue from the film "The Duelists” Ridley Scott’s. #leviackerman #animation #3danimation #2danimation
0:16

"Levi vs Willy" by Alp .K The idea of the exercise was to integrate 2D animation (Levi) with 3D animation (Willy) over a dialogue from the film "The Duelists” Ridley Scott’s. #leviackerman #animation #3danimation #2danimation

GOBELINS Paris

59,773 views • 2 years ago

Many industrial robotics applications still rely on 2D lidar for simple zone monitoring, which limits the field of view and vertical resolution required to detect complex or overhanging obstacles Ouster Firmware 3.2 embeds native 3D zone monitoring directly onto our digital lidar sensors, including the OS0, OS1, and OSDome
0:11

Many industrial robotics applications still rely on 2D lidar for simple zone monitoring, which limits the field of view and vertical resolution required to detect complex or overhanging obstacles Ouster Firmware 3.2 embeds native 3D zone monitoring directly onto our digital lidar sensors, including the OS0, OS1, and OSDome

Ouster

25,792 views • 3 months ago

one 2d photo --> 3d gaussian splat quick test with Echo-2 by SpAItial -- these 3d scene generation models are getting better! already at a sufficient quality to serve as a virtual set / backdrop in your 3d tool of choice
0:17

one 2d photo --> 3d gaussian splat quick test with Echo-2 by SpAItial -- these 3d scene generation models are getting better! already at a sufficient quality to serve as a virtual set / backdrop in your 3d tool of choice

Bilawal Sidhu

23,722 views • 3 months ago

More PixiJS 3D testing 👀 Take any live 2D PixiJS content (sprites, text, graphics) and render it onto 3D geometry Our goal is to make working between 2D and 3D as seamless as possible in our WebGPU/WebGL renderer Let us know what features you'd like to see!
0:17

More PixiJS 3D testing 👀 Take any live 2D PixiJS content (sprites, text, graphics) and render it onto 3D geometry Our goal is to make working between 2D and 3D as seamless as possible in our WebGPU/WebGL renderer Let us know what features you'd like to see!

PixiJS

30,996 views • 1 month ago

It’s never been easier for 3D animators to bring their creative vision to life. This is motion capture powered by AI. No suits or markers needed. You can now capture multiple people anywhere using standard cameras, and Move AI will generate 3D motion data from the 2D video.
0:11

It’s never been easier for 3D animators to bring their creative vision to life. This is motion capture powered by AI. No suits or markers needed. You can now capture multiple people anywhere using standard cameras, and Move AI will generate 3D motion data from the 2D video.

Move AI

116,379 views • 2 years ago

7. Introduction to Development Learn practical lessons in developing 2D and 3D interactive games. URL:
0:10

7. Introduction to Development Learn practical lessons in developing 2D and 3D interactive games. URL:

HR Swati sharma

92,260 views • 7 months ago