Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

here's a nicer live-mapping method on Quest. Sampling depth texture and writing greatest y position per x,z world heightmap pixel. It's rough but semi-functional Bad performance is from the debug mesh rather than the world mapper itself. Runs fine with the mesh off.

10,374 Aufrufe • vor 1 Jahr •via X (Twitter)

9 Kommentare

Profilbild von Thomas Van Bouwel
Thomas Van Bouwelvor 1 Jahr

Inaccuracy in the mesh is a huge problem right now, especially if you want to do things like player pathfinding safely without accidentally sending players down a staircase or through a wall - something like this would be a great solution for that. Great work!

Profilbild von sneeth
sneethvor 1 Jahr

love all this stuff you’re doing with the quest, looks like fun to tinker around with it

Profilbild von jt
jtvor 1 Jahr

This particular project just becomes a hyperfixation every once in a while and it isn't fun at all. It's awful. It obliterates my sleep schedule every time I get into it.

Profilbild von mikolaj power
mikolaj powervor 1 Jahr

looks awesome. how do you access the depth data? and what framework are you using if any?

Profilbild von jt
jtvor 1 Jahr

@andrzejthefirst Meta provides depth frame access in their Quest SDKs

Profilbild von Maebbie
Maebbievor 1 Jahr

wow this is so fast, looks to be in a very usable state now

Profilbild von jt
jtvor 1 Jahr

There is still a lot to do. Notably some sort of noise filtering i.e. only write to the heightmap if a height reading is consistent for some number of frames. This would probably solve issues with doorways filling in.

Profilbild von Gikis
Gikisvor 1 Jahr

So now you're able to scan a large area? I tried disabling guardian in dev options a half year ago, but it would still only allow scanning a pretty small space.

Profilbild von jt
jtvor 1 Jahr

This is my own Unity project, not the system space scanner

Ähnliche Videos

[NeurIPS '24] DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representation Abstract (excerpt) We introduce DreamMesh4D, a novel framework that combines mesh representation with sparse-controlled deformation technique to generate high-quality 4D object from a monocular video. To overcome the limitation of classical texture representation, we bind Gaussian splats to the surface of the triangular mesh for differentiable optimization of both the texture and mesh vertices. In particular, DreamMesh4D begins with a coarse mesh provided by a single image based 3D generation method. Sparse points are then uniformly sampled across the surface of the mesh, and are used to build a deformation graph to drive the motion of the 3D object for the sake of computational efficiency and providing additional constraint. For each step, transformations of sparse control points are predicted using a deformation network, and the mesh vertices as well as the bound surface Gaussians are deformed via a geometric skinning algorithm. The skinning algorithm is a hybrid approach combining LBS (linear blending skinning) and DQS (dual-quaternion skinning), mitigating drawbacks associated with both approaches. The static surface Gaussians and mesh vertices as well as the dynamic deformation network are learned via reference view photometric loss, score distillation loss as well as other regularization losses in a two-stage manner. Extensive experiments demonstrate that our method outperforms prior video-to-4D generation methods in terms of rendering quality and spatial-temporal consistency.

MrNeRF

12,323 Aufrufe • vor 1 Jahr