Started experimenting with hand-tracking-based interactions for my little voxel editor – BoxelXR. There are a lot of things to tune, but overall it feels good. Though it would require some decent work in sound design to compensate for the lack of haptic feedback.

This is my editor I mentioned — BoxelXR. Right now it works only with controllers, but I will add hand-tracking support soon!

imagine building in minecraft like this

That is some real-life Iron Man-type interface tech. Sick

Lack of haptic feedback is exactly what has bugged me about AR for the longest time. I’m hopeful that a wristband or set of rings will solve this

One of the most important UX patterns to consider when designing a tool is "recognition over recall" (see Nielsen's UX heuristics). It states that you need to leverage the user's understanding of how it should work based on their past experiences: they will start using your app faster, and it will reduce the amount of errors caused by unfamiraty with the UI. In the case of BoxelXR, I used a 3D face extrusion pattern to make it obvious how to use it. If you have used 3D editors before, you are familiar with this pattern. If you haven't, it will prepare you for polygonal modeling 😊

Oh, 10K! Thanks, everyone! If you like voxel-based content, check out this awesome project by @ephtracy -- MagicaVoxel. It is my source of inspiration for the project ❤️

💡 For everyone who is wondering how it was done: 1️⃣ I used Unity3D + C#; 2️⃣It's running on Meta Quest 3; 3️⃣ I used XR Interaction Toolkit to communicate with the device's API and get all hand-tracking data; 4️⃣ The interactions themselves are pure math essentially. 📐 If you are curious about what math I use to design such things, I wrote an article for you, where I explain this —

That's how innovation starts rough edges first, immersion second. Hand tracking will feel natural once the audio cues hit the right balance.

The direct source of inspiration for the first iteration of BoxelXR with controllers (released 2.5 years ago) and the current iteraction with hand-tracking is MagicaVoxel by @ephtracy. Magica uses a similar 3D face extrusion pattern, but adapts it to the "flooring" nature of discrete voxels. And this drag-to-create pattern you can naturally convert into the pinch-to-create pattern that you saw in my AR demo (which is common for hand-tracking applications).

If you want to try this prototype, you are welcome 😊

Minecraft but in the real world. 😀

this shittt is cool as fuckkkkk!!!!!

Looks promising. Sound design might actually become the new “touch” for VR. Have you tried pairing subtle spatial audio cues with gestures yet?

Hey man, tuning an app like that sounds like a ton of work! btw writes your blog posts and gets them in front of AI for recommendations.

this is so fkn cool

Mixed reality Empyrion build mechanics

Awesome! How fast can you go before it starts to lose track of your hands?

WOAHHH! Build JARVIS GUI! I got the model and logic stuff, but this...... this is next level!

That’s really cool , which VR are you using ?

this would be incredible for videogame level-prototyping being able to build a miniature version then enlarge it and experience it would give you a much better perspective

The future is here.

i know nothing about hardware of this sort but wouldn't haptic feedback just be a matter of like a little weight spinning inside a small bluetooth gadget, possibly a ring or wrist band?

Wow this is impressive. Now I wonder if this tracking model could be enabled inside of Minecraft - my kids would be mind-blown.

this is solid, architects gotta be scared now, their jobs are in serious danger lol

@heidy_dms would be cool to feel those pops with your band

This is very cool. I wanted to use VR painting to create worlds for my kids to run around in, but that's still quite intensive to do 100% with strokes and fitting/positioning shapes together. Having something like this would be a dream and make voxel building so much fun.

It looks cool, man. Do I need a specific system or VR glasses to visualize this, or does it Just randomly run in our mobile's camera?

Tony stark had this 10 years ago

AR Minecraft, the building looks so much quicker and easier! 😍😂

very cool oleg

@grok what all i need to learn to do this

Hand tracking for VR creation tools is the future! No controllers = more intuitive, natural interaction. Voxel editors benefit hugely from spatial input. The lack of haptic feedback is real challenge though - sound design becomes crucial. Excited to see where BoxelXR goes. Keep building!