WebGPU + WebXR Three.js physics stress tests w/ eye-tracking input on Apple Vision Pro M5 🏟️ 12k bricks, 100 ragdolls, rope & cloth Based on @[email protected]'s work on AVBD sim, rec'd by Cody Bennett. The scale is amazing + little-to-no clipping! Fun to play w/ on next gen h/w

AVBD paper: Jure's WebGPU AVBD implementation & blog post: My Vision Pro physics sandbox (also w/ MLS and Weta fire sims):

@threejs @JureTriglav @CodyBennett Looks incredible! Thanks for sharing this. What are some of the biggest challenges with this work?

@threejs @JureTriglav @CodyBennett WebKit’s WebGPU is a bit unique & required some workarounds. Have to be on visionOS developer beta w/ the very latest three.js commits and avoid some other pitfalls Also very few people have Vision Pro && care about WebXR + WebGPU so little community support at this time

I’m running the Visions OS dev beta and definitely understand the issue with not having wide adoption on VisionPro just yet, but hope for the future that that might change. Am I correct in understanding that you cannot use controllers at all, like the PSVR two controllers for WebXR?

@threejs @JureTriglav @CodyBennett We're in 1984 - the Vision Pro is the Macintosh - which kind-of sucked at first. It's going to take decades of progress at all levels to get where we're going But I don't know about the controller thing, sorry!

My family had a Mac Plus in '84. I remember those days well. (Fokker Triplane, Dark Castle, Mac Paint). I agree, it's going to take time, however I hope with some of the tech that's already in place we might see more mainstream adoption in a solid decade rather than multiple decades.

@threejs @JureTriglav @CodyBennett Would love to have a play with this - is there a way to access this on the AVP?

@threejs @JureTriglav @CodyBennett Last link below - fwiw I am on M5 Vision Pro on the latest visionOS developer beta - very prototypal!

@threejs @JureTriglav @CodyBennett Now activate the passthrough so you have AR

@threejs @JureTriglav @CodyBennett @Apple doesn't let users access `immersive-ar` passthrough via WebXR at this time sadly! Hoping they get around to it eventually - I see this feedback from devs and users all the time ⏳

@threejs @JureTriglav @CodyBennett @Apple I know. They want to force you through the app store for that.

@threejs @JureTriglav @CodyBennett 12k rigid bodies with eye-tracking input on-device is a real stress test, the physics solver and the render both fighting for the same frame budget on mobile-class silicon. webgpu compute doing the broadphase is probably what makes it hold. what fps did it sustain.

@threejs @JureTriglav @CodyBennett 12k bricks with barely any clipping, solid.

@threejs @JureTriglav @CodyBennett How’d you get eye track input