💡 Idea: Visual -> audio channel. Project world distances to nearest surface on player sphere to create a 360 collision grid (interpolate -> smooth field). This data can be used to generate a fully 3D, realtime soundscape. You can "hear" collisions like a sonar/pressure field.

very cool ! we recently put together some prototypes with the unity raytracing api to exploit raytracing hardware for particle and rigid body physics including collision detection, attraction/repulsion forces, etc it seems aligned with your goals here. feel free to reach out :)

if you're an aspiring (or established) filmmaker looking to package your next project effectively...Let's connect.

This is a cool concept! I’m curious how different this sounds to having it bounce instead, would definitely be interesting to see / hear a comparison

But you would need to handle bounces to have proper sound propagations, right ?

The goal isn't sound environmental propagation, but to generate a distance field texture as audio. Related to HRTF. (no audio here, just a quick demo)

What about Ray tracing sound? Adding bouncing is a similar concept right?

This could be extended for sound propagation, but that's actually more simple and easier to simulate. There are also plugins and such that do these things well, but I don't know of anything like what I've described in my original post.

have you tried the "realtime audio" solutions from others? like from microsoft, or other places.

Do you mean plugins like Steam Audio, Google Resonance, etc? Yes.

classic sensory substitution, we need more of this!!

I wonder if this can be baked and used later, kinda like a 3D sound field