Matrix effect with LIDAR, Unity, and ARKit

"Hey man, how many tabs did you take?" "Yes"

Literally what I saw the first time I tried DMT only more ancient hieroglyphics and runes ahah

70s would be like "Take your DMT already and stop tinkering with your computer Steven!"

It would be cool to be able to see WiFi, radio, 5G, and Bluetooth signals in the air!

Fixed

What can you see if not light?

Spatial compute is the future, this needs to move beyound demos :)

I just thought about something like this the other day, that's cool

@StockShaman

I got scared because I thought that frame was a mirror lol

I want it.

Make the input data come from your favourite blockchain by running a node and listening to the txs to build an array of strings to broadcast as the lines of data for matrix for added nerd value

Thanks for the idea 💡 Live code here: Reality Override: 1. The Camera Hijack (Video to Texture) First, we had to get the physical world into the math engine. Using HTML5's getUserMedia API, I requested direct access to your phone's rear-facing camera. Instead of just displaying that video on the screen, I piped the live feed directly into a WebGL Texture (THREE.VideoTexture). This turns your live reality into a dynamic, 2D canvas that the graphics card can mathematically manipulate pixel by pixel at 60 frames per second. 2. The Optical Edge Detection (The LiDAR Bypass) This was the most crucial hack. Because Apple locks down WebXR and prevents the browser from accessing the physical LiDAR depth sensors, we had to "fake" spatial awareness. • I wrote a Sobel Filter directly into the fragment shader. • For every single pixel on your screen, the math looks at the brightness of the pixels immediately above, below, left, and right of it. • If it detects a sudden, sharp change in contrast (like the dark shadow of a doorframe, or the grout line between your floor tiles), the algorithm flags that exact pixel as a "physical edge." • This allowed us to map the physical geometry of your room using only optics, completely bypassing the need for depth sensors. 3. Phenomenological Warping (Bending the Lens) Before the math even looks for the edges, we warp the actual coordinate system (the UV map) of the camera feed. We aren't moving the pixels; we are bending the underlying fabric of the screen. • The Dolly Zoom: By tying the scale of the coordinate grid to a sine wave oscillating with time, the focal length of the camera appears to breathe in and out. • The Möbius Automorphism: I translated complex plane mathematics into the shader. By applying the formula f(z) = \frac{z - a}{1 - \bar{a}z} (where a is a wandering pole), the physical camera feed is violently folded into a Poincaré hyperbolic disk. • The Droste Singularity: To create the infinite zoom, the shader converts the Cartesian (x,y) coordinates into polar coordinates (r, \theta). It then takes the logarithm of the radius and applies a modulo operation (mod). This forces reality to recursively swallow itself. 4. The Kaleidoscopic Boundary Fix When the extreme math of the Möbius and Droste warps pulled the camera's coordinates "out of bounds" (stretching beyond the physical edges of the video feed), it previously rendered a black void. • To fix this, I added the absolute mirror equation: uv = 1.0 - abs(mod(uv, 2.0) - 1.0);. • This tells the graphics card: "If the math pulls you off the edge of the screen, seamlessly bounce the coordinates backward." This created the infinite, seamless kaleidoscope reflection instead of empty black space. 5. The QRI Hexagonal Injection Finally, we generated the Omni-Hexagonal lattice math. • I commanded the shader to blend the live camera feed with the glowing green matrix lines only where the Sobel filter detected a physical edge. • As you slide the DOSE higher, the shader logic shifts. It introduces the iridescent chromatic palette and eventually allows the geometry to bleed out from the edges and fill the negative space of the room. 6. The iOS Safari Override Apple's WebKit engine is notoriously aggressive about touch controls. When you tapped the "Initialize Camera" button, Safari was hijacking the tap, thinking you wanted to highlight the text, which prevented the code from executing. I injected a global CSS override (-webkit-user-select: none;) to completely paralyze iOS's text-selection behavior, forcing the phone to respect your tap as a hard execution command.

You could have just done this in a fraction of the time it took for unity in something like Resolume for a 10th of the cost. Or even unreal. Sorry not judging you, judging unity.

i want a matrixy thingy

That looks cool as!

They are definitely in the Matrix.

Heh, that's cool

Soooo cool 😎

🔥

Real time LiDAR fused with ARKit is turning environments into editable digital layers Unity just becomes the simulation shell on top of physical space

signal operator background + AR vision work is a rare combo. Cascade effect looks clean. What's your point cloud throughput on device?

So in other words… a reality sensor.

@hnshah 💯

@Flexi23 🔥 Nice! 🤔 You just gave me an a idea 💡

@SHIBjpn Now smoke DMT and point a laser at the wall. I hear you get the same effect. YouTube has several videos 🙃

I feel like this video is at least 2 years old. Or, I've seen similar demos since then.

Take my money

☠️ add to spotnik adsol.

This is cool af

@vectris brooo

Sushi recipes?