Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

VPS tracking on smart glasses Experimenting with MultiSet AI’s Visual Positioning System (VPS) to determine a person’s position relative to a 3d scanned environment, using just an image Works with compatible smartglasses, in this case Omi AI’s Omi Glass (based on Seeed Studio’s ESP32-S3 Sense) running our custom firmware

Brilliant Wear

1,597 subscribers

13,290 views • 4 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Positional tracking on smart glasses and webcams Experimenting with Niantic Spatial 🌎’s 8th Wall engine to get position using camera and motion data Works with compatible smartglasses, in this case a modified Omi AI’s Omi Glass (based on the Seeed Studio ESP32-S3 Sense) running custom firmware Also works on webcams using our motion modules for supplying imu data

Positional tracking on smart glasses and webcams Experimenting with Niantic Spatial 🌎’s 8th Wall engine to get position using camera and motion data Works with compatible smartglasses, in this case a modified Omi AI’s Omi Glass (based on the Seeed Studio ESP32-S3 Sense) running custom firmware Also works on webcams using our motion modules for supplying imu data

Brilliant Wear

14,800 views • 4 months ago

3D hand tracking on smart glasses Not only can we stream camera data with our Wearables SDK, but we can run MediaPipe for hand tracking, applying it in WebXR environments like Three.js and A-Frame Works with compatible smart glasses, in this case the Omi AI Omi Glass, based on the Seeed Studio XIAO ESP32-S3 Sense (running our custom Firmware)

3D hand tracking on smart glasses Not only can we stream camera data with our Wearables SDK, but we can run MediaPipe for hand tracking, applying it in WebXR environments like Three.js and A-Frame Works with compatible smart glasses, in this case the Omi AI Omi Glass, based on the Seeed Studio XIAO ESP32-S3 Sense (running our custom Firmware)

Brilliant Wear

79,410 views • 6 months ago

Streaming Smart Glasses Camera to a Quest 3 in WebXR A-Frame Using our Wearables SDK, we’re able to stream camera data from the Omi AI Omi Glass (based on the Seeed Studio XIAO ESP32-S3 Sense running custom firmware) to the Quest 3 over WebSockets in the Oculus Browser

Streaming Smart Glasses Camera to a Quest 3 in WebXR A-Frame Using our Wearables SDK, we’re able to stream camera data from the Omi AI Omi Glass (based on the Seeed Studio XIAO ESP32-S3 Sense running custom firmware) to the Quest 3 over WebSockets in the Oculus Browser

Brilliant Wear

18,473 views • 6 months ago

Ported our firmware to the Seeed Studio XIAO ESP32-S3 Sense, allowing us to use our Wearables SDK with devices like the Omi Glass from Omi AI Stream camera and microphone data via Bluetooth or WiFi (WebSockets), and even motion data if you attach a compatible imu

Ported our firmware to the Seeed Studio XIAO ESP32-S3 Sense, allowing us to use our Wearables SDK with devices like the Omi Glass from Omi AI Stream camera and microphone data via Bluetooth or WiFi (WebSockets), and even motion data if you attach a compatible imu

Brilliant Wear

64,936 views • 7 months ago

Experimenting with wrist controls on smart glasses Using our Wearables SDK, we can display a 3D wireframe A-Frame scene on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware) We’re also using our SDK to control a virtual cursor with our Ukaton motion module strapped to our wrist, running an Edge Impulse model to detect pinches There’s a lot of third party hand control accessories we could use (TapWithUs, Doublepoint, and Mudra Band), but using our own hardware/software is great for creating custom gestures

Experimenting with wrist controls on smart glasses Using our Wearables SDK, we can display a 3D wireframe A-Frame scene on compatible smartglasses (in this case the Brilliant Labs Frame running our custom firmware) We’re also using our SDK to control a virtual cursor with our Ukaton motion module strapped to our wrist, running an Edge Impulse model to detect pinches There’s a lot of third party hand control accessories we could use (TapWithUs, Doublepoint, and Mudra Band), but using our own hardware/software is great for creating custom gestures

Brilliant Wear

29,397 views • 9 months ago

Realtime Maps on Smart Glasses Using our Wearables SDK and OpenStreetMap to display a simple map overlay, allowing us to see what places are around us (and if they’re open or not) Works with compatible smart glasses (in this case the Brilliant Labs Frame running custom firmware)

Realtime Maps on Smart Glasses Using our Wearables SDK and OpenStreetMap to display a simple map overlay, allowing us to see what places are around us (and if they’re open or not) Works with compatible smart glasses (in this case the Brilliant Labs Frame running custom firmware)

Brilliant Wear

53,763 views • 9 months ago

VPS or visual positioning system... this is how AR glasses and robots will understand where they are in the real world and know where to go. It’s also how you can spatially annotate reality with cm level accuracy - all using a machine readable model of the world.

VPS or visual positioning system... this is how AR glasses and robots will understand where they are in the real world and know where to go. It’s also how you can spatially annotate reality with cm level accuracy - all using a machine readable model of the world.

Bilawal Sidhu

27,898 views • 7 months ago

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion

Wall hacks in real life using augmented reality. New mad science experiment -- I used $300 Meta RayBan glasses and an iPhone to recreate EagleEye's "see-through-walls" tracking. The answer was visual positioning tech to build a shared spatial map. Code on GitHub. 0:00 - The Experiment 0:38 - Dumb Glasses, Smart Tracking 1:54 - No Street View? No Problem 3:17 - Seeing Through Walls With a Phone 4:24 - The Indoor Problem Nobody Solved 5:20 - This Is What It Can Do 6:38 - Use Cases Beyond the Flex 7:44 - The Double Edge 10:32 - Conclusion

Bilawal Sidhu

26,863 views • 2 months ago

How lighting works in Simulon Using a professional on-set VFX kit, we showcase our unique Image Based Lighting system. Simulon employs ML to create a 360° HDRI from just one LDR shot, aligning it perfectly with scene data. Our virtual camera replicates the physical camera’s response to light, syncing seamlessly with the HDR info. Notice how the dynamic exposure changes on the virtual VFX kit closely match the real one. Bonus: At the end of the video I place a 3D Dragon in the scene and make a capture, this capture is then rendered in the cloud to produce a photorealistic VFX video 🐉 More on the capture and render process in upcoming videos!

How lighting works in Simulon Using a professional on-set VFX kit, we showcase our unique Image Based Lighting system. Simulon employs ML to create a 360° HDRI from just one LDR shot, aligning it perfectly with scene data. Our virtual camera replicates the physical camera’s response to light, syncing seamlessly with the HDR info. Notice how the dynamic exposure changes on the virtual VFX kit closely match the real one. Bonus: At the end of the video I place a 3D Dragon in the scene and make a capture, this capture is then rendered in the cloud to produce a photorealistic VFX video 🐉 More on the capture and render process in upcoming videos!

Divesh Naidoo

275,117 views • 2 years ago

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning

Enabling autonomous vehicles perceive their environment using only off-the-shelf cameras has been a long term research objective at Swaayatt Robots. This demo highlights the capabilities of our on-road perception system which is able to detect obstacles, road boundaries, lane markers in images, as well as compute depth of the complex scenes in the environment. The output shown in this video is end-to-end raw output from our deep learning system, without any post processing. The current system, with joint computation of obstacles, lane/road boundaries, and depth, works at 30 FPS on an embedded GPU in our autonomous vehicle, and can achieve higher FPS with further optimization -- which is currently a research in progress. This system is being scaled up for both the day and night operations, and we will showcase its strength towards enabling autonomous driving on a mountainous environment with unpaved roads, in the absence of any delimiters. #deeplearning #autonomousdriving #autonomousvehicles #machinelearning

Sanjeev Sharma

18,478 views • 1 year ago

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at

Ilir Aliu

101,689 views • 1 year ago

Robots are gaining spatial awareness with Over the Reality. 🤖 Powered by the world’s largest 3D dataset, we enable robots to see, sense, and collaborate in real-world environments. This early test shows how a Unitree Go2 perceives its surroundings through OVER’s VPS — on the right, the real life robot; on the left, its real-time perception of the position inside one of OVER’s 3D maps. Robotic spatial awareness is here, living in 155,000+ mapped locations and expanding every day. One step closer to Physical AI. Stay tuned for our upcoming ROS 2 integration, unlocking a universal framework for robotic perception leveraging OVER's infrastructure!

Robots are gaining spatial awareness with Over the Reality. 🤖 Powered by the world’s largest 3D dataset, we enable robots to see, sense, and collaborate in real-world environments. This early test shows how a Unitree Go2 perceives its surroundings through OVER’s VPS — on the right, the real life robot; on the left, its real-time perception of the position inside one of OVER’s 3D maps. Robotic spatial awareness is here, living in 155,000+ mapped locations and expanding every day. One step closer to Physical AI. Stay tuned for our upcoming ROS 2 integration, unlocking a universal framework for robotic perception leveraging OVER's infrastructure!

Over the Reality 🌐

263,818 views • 9 months ago

One of our favourite things about this project, The Enigmatic Valley from 3D Environment Artist Abanob Atef, is how much joy he expressed in creating it 🏔️ Using the project as a means to try out new techniques, Abanob worked with everything from Megascans to custom shaders and Lumen. You can dig into the specifics over on his ArtStation:

One of our favourite things about this project, The Enigmatic Valley from 3D Environment Artist Abanob Atef, is how much joy he expressed in creating it 🏔️ Using the project as a means to try out new techniques, Abanob worked with everything from Megascans to custom shaders and Lumen. You can dig into the specifics over on his ArtStation:

Unreal Engine

460,173 views • 1 year ago

$Whitney Webb Sounds Alarm on AI’s Threat to Humanity’s Future Investigative journalist Whitney Webb unpacks the chilling vision in Eric Schmidt and Henry Kissinger’s book The Age of AI: Our Human Future. While framed as warnings, Webb argues their words reveal a troubling agenda that demands urgent attention. Thread below. Schmidt and Kissinger predict AI will fracture society into two classes: an elite tier of AI programmers and decision-makers who control its objectives, and a vast underclass acted upon by AI, stripped of the ability to understand its influence. Webb calls this a modern haves and have-nots divide, driven by what the authors term cognitive diminishment. Cognitive diminishment is simple yet insidious: the less we use our skills, the more we lose them. Just as relying on calculators erodes mental math, outsourcing creativity and decision-making to AI risks eroding our ability to think independently. From AI curating your music to dictating your wardrobe, Webb warns of a future where humans lose the capacity to form preferences without AI’s nudge. Webb points to the rise of generative AI as a tipping point. People are creating less, she says. They’re outsourcing art, writing, and music to AI. In a generation, we could lose the ability to create entirely. This isn’t just a loss of skills—it’s a surrender of humanity’s essence. Without creativity, we risk becoming dependent on machines, reduced to mere processors in a system not unlike The Matrix. The solution? Resist. Create. Everyone can make something, Webb urges. Whittle wood, weave, paint, sing—humans have always created. If we abandon that, a posthuman future is inevitable. She stresses that this fight is accessible to all, regardless of wealth or status. Creativity is our defense against AI’s creeping control. Webb also critiques the blind trust in AI’s hallucinations—false realities it perceives that humans can’t verify. Schmidt and Kissinger suggest these unobservable truths should shape society, given AI’s supposed superiority. Why trust a human-made machine’s illusions? Webb asks. Powerful figures like Schmidt and Kissinger seem to worship AI with religious fervor, but that’s no reason to reshape humanity around its flaws. The stakes are high. If we stop creating, stop thinking, we risk a world where independent thought is impossible, and AI’s errors—or agendas—dictate reality. Webb calls on everyone to resist by reclaiming creativity and questioning AI’s role in our lives. Read what AI’s architects are saying, she insists. The long-term consequences are too grave to ignore. Will we let convenience erode our humanity, or will we fight for our creative soul? The choice is ours—but time is running out.$

Whitney Webb Sounds Alarm on AI’s Threat to Humanity’s Future Investigative journalist Whitney Webb unpacks the chilling vision in Eric Schmidt and Henry Kissinger’s book The Age of AI: Our Human Future. While framed as warnings, Webb argues their words reveal a troubling agenda that demands urgent attention. Thread below. Schmidt and Kissinger predict AI will fracture society into two classes: an elite tier of AI programmers and decision-makers who control its objectives, and a vast underclass acted upon by AI, stripped of the ability to understand its influence. Webb calls this a modern haves and have-nots divide, driven by what the authors term cognitive diminishment. Cognitive diminishment is simple yet insidious: the less we use our skills, the more we lose them. Just as relying on calculators erodes mental math, outsourcing creativity and decision-making to AI risks eroding our ability to think independently. From AI curating your music to dictating your wardrobe, Webb warns of a future where humans lose the capacity to form preferences without AI’s nudge. Webb points to the rise of generative AI as a tipping point. People are creating less, she says. They’re outsourcing art, writing, and music to AI. In a generation, we could lose the ability to create entirely. This isn’t just a loss of skills—it’s a surrender of humanity’s essence. Without creativity, we risk becoming dependent on machines, reduced to mere processors in a system not unlike The Matrix. The solution? Resist. Create. Everyone can make something, Webb urges. Whittle wood, weave, paint, sing—humans have always created. If we abandon that, a posthuman future is inevitable. She stresses that this fight is accessible to all, regardless of wealth or status. Creativity is our defense against AI’s creeping control. Webb also critiques the blind trust in AI’s hallucinations—false realities it perceives that humans can’t verify. Schmidt and Kissinger suggest these unobservable truths should shape society, given AI’s supposed superiority. Why trust a human-made machine’s illusions? Webb asks. Powerful figures like Schmidt and Kissinger seem to worship AI with religious fervor, but that’s no reason to reshape humanity around its flaws. The stakes are high. If we stop creating, stop thinking, we risk a world where independent thought is impossible, and AI’s errors—or agendas—dictate reality. Webb calls on everyone to resist by reclaiming creativity and questioning AI’s role in our lives. Read what AI’s architects are saying, she insists. The long-term consequences are too grave to ignore. Will we let convenience erode our humanity, or will we fight for our creative soul? The choice is ours—but time is running out.

Camus

49,825 views • 1 year ago

Here is a full showcase of my custom Source 2-like light baking and probe implementation for Unreal Engine 5. Features showcased: - Volume based probe system (full density control, fade between volumes with a volume fade slider, automatic volume priority and custom priority overrides) - GPU Lightmass with expanded features like a custom denoiser and overall QOL (like a bake volume which only bakes objects inside the volume for fast prototyping) - Custom Box Capture implementation to make it easier to use (basically just a custom way of scaling the box capture component since its so annoying) - A custom fullbright viewport option for easier map editing (without having to get visual clutter in the way) Will eventually release on FAB as a plugin for UE 5.7. Everything is running inside an Epic Launcher Downloaded version of UE5 in this showcase. #UnrealEngine5 #UE5 #GameDev

Here is a full showcase of my custom Source 2-like light baking and probe implementation for Unreal Engine 5. Features showcased: - Volume based probe system (full density control, fade between volumes with a volume fade slider, automatic volume priority and custom priority overrides) - GPU Lightmass with expanded features like a custom denoiser and overall QOL (like a bake volume which only bakes objects inside the volume for fast prototyping) - Custom Box Capture implementation to make it easier to use (basically just a custom way of scaling the box capture component since its so annoying) - A custom fullbright viewport option for easier map editing (without having to get visual clutter in the way) Will eventually release on FAB as a plugin for UE 5.7. Everything is running inside an Epic Launcher Downloaded version of UE5 in this showcase. #UnrealEngine5 #UE5 #GameDev

Leon

15,647 views • 22 days ago

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 views • 3 years ago

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Prince Canuma

82,169 views • 2 months ago

It was a pleasure to create The Game first-ever official AI music video 🎥🔥 This video came together thanks to a full team of our talented creators, each bringing their own unique skills, all focused on one goal— delivering an impressive and epic final video. While some still doubt AI’s role in the industry, we’re proud to show that AI music videos are here to stay. With more iconic rappers on our list in the coming months, this is just the beginning. 💪🎬 Link to the official release in the first comment 👇

It was a pleasure to create The Game first-ever official AI music video 🎥🔥 This video came together thanks to a full team of our talented creators, each bringing their own unique skills, all focused on one goal— delivering an impressive and epic final video. While some still doubt AI’s role in the industry, we’re proud to show that AI music videos are here to stay. With more iconic rappers on our list in the coming months, this is just the beginning. 💪🎬 Link to the official release in the first comment 👇

The Dor Brothers

115,557 views • 1 year ago

GROK 3 JUST BUILT A FULL 3D MULTIPLAYER GAME—NO SWEAT AI-powered game development just took a massive leap. A full 3D single & multiplayer game—complete with raycasting, AI-driven enemies, and procedural city generation—was built using Grok 3. No struggling with token limits, no hitting walls with confused models. Just a working game. Grok 3 handled everything: speed boosters, AI bot patrols, collision detection, health systems, and even an algorithm to auto-generate cities. Multiplayer? Grok synced player movement, shots, and even built a login system with character selection. Running on an Express server with WebSockets, this game proves one thing—Grok 3 is amazing. Source: Dan

GROK 3 JUST BUILT A FULL 3D MULTIPLAYER GAME—NO SWEAT AI-powered game development just took a massive leap. A full 3D single & multiplayer game—complete with raycasting, AI-driven enemies, and procedural city generation—was built using Grok 3. No struggling with token limits, no hitting walls with confused models. Just a working game. Grok 3 handled everything: speed boosters, AI bot patrols, collision detection, health systems, and even an algorithm to auto-generate cities. Multiplayer? Grok synced player movement, shots, and even built a login system with character selection. Running on an Express server with WebSockets, this game proves one thing—Grok 3 is amazing. Source: Dan

Mario Nawfal

1,129,147 views • 1 year ago