Andrew Hart's banner
Andrew Hart's profile picture

Andrew Hart

@AndrewHart28,054 subscribers

Building robot brains at Base Intelligence 🤖🧠. Prev. Founder @ Hyper “Indoor Google Maps”, rolled out with IKEA. Pioneered AR navigation 🏳️‍🌈

Shorts

Apple Battersea store has closed early, they’re assembling a stage and lighting inside of the store.. and people are being checked in for an “invite only” event. 🤔 #Apple50

Apple Battersea store has closed early, they’re assembling a stage and lighting inside of the store.. and people are being checked in for an “invite only” event. 🤔 #Apple50

146,513 Aufrufe

I’ve just finished watching Apple’s visionOS sessions. Some of these concepts and the way Apple has implemented the design is mind blowing 🤯 Here are 5 examples that I loved: 1. Eye zoom. Just look at _where_ on an image you want to zoom in, and pinch your fingers apart

I’ve just finished watching Apple’s visionOS sessions. Some of these concepts and the way Apple has implemented the design is mind blowing 🤯 Here are 5 examples that I loved: 1. Eye zoom. Just look at _where_ on an image you want to zoom in, and pinch your fingers apart

1,141,143 Aufrufe

2020-2024: we built hyper-accurate indoor location tech for retail stores, but the sales process is super slow. 2025: we get enquiries from event venues, airports, warehouses, campuses, every other type of space, so now we’re opening it up to all.

2020-2024: we built hyper-accurate indoor location tech for retail stores, but the sales process is super slow. 2025: we get enquiries from event venues, airports, warehouses, campuses, every other type of space, so now we’re opening it up to all.

161,178 Aufrufe

Mirador is now open-source! Mirador makes it easy to build impressive point-of-interest experiences with Apple’s new AR framework, RealityKit. Demo: Tunnel View, Yosemite National Park

Mirador is now open-source! Mirador makes it easy to build impressive point-of-interest experiences with Apple’s new AR framework, RealityKit. Demo: Tunnel View, Yosemite National Park

122,857 Aufrufe

I’ve been working on a new open-source project for Apple’s RealityKit. Introducing Mirador. Mirador makes it easy to build impressive “Point of Interest” AR experiences, from anywhere in the world. Coming soon. Demo 1: Miradouro da Senhora do Monte, Lisbon.

I’ve been working on a new open-source project for Apple’s RealityKit. Introducing Mirador. Mirador makes it easy to build impressive “Point of Interest” AR experiences, from anywhere in the world. Coming soon. Demo 1: Miradouro da Senhora do Monte, Lisbon.

121,814 Aufrufe

Airports need Hyper location

Airports need Hyper location

10,633 Aufrufe

Videos

AndrewHart's profile picture

Craziest DM I ever received, from a VP at a global retailer: "Our app is shit and we know it's shit". I met her for coffee and she asked me if I could solve the biggest unsolved problem in retail. This is a deep dive into why and how Hyper built a 1m-accurate indoor GPS. This DM arrived in 2017. My outdoor AR navigation demos had just gone viral, and my new open-source project for Apple had elevated me to be the top trending iOS developer on GitHub. The retail exec told me they wanted to bring indoor maps and navigation to their retail stores, so customers could find what they’re looking for, and they could pop up relevant promotions along the way. It turns out that every office, university, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency. I assumed this was a solved problem. No. They do have maps on their app, but they aren’t able to navigate people because GPS doesn’t work indoors. They tried every solution out there to provide the blue dot, but nothing worked. I did know something about maps and location already — the first startup I worked at built an early version of Pokemon Go. I’d been tasked with generating the gamified maps, and populating the monsters and rewards. So I knew a bit about maps, coordinates and GPS — and monster training. But indoor navigation was new to me. Over the years, I’ve slowly become an expert in this, so let me explain. For indoor navigation to work well, the blue dot location needs to be 2x as accurate as a strong GPS signal. An aisle in a store is usually about 2 meters wide, so an accuracy wider than 2 meters would be fixing you in the wrong aisle. There are many research studies aimed at solving this, and Apple and Google have made acquisitions to help them in this area over the years. There were also many startups who claimed to have solutions, but when I spoke to their customers, I discovered that they weren’t happy with anything they’d tried: - Bluetooth beacons. Install thousands of these small sensors, which are a bit like AirTags, and use them for triangulation. But the bluetooth signals are noisy, making the location about 5 meters accurate, so it would jump you between multiple aisles in a store. Plus, lots of infrastructure to maintain. - WiFi. More promising than beacons, because every business has WiFi installed already. But the same radio signals problem means the location isn’t accurate enough. - Magnetomers, which use the earth’s magnetic field. This one sounded more promising. But it takes several minutes of walking around until it will give you a “blue dot”. So this was a bad user experience. - Computer Vision, which works like Google Street View. The user holds up their phone to scan the environment, it recognises their surroundings and locks them in. But this is clunky for the user, and they need to do this repeatedly every time they want a location update. Once again, bad user experience. Here’s an example of Apple’s own accuracy using WiFi. (I’ll show our own performance on these same sessions further down).

Andrew Hart

2,798,970 Aufrufe • vor 10 Monaten

AndrewHart's profile picture

Once we started to work with large global retailers, we needed a better way to scale this process. Ideally, the staff at the store could do this themselves — rather than us flying our team across the world — and then we could lower the cost and timelines. So we built a self-serve version of our survey app, with a tutorial mode designed for beginners. Over time, we collected millions of data points, and so we were able to develop an algorithm which would auto-correct mistakes. In other words, if the surveyor accidentally placed their ground-truth location in the wrong place on the map, we could use our algorithms to detect it, and correct it. So now we have WiFi, and with and our efforts on producing a high quality survey, we have the best WiFi positioning available. With WiFi on its own, it’s achieving 3 meter accuracy. This is a great foundation to build on. WiFi + Motion data To refine this down to 1-meter accuracy, we realised that we could combine WiFi with the same technology behind self-driving cars and robotics: a motion system called SLAM (Simultaneous Localization and Mapping). SLAM uses the accelerometer, gyroscope and camera system to understand precise device motion. Imagine a car driving through a tunnel, using the motion since its last GPS ping to keep location accurate until it comes out the other side. On a phone, this technology is very reliable, and measures device motion with high precision. But SLAM is measuring motion within its own coordinate space, it’s not aligned with the real world. SLAM tracks the user’s relative motion, like “moved forward 2 meters, then turned left”, but does “forward” mean “north”, or some other direction? It’s not calibrated, so it could mean any location, any direction. We can’t rely on the compass to help us out with this, because phone compasses are notoriously incorrect — everyone knows the frustration of being sent the wrong way down a street. So our job was to align this motion data with the triangulation data we were receiving from WiFi. We designed an algorithm that could simulate every possibility, filter the unlikely scenarios, and hone in your location, using WiFi as an anchor. So WiFi gives us the initial blue dot, SLAM gives us motion, and as the user starts walking and we receive more data, our algorithms can refine location accuracy down to a consistent 1-meter accuracy. We’ve tested these algorithms in many locations, on hundreds of hours of ground-truth data:

Andrew Hart

90,944 Aufrufe • vor 10 Monaten

Keine weiteren Inhalte verfügbar