Let’s talk about risks w/ Apple’s new camera button & Visual Intelligence AI tools + integrations -- the potential ability to learn a stranger’s identity by simply taking a picture. Without big 3rd party integration guardrails, this new camera button + AI could invade privacy.

Yes, it’s exciting to be able to snap a pic and learn what you see with AI! Within the Apple ecosystem, there are guardrails to prevent those AI tools from invading privacy. For example, if you upload a pic of a person to the integrated ChatGPT it refuses to tell you who it is.

The camera control button is described above as “your gateway to 3rd party tools”. If 3rd parties can build for the AI integration feature w/ the Camera Control button, and AI already exists to allow you to learn identity from a picture, will Apple prevent those integrations?

We already know AI tools that allow you to upload a photo to learn someone’s identity — PimEyes. Will Apple allow for a 3rd party integration with the camera control button to allow identity to be uncovered at the time of snapping a pic? I bring this up because right now PimEyes takes a few steps, photos are taken on camera, uploaded to PimEyes to reveal identity, it’s not seamless. But we can imagine how much easier and accessible this will be for nefarious folks if they can easily, without taking any extra steps, snap a picture on their iPhone and reveal someone’s identity.

We already see people uploading strangers to their social media to help them “figure out who this hot person is” and other privacy invading behaviors. We know folks already try to uncover people’s identity — let’s avoid making it seamless and 1-step.

I’m looking forward to hearing Apple’s standards for allowing 3rd party integrations with their Camera Control Visual Intelligence. I’m hoping Apple is very much aware of nefarious folks hoping to build AI integrations that uncover identity using AI tools in a 1-step solution. Here’s to hoping Apple is strict in their 3rd party requirements to avoid making it easy, simple, and seamlessly integrated for bad folks trying to uncover someone’s identity the moment they take their photo.

Question: but folks can already use AI tools to uncover someone’s identity by taking a picture and then uploading it to a site like PimEyes? Answer: yes, this is already possible, which stinks. Let’s ensure Apple doesn’t make these types of privacy invading tools much more accessible, easy to use, and integrated right at the time of taking a photo. I wish these tools weren’t out there and we want more friction when using them, not less. We want it to be annoying to snap a photo and then take it to a tool and upload it separately. The easier and more integrated the AI privacy invading tool is into the camera in the pocket, the more we’ll see privacy invasion happen and normalized.

Question: But wait, isn't this the same thing as Google Lens, how is this any different of a risk? Answer: Google Lens doesn't do a great job at uncovering identity for *non celebrities* from a simple stranger photo on the street. PimEyes AI tool however, does succeed at uncovering identities for everyday folks (non celebs). We want to add more friction into using tools like this, not make it *more seamless* at the time of snapping a photo on a device that many people have in their pocket at all times.

And here is the exact situation I warned about, less than a month later (on Meta glasses). The more cameras that ship with w/ AI tools without hard boundaries around integrations with tools like Pimeyes, the more we'll have massive privacy violations IRL.

Snap a regular pic and pop it into google. I get where you are coming from, but that ship sailed a long time ago

Here's some thoughts about how decreasing friction with an AI integration like this slightly different, Google Lens sometimes works to uncover identity with people face photos, but often does not, especially for non-celebs!