Skip to main content

Is your iPhone sharing photos with Apple by default?

Screenshot of the Enhanced Visual Search toggle.
iOS 18 introduced an “Enhanced Visual Search” toggle for the Photos app. | Screenshot: iOS 18 Settings app

Apple occasionally makes choices that tarnish its strong privacy-forward reputation, like when it was secretly collecting users’ Siri interactions. Yesterday, a blog post from developer Jeff Johnson highlighted such a choice: an “Enhanced Visual Search” toggle for the Apple Photos app that is seemingly on by default, giving your device permission to share data from your photos with Apple.

Sure enough, when I checked my iPhone 15 Pro this morning, the toggle was switched to on. You can find it for yourself by going to Settings > Photos (or System Settings > Photos on a Mac). Enhanced Visual Search lets you look up landmarks you’ve taken pictures of or search for those images using the names of those landmarks.

To see what it enables in the Photos app, swipe up on a picture you’ve taken of a building and select “Look up Landmark,” and a card will appear that ideally identifies it. Here are a couple of examples from my phone:

A split-screen image showing two searches, one correctly identifying a cathedral, the other misidentifying a building as the New Melleray Abbey near Dubuque, Iowa. Screenshots: Apple Photos
That’s definitely Austin’s Cathedral of Saint Mary, but the image on the right is not a Trappist monastery, but the Dubuque, Iowa city hall building.

On its face, it’s a convenient expansion of Photos’ Visual Look Up feature that Apple introduced in iOS 15 that lets you identify plants or, say, find out what those symbols on a laundry tag mean. But Visual Look Up doesn’t need special permission to share data with Apple, and this does.

A description under the toggle says you’re giving Apple permission to “privately match places in your photos with a global index maintained by Apple.” As for how, there are details in an Apple machine-learning research blog about Enhanced Visual Search that Johnson links to:

The process starts with an on-device ML model that analyzes a given photo to determine if there is a “region of interest” (ROI) that may contain a landmark. If the model detects an ROI in the “landmark” domain, a vector embedding is calculated for that region of the image.

According to the blog, that vector embedding is then encrypted and sent to Apple to compare with its database. The company offers a very technical explanation of vector embeddings in a research paper, but IBM put it more simply, writing that embeddings transform “a data point, such as a word, sentence or image, into an n-dimensional array of numbers representing that data point’s characteristics.”

Like Johnson, I don’t fully understand Apple’s research blogs and Apple didn’t immediately respond to our request for comment about Johnson’s concerns. It seems as though the company went to great lengths to keep the data private, in part by condensing image data into a format that’s legible to an ML model.

Even so, making the toggle opt-in, like those for sharing analytics data or recordings or Siri interactions, rather than something users have to discover seems like it would have been a better option.



from The Verge - All Posts https://ift.tt/qNISFQs
Labels:

Comments

Post a Comment

Popular posts from this blog

Pandora Stories lets artists add commentary to their own playlists

Pandora launched Stories today, a tool that lets artists and creators add voice commentary to their own playlists. The Stories feature merges podcasts with music playlists, and is meant for artists to add context to an album, or for podcasters to experiment with new storytelling formats. The feature is part of Pandora AMP, the streaming service’s free Artist Marketing Platform that helps creators promote their work. To kick off the launch, Pandora’s prepared some Stories by artists like John Legend and Daddy Yankee, who tell listeners their personal stories interspersed between their own songs. There’s also a Stories playlist called Love Songs That Aren’t Really Love Songs , which includes commentary on individual songs like a podcast... Continue reading… from The Verge - All Posts https://ift.tt/2Xz1oNc
Post a Comment
Read more

Instagram’s updated algorithm prioritizes original content instead of rip-offs

Image: Kristen Radtke / The Verge Instagram is making significant changes to how its system recommends content, with a focus on original content and increased distribution for smaller accounts. The slew of changes were announced by the company in a blog post today. The biggest change deals with aggregators — accounts that download or screenshot other users’ videos and photos and repost them. Sometimes aggregators will credit the original poster by tagging them in the post or caption, but often, content is wholesale ripped off with no acknowledgment, and engagement is siphoned off from the person who created the content in the first place. Instagram clearly has a problem with this and will begin removing reposted content from recommendations across the platform. The... Continue reading… from The Verge - All Posts https://ift.tt/ECgcPAU
Post a Comment
Read more

We asked camera companies why their RAW formats are all different and confusing

When you set up a new camera, or even go to take a picture on some smartphones, you’re presented with a key choice: JPG or RAW? JPGs are ready to post just about anywhere, while RAWs yield an unfinished file filled with extra data that allows for much richer post-processing. That option for a RAW file (and even the generic name, RAW) has been standardized across the camera industry — but despite that, the camera world has never actually settled on one standardized RAW format. Most cameras capture RAW files in proprietary formats, like Canon’s CR3, Nikon’s NEF, and Sony’s ARW. The result is a world of compatibility issues. Photo editing software needs to specifically support not just each manufacturer’s file type but also make changes for each new camera that shoots it. That creates pain for app developers and early camera adopters who want to know that their preferred software will just work. Adobe tried to solve this problem years ago with a universal RAW form...
Post a Comment
Read more