Apple’s relationship with Google as a search partner is taking a new turn with the introduction of visual search, or “visual intelligence” as the iPhone maker called it on Monday during the company’s “Glowtime” event. Apple already pays Alphabet around $20 billion a year to make Google the default search engine in its Safari browser. Now, iPhone 16 users can access Google’s search engine – and its visual search features – with a click of the device’s new camera control button.
OpenAI’s ChatGPT, which will soon be accessible via Siri, was also featured as a third-party partner in a demo where you could point your phone’s camera at your class notes and get help understanding the concept or problem with the click of a mouse.
With the camera controls, Apple explained how users can quickly take a photo or video, and how they can hover their finger over the button to frame their shot and adjust options like zoom, exposure, or depth of field in a new camera preview. However, the button also gives iPhone 16 users access to Apple’s new “visual intelligence” search feature, and that’s where the Google partnership comes in.
When first unveiled, the iPhone 16’s camera controls seemed like Apple’s jargon for “shutter,” but as the event went on, Apple explained that there’s more you can do with this new hardware feature. With Visual Intelligence, there’s more than just an easy way to learn more about the things in the camera’s field of view; you now have another way to access third-party services without having to launch standalone apps.
Essentially a visual search feature, similar to Google Lens or Pinterest Lens, Apple described Visual Intelligence as a way to instantly learn about everything you see. Using a few examples, Apple demonstrated how a click of the camera control button could get information about a restaurant you saw while out on the town, or how the feature could identify the breed of a dog you saw while out walking. The feature could also turn an event poster pinned to the wall into a calendar entry with all the details.
Craig Federighi, Apple’s senior vice president of software engineering, then casually mentioned that the feature could also be used to access Google search.
“The camera control is also your gateway to third-party tools, giving you quick and easy access to their specific expertise. So if you come across a bike that looks exactly like the one you’re looking for, just tap Google to find where you can buy something similar,” he said.
The demo showed a person tapping the camera control button while pointing their iPhone at a bike, then reviewing a series of similar options to purchase in a pop-up window above the camera view. The grid of images and descriptions of the matching bikes was then followed by a smaller button on the screen that said “More results from Google,” indicating that you could continue your Google search with another tap.
What Apple didn’t explain is how or when a press of the camera control button would detect that one should turn to a third-party partner for an answer rather than a built-in Apple service — like Apple Maps, which was shown in the demo about the restaurant. Nor did the company fully explain how users can control or configure this feature. Instead, Federighi said somewhat vaguely, “Of course, you always have control over when third-party tools are used.”
When contacted, a Google spokesperson said the company could not comment on its partnership at this time. Apple did not respond to a request for comment. However, we assume the deal is part of the two companies’ existing relationship and has nothing to do with Google’s Gemini AI.
What’s interesting about this feature is that it represents a new paradigm for interacting with software and services beyond those Apple ships with the iPhone. And it comes at a time when the concept of an app store seems outdated.
AI technology allows users to ask questions, perform productivity tasks, be creative with images and videos, and more. These are things consumers once relied on apps for, but they can now do through a new interface for speaking and texting with an AI assistant.
Rather than immediately jumping into developing its own competitor to ChatGPT, Apple is pitching itself as a platform to reach third-party services, including AI technologies, search services, and likely other providers in the future. Moreover, the company can make these connections through behind-the-scenes agreements with partners – such as its partnership with OpenAI on select AI features – rather than relying on in-app transactions as a revenue stream.
It also intelligently prevents Apple’s reputation from being damaged when a third-party developer, such as ChatGPT, makes mistakes (which is common with AI) or a Google search doesn’t produce helpful results.
