Apple's AI-powered take on Google Lens breaks cover
Apple has unveiled a new artificial intelligence (AI)-powered visual search feature for its latest iPhone models, the iPhone 16 series. The announcement was made at the "It's Glowtime" event. This innovative feature, known as "visual intelligence," is accessible through a button called Camera Control on the devices. It combines reverse image search with text recognition capabilities, offering users a unique way to interact with their surroundings.
How does the visual intelligence feature work?
The visual intelligence feature is activated by a touch-sensitive button on the right side of the iPhone 16 models. Once activated, it can identify objects and provide relevant information or suggest actions based on what is captured. For instance, if a user points their device at a restaurant, the feature could display operating hours, ratings, and even options to view the menu or book a table.
A tool for everyday tasks
The visual intelligence feature can also be used to quickly add event details from a flyer to a user's calendar. It could identify the breed of a dog or help find where to purchase a product online. Apple has emphasized that this feature respects user privacy, as all data processing occurs on the device itself without storing images or informing Apple about what was clicked on by the user.
Apple's partnership with OpenAI enhances visual intelligence
In collaboration with OpenAI, Apple is enhancing the functionality of the Camera Control button on iPhone 16 models. Users can now send queries to ChatGPT, a feature that could prove useful for tasks such as homework assignments. Later this year, Camera Control will also integrate third-party tools with specific domain expertise, allowing users to use Google for product searches or ChatGPT for problem-solving, while maintaining control over data access and sharing.
Availability and future plans
The visual intelligence feature is set to launch in beta for US English language users in October, with a broader rollout expected in December and early 2025. However, the full suite of Apple Intelligence features won't be readily available upon purchase of the iPhone 16 series.
