Google is further expanding the capabilities of Google Photos with Gemini’s GenAI features. With the capabilities of the special AI assistant Ask Photos, users will soon no longer have to scroll endlessly through their large amounts of stored photos and videos.
The arrival of the Ask Photos GenAI features for Google Photos builds on the already existing advanced search functionality in this app. Currently, the app allows users to search their photo and video galleries for people, animals, locations and more. It can also put old photos and videos into a timeline in a draft book format.
Uncovering details
Gemini’s Ask Photos GenAI features in the app extend these existing intelligent search features to include understanding context and the subject matter of photos. This makes it possible to retrieve details from these photos or videos.
In this way, with normal queries, users can surface certain memories or other information. This should prevent users from having to scroll endlessly through their numerous stored photos and videos to find the information they need.
How GenAI functionality works
Under the hood of Ask Photos, via the GenAI feature, the app first interprets the search query and then creates a plan to find the answer. To do this, it undertakes a search in which it identifies not only relevant keywords, such as places, people or dates, but also language concepts such as “birthday theme”.
The search results are then examined, identifying which are the most relevant and which are exactly what the end user is looking for. To do this, Gemini’s multimodal properties are used to understand exactly what is happening in each image and also reads any relevant text in the image.
Ultimately, an answer is distilled from this and the relevant photos and/or videos are displayed based on this.
Attention to privacy
Google explicitly states that the addition of the GenAI functionality to the Photos app will not affect the privacy of end users. Especially since photos are often sensitive personal data.
According to the tech giant, personal data in Google Photos is never used for advertisements and conversations and photos are never viewed by third parties except in very extreme cases. Furthermore, the photos and videos are not used to train other GenAI products outside of Google Photos, including Google’s own Gemini LLMs.
Ask Photos’ new GenAI features will be gradually rolled out within the Google Photos app this summer. Google indicates that the features are still an experiment and thus subject to change.
Also read: Google’s AI assistant Gemini now searches your mail and documents