Google Bard can now analyse photos, get Instagram and Facebook captions written by AI in seconds

By Abhik SenguptaGoogle Bard received its biggest update earlier this week. The most significant feature added to the AI chatbot was the support for image inputs. It means users can now upload photos and ask the AI chatbot to analyse them to help write quick captions for Instagram or Facebook and much more. The feature is free-to-use for all Bard users. It may also give Google Bard an edge over ChatGPT, which does not offer photo-analysis functionality to regular users (yet).

How to upload photos on Bard

The photo-upload feature is rolling out on Bard, though it only works with English. On the other hand, the AI chatbot can understand text inputs in 40 languages, including Hindi, Bengali, Tamil, Gujarati, and more.

Once you open Bard, you will notice a ‘plus’ icon next to the search bar at the bottom. Select the photo you want to upload and ask Bard to analyse it accordingly. During our test, we uploaded a photo of a rainy day and asked Bard to provide captions for Facebook/Instagram, which it did.

Similarly, if you have a list of ingredients written on paper, take a snapshot, upload it on Bard, and ask the AI chatbot to offer easy recipes. In some cases, Bard can also reply with relevant images, making the chat experience more appealing and relevant.

However, there are some visible limitations. When we uploaded a screenshot of the tweet and asked Bard to write a short paragraph, the AI chatbot was puzzled. It said, “I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited.”

In one case, Bard could not process the file, despite uploading a supported format (JPEG, PNG, and WebP).

How to use Bard with photo prompts

Google explains that Bard is using Google Lens’ underlying tech to understand images. Way before Bard, Google introduced Google Lens, which also leverages the power of AI to help users analyse photos and offer information accordingly. It has its use cases to identify options or even translate text on the image. At the Google IO 2023, Google said the idea was to turn “visual” to let more users use the platform.

Google said, “Let’s say you want to have some fun using a photo of your dogs. You can upload it and prompt Bard to ‘write a funny caption about these two.’ Using Google Lens, Bard will analyse the photo, detect the dogs’ breeds, and draft a few creative captions — all within seconds.”

But the feature, if utilised properly, could have many advantages. For instance, users can upload their calendars and ask Bard for help with creating schedules. Users can also effectively understand the food in front of them and get ingredients (general overview) to check if it uses anything that they’re allergic to.