Since the early 2010s, visual search has been offering users a novel alternative to keyword-based search results.
But with the sophistication of visual search tools increasing, and tech giants like Google and Microsoft investing heavily in the space, what commercial opportunities does it offer brands today?
Visual search 101
There are two types of visual search. The first compares metadata keywords for similarities (such as when searching an image database like Shutterstock).
The second is known as ‘content-based image retrieval’. This takes the colour, shape and texture of the image and compares it to a database, displaying entries according to similarity.
From a user perspective, this massively simplifies the process of finding products they like the look of. Instead of trying to find the words to describe the object, users can simply take a photo and see relevant results.
Visual search engines: A (very) brief history
The first product to really make use of this technology was ‘Google Goggles’. Released in 2010, it offered some fairly basic image-recognition capabilities. It could register unique objects like books, barcodes, art and landmarks, and provide additional information about them.
It also had the ability to understand and store text in an image – such as a photo of a business card. However, it couldn’t recognize general instances of objects, like trees, animals or items of clothing.
CamFind took the next step, offering an app where users could take photos of any object and see additional information alongside shopping results. My tests (featuring our beautiful office plant) yielded impressively accurate related images and web results.
More importantly for brands, it offers advertising based on the content of the image. However, despite the early offering, the app has yet to achieve widespread adoption.
A Pinterest-ing development
A newer player in the visual search arena, image-focused platform Pinterest has what CamFind doesn’t – engaged users. In fact, it reached 150m monthly users in 2016, 70m of which are in the US with a 60:40 split women to men.
So what do people use Pinterest for? Ben Silbermann, its CEO and co-founder, summed it up in a recent blog post:
“As a Pinner once said to me, “Pinterest is for yourself, not your selfies”—I love that. Pinterest is more of a personal tool than a social one. People don’t come to see what their friends are doing. (There are lots of other great places out there for that!) Instead, they come to Pinterest to find ideas to try, figure out which ones they love, and learn a little bit about themselves in the process.”
In other words, Pinterest is designed for discovery. Users are there to look for products and ideas, not to socialize. Which makes it inherently brand-friendly. In fact, 93% of Pinners said they use Pinterest to plan for purchases, and 87% said they’d bought something because of interest. Adverts are therefore less disruptive in this context than platforms like Facebook and Twitter, where users are focused on socializing, not searching.
Pinterest took their search functionality to the next level in February 2017 with an update offering users three new features:
Shop the Look allowed users to pick just one part of an image they were interested in to explore – like a hat or a pair of shoes.
Related Ideas gives users the ability to explore a tangent based on a single pin. For example, if I were interested in hideously garish jackets, I might click ‘more’ and see a collection of equally tasteless items.
Pinterest Lens was the heavyweight feature of this release. Linking to the functionality displayed in Shop the Look, it allowed users to take photos on their smartphone and see Pins that looked similar to the object displayed.
In practice, this meant a user might see a chair they were interested in purchasing, take a photo, and find similar styles – in exactly the same way as CamFind.
Pinterest Lens today
What does it mean for ecommerce brands?
Visual search engines have the potential to offer a butter-smooth customer journey – with just a few taps between snapping a picture of something and having it in a basket and checking out. Pinterest took a big step towards that in May this year, announcing they would be connecting their visual search functionality to Promoted Pins – allowing advertisers to get in front of users searching visually by surfacing adverts in the ‘Instant Ideas’ and the ‘More like this’ sections.
For retail brands with established Pinterest strategies like Target, Nordstrom, Walgreens and Lululemon, this is welcome news, as it presents a novel opportunity for brands to connect with users looking to purchase products.
Product images can be featured in visual search results
Nearly 2 million people Pin product-rich pins every day. The platform even offers the ability to include prices and other data on pins, which helps drive further engagement. Furthermore, it has the highest average order value of any major social platform at $50, and caters heavily to users on mobile (orders from mobile devices increased from 67% to 80% between 2013-2015).
But while Pinterest may have led the way in terms of visual search, it isn’t alone. Google and Bing have both jumped on the trend with Lens-equivalent products in the last year. Both Google Lens and Bing Visual Search (really, Microsoft? That’s the best you have?) function in an almost identical way to Pinterest Lens. Examples from Bing’s blog post on the product even show it being applied in the same contexts – picking out elements of a domestic scene and displaying shopping results.
One interesting question for ecommerce brands to answer will be how to optimize product images for these kinds of results.
Google Lens, announced at Google’s I/O conference in May to much furore, pitches itself as a tool to help users understand the world. By accessing Google’s vast knowledge base, the app can do things like identify objects, and connect to your WiFi automatically by snapping the code on the box.
Of course, this has a commercial application as well. One of the use cases highlighted by Google CEO Sundar Pichai was photographing a business storefront and having the Google Local result pop up, replete with reviews, menus and contact details.
The key feature here is the ability to connecting a picture taken with an action. It doesn’t take too much to imagine how brands might be able to use this functionality in interesting and engaging ways – for example, booking event tickets directly from an advert, as demonstrated at I/O:
The future
Many marketers think we’re on the brink of a revolution when it comes to search. The growing popularity of voice search is arguably an indicator that consumers are moving away from keyword-based search and towards more intuitive methods.
It’s too soon to write off the medium entirely, of course – keywords are still by the far the easiest way to access most information. But visual search, along with voice, are certainly still useful additions to the roster of tools we might use to access information on the internet.
Ecommerce brands would be wise to keep close tabs on the progress of visual search tools; those that are prepared will have a significant competitive advance over those that aren’t.
This post was originally published on our sister site, ClickZ, and has been reproduced here for the enjoyment of our audience on Search Engine Watch.