Patents are interesting things. Just because a company patents an idea or technology, does not mean it will be incorporated into their business.
They may patent an idea that proves too difficult to incorporate. The idea/technology may not be as valuable in practice as anticipated. They may just patent the idea to stop others from using it.
There are a variety of motivations for researching and filing for a patent. And there are many ends it can come to.
I mention all this as it’s important for anyone reading about a patent to know that conclusions drawn are educated guesses.
With this in mind, a recent patent granted to Google seems highly related to other patents they have been granted and was filed just a few months after them.
I would suggest it is highly likely that this one will be implemented – though likely in ways not directly outlined in it.
The patent I am referring to is patent US 2019/0325444A1 – Automatic Delivery Of Customer Assistance At A Physical Location.
The patents I feel it builds on I have written about previously. They are:
In the first of these two patents, we saw Google outlining systems to monitor what people were doing and correct potential errors. This would include things like potentially missing preparing for an event on a calendar, driving to LaGuardia instead of JFK for a flight, etc.
We then saw Google build on this and offer purchase suggestions (do you need a new outfit for that wedding we see on your calendar?).
No surprise there.
In the patent, they were just granted we see systems and methods outlined, that facilitate the automated sending of data and notification to a user within a brick-and-mortar store (the main focus) or by extension, within the vicinity of a desired retail location (touched on).
TL;DR
I’m about to summarize the key sections of patent US 2019/0325444A1 – specifically, what is practically applicable to marketers.
In short, the system is designed to monitor:
- Where you are in a store.
- Combine everything that Google and the store know about you.
- Use that to send notifications to assist you in buying more stuff from the brick-and-mortar location, as opposed to going online.
Now we all know that Google makes a lot of money from serving ads of online businesses.
So why are they doing this? Keep reading! Aat the end of The Patent Outline section, I will discuss my own take on why Google would patent and implement this strategy.
But first, let’s go through the patent.
The Patent Outline
I’ll save you reading the patent and outline some of the key passages and how they may impact you.
The headings of these sub-sections will simply be the sections of the patent being discussed for easy reference.
ABSTRACT
The system is intended to monitor user behavior and combine said behavior with their location within a store, send them information that may support their purchase decision and, failing that, provide access to remote assistance and, should that fall short as well, send a signal to the store that a user is in need of information and dispatch a sales associate to their location.
Section 0002
The system will detect a user’s location using their phone or other device and using their location relative to products in that area, provide that users with information they may find useful in completing a purchase.
Section 0011
The first tier (of a multi-tier support system) uses AI and machine learning systems to determine the likelihood that specific product information would be useful to a user in making a purchase decision.
If it is, then that information (generally product-related) will be set to the user in a popup, notification, or other similar methods.
This information may change as the user moves or provides additional input.
Section 0012
In the second tier of the system, if the information available to the system is not considered likely to provide information that will promote a sale or if the user requests, a chat (via test or voice) can be initiated with a human in a remote location to provide assistance.
Section 0015
The most interesting point here is in the generalization of the location, such as ZIP codes.
Given that this would be useless in assisting sales staff within a brick-and-mortar location, one can extrapolate that the patent is developed with external intents as well.
Section 0025
The consideration of monitoring a user’s behavior over multiple devices and multiple sessions is discussed.
They also include the condition that the user and device may be at different locations, implying that a user may have run a query on one device (their PC, gaming console, etc.) and have a different device on their person.
Section 0029
A new concept is introduced: additional sensors (beacons) being used within the location to help determine where a user is physically located with a high degree of accuracy.
These devices are outlined to potentially include a wide array of types from position sensors to microphones to barometers (that one still has me scratching my head).
Section 0030
The system can be used to pinpoint a user to an aisle within a physical store.
Section 0034
This section provided the Eureka moment for me personally. The one that describes the direct purpose of the patent.
We see the introduction of promotional information being presented to the user.
Until this point, the information being discussed was generally product details and similar contexts. It’s in this section we read about serving a user ads, coupons, etc.
Section 0038
Here, the promotional theme is carried to the next level.
We see the device and user data combined with loyalty program information, past purchase and customer info data if applicable, etc. Basically, combining what the store may know about you with what Google does to tailor the data served.
Section 0040
Touched on previously, but outlined more clearly in this section, is the notion of serving a user product information they are likely interested in when their location within the store is calculated to make that a likely purchase consideration.
Section 0043
Everything discussed so far may be done without the explicit request of the user.
Section 0046
This is where it gets a bit creepy (if you didn’t think it was already).
The system is described to use AI, machine learning, and natural language processing systems to learn and model the behavior of users.
This extends beyond the one being tracked in the example to others with similar behavior and characteristics, to serve relevant information.
Basically, they’re tracking everyone and will model around not just the signals you have sent but what the model of others makes likely based on common contexts.
Section 0048
And if that wasn’t creepy enough, in my printed copy of this patent, this section literally has a “WTF!?” in its margin.
A system is outlined that would create autonomous search queries (the non-explicit ones noted above) based on the user’s calendar, emails, texts, social media, instant chat, purchase histories, reminder lists, and (this is the one that really inspired the margin note) “communication information associated with a family member.”
The example they give is of knowing a user purchased a lamp on one visit and, on noticing them in the lightbulb section of the store, generating a notification related to the specific bulb the user may be looking for.
That seems inert to be sure, but that list of data points the patent discusses is (in my mind) more than a little disturbing.
Section 0055
We see the specific calculation discussed that triggers the connection to remote chat support.
The patent image used to illustrate this feature is:
Section 0056
We see outlined the specific scenario where an in store sales associate is dispatched after the second tier was unable to assist.
The second tier escalation is illustrated by the image:
Section 0057
Here we get the core ideology of the system. And it’s spot on.
As outlined in the patent, the purpose of the entire system is to solve the problem:
“… the customer, while located in the physical store, may simply turn to his or her mobile device to manually execute a search (e.g., at an online retail website) for any additional information that the customer may need before completing a purchase in the store. With the option to purchase almost anything online, and whether it be due to time wasted searching for products on shelves, searching for sales associates, or searching for additional information needed to make an informed purchase of a product while in the store, some retailers may find that their customers are becoming less interested, if not increasingly dissatisfied, in buying goods from their physical stores.”
Basically, the system is developed to stop a user from going online where they may complete the purchase.
One can’t argue the issue they are trying to solve for brick-and-mortar stores.
Section 0069
We see a dangerous possibility arise that has potentially massive ramifications for users in the privacy realm. I’m not just referring to theoretical privacy, but practical, daily issues with those around us.
There’s an example of family members’ patterns used to generate suggestions for others.
The example they give is of an electric toothbrush being purchased by one family member and when a different member of the same household is in that section of the store, replacement heads are displayed.
Sounds pretty useful, doesn’t it?
Unfortunately, it reminds me of a lot of the scenario Target created back in 2012. Target was using purchase data to profile their customers at key moments in time, when purchase behavior can be drastically changed.
There are a few such moments that impact brand loyalty and patterns more than others and Target wanted to figure out who fits these and get to them at that time. Pregnancy is one of those moments.
From her purchase patterns, Target determined that a young woman was pregnant. They started mailing her coupons related to pregnancy and babies.
Well … her father wasn’t pleased with Target for promoting having a baby, directed at his teenage daughter. He later apologized when it turned out, they knew she was pregnant before she had told her family.
Can you see the potential problems with the system Google is outlining here now?
Section 0071
Heading back to the more positive applications, a system is described that offers suggestions to a user based on the queries (presumably including non-explicit) and data collected by other users at the same location.
I can’t help but think that, when combined with purchase and review data, this could be extremely helpful for restaurants.
Section 0087/0088
These sections combined describe the processes of the system to determine that the user of a device is near several products that they may be interested in and jumping right to the third tier outlined above and dispatching a sales associate.
I would assume that in practice, the store itself would prioritize which products and product categories would warrant this level of attention.
Section 0115
This section describes a system that, based on the aisles a user has visited, will indicate high profit users and escalate them to the second and third tier notifications.
One thing that intrigues me about this stage of the process is how it could be used in conjunction with machine learning systems to monitor store visitors for patterns like those shoplifters exhibit and trigger a sales associate to engage the user or cameras to monitor them.
Section 0116
Here, the system adds the used of RFID readers or similar mechanism into carts to monitor products with related add-ons and presenting notifications on those products.
An example would be a new TV in a cart being met with an add for new, high end cables.
Why Was This Patent Filed?
OK, now that we know what’s in the patent, what does it potentially all mean?
Clearly, the patent addresses a problem for brick-and-mortar stores. That’s great.
But implementing a system like this would have solid, real-world costs for both the store owner and Google. It’s most likely to be implemented by large chains over mom and pops due to this.
It would clearly help Google provide better data to its advertisers and monetize the promotional aspects, but I honestly think the goal on their end will be to break even.
That is to say, Google doesn’t need to make money on this.
Let’s look at a scenario you yourself have likely found yourself in.
You’re in a store and you see a product you’re interested in. Maybe it’s a new laptop. Maybe you want to find some reviews of it or more information on the specs than you can read on the box.
You pick up your phone and enter the model number and the word “reviews” or “specs”. You’ll likely see reviews right on the page, provided by Google so you don’t have to scroll.
If you click through to any of the additional organic results, the carousel of info above the reviews or the organic results below, you’re suddenly on pages like PC Mag’s review of the device or worse, a page reviewing multiple laptops and adding decision and potentially confusion.
But let’s say you just end up on PC Mag’s review. Interesting – they have a link to buy the product online … from Amazon.
They also have Walmart links but it’s Amazon I’m certain is the focus here. Many review sites link to Amazon through their affiliate programs.
So not only is the brick-and-mortar losing a sale, it gets worse for Google, Amazon is getting it.
Amazon is making huge gains year over year and challenging Google for ad revenue.
Hurt Amazon and you win.
And that’s why Google can simply break even and I would dare say, take small hits in the deployment of these types of systems.
Google’s goal would not be to make money but rather to keep the money they’re making.
They are predicted to lose about 2.5% market share over the next two years. That’s many billions of dollars per year.
If they can keep Amazon from getting a sale that would have gone to a brick-and-mortar store, even if that doesn’t produce revenue for Google, it takes it away from Amazon.
And that is a win worth its weight in dollars.
More Resources:
Image Credits
Featured Image: Created by author, November 2019
All screenshots taken by author, November 2019