Just
a few weeks ago, Apple announced with great fanfare its new credit card. Behind
the scenes was JP Morgan Chase, entering the credit card market for the first
time. Very soon it emerged that men were receiving 10 to 20 times more in
credit lines than their wives who were giving identical information to the bank
in terms of their financial status. The only difference in their applications
was their gender. Bank officials were at a loss to explain the discrepancies
and blamed the algorithm used to determine credit-worthiness.
Therein
lay the problem. The algorithm had somehow incorporated biases that favored
males as credit risks over females. Machine Learning then very likely had built
on the algorithm and magnified the bias.
The
fact is that algorithms are written by humans and whatever biases these humans
may have (conscious or unconscious) can very easily end up being incorporated
into the algorithms themselves. And, in an age where we seem all too willing to
trust the black boxes of Machine Learning, such biases become embedded,
exaggerated and, in some cases, permanent. This is bad enough when the bias in
question is unconscious, but there have been cases recently where data
scientists have proudly boasted of having deliberately introduced biases into
their algorithms because this “reflects more accurately the world we live in”.
Such an argument is not only specious, it is downright dangerous. Imagine that
AI and ML had existed in the age when eugenics was all the rage and was used to
explain away racism. Had those biases (or beliefs) been incorporated into the
algorithms of the time, it is entirely possible that the US Civil Rights movement
would never have happened.
It’s up to us
If
we recognize that biases are inherent in algorithms, what can we do to mitigate
the danger? First off, we need to elevate the conversation about bias, as well
as about integrity and ethics in all that we as insights professionals do. The
world’s major trade and professional insights associations are all putting a
major effort into doing this, raising the subject not only within the industry itself
but also with governments and academia. But the conversation needs to occur
earlier in the process of producing insights professionals and data scientists.
Awareness of the issue of bias needs to be introduced at all levels of tertiary
education and across a much broader swathe of subjects than merely business
analytics or market research.
While
we are on the subject of market research, let us also acknowledge not only that
bias exists at many levels within the research process but that, in many instances,
this inconvenient truth has been forgotten or is willfully ignored. While
seasoned researchers (especially in the political and social fields) are fully
aware of a myriad of biases – explicit, implicit, mode, question order,
question wording (you name it) – and work hard to mitigate it, many others may
only be dimly aware of what they are or even that they exist.
In
an age of convenience samples, digital sample sourcing, automated scripting and
DIY platforms, discussion of bias has all but fallen away. Yes, those who have
earned degrees in MR or who have taken training courses are very likely to be
aware, and may even diligently apply mitigation, but they are a small minority
of all of those who are now practicing some form of primary research. Which
means that the good old maxim “garbage in, garbage out” is more relevant than
ever, and that there is an omnipresent danger of decisions being taken on the
basis of bad data. For this to be the case at all – and for the industry not to
deal with it in an age that combines data overload with data illiteracy – would
not only be unfortunate. It would be unethical.
A call to arms
It
is time for us all to have a very robust conversation about the issues of bias,
integrity and ethics in the insights and data sciences space. Convenience
should never prevail over data quality (and please don’t quote “directional” or
“good enough, move on” at me). However we derive our insights – whether through
primary research or data analytics or a combination of both – we need a
sustained campaign of education to introduce these concepts to a much larger
audience than is happening today. To my mind, all BBA and MBA students should
be taught the basics of bias and how to mitigate for it. Only in this way can
we ensure that we remain true to our roots – impacting business success,
guiding social policy and political decision-making, all based on data of the
highest quality and integrity.