I was delighted to be invited by Confirmit to a dinner held at the superb Kerridge’s Bar & Grill, together with a number of insight agency leaders. The aim of the evening, apart from having an extremely enjoyable time, was to discuss the future of the customer insight industry.
Specifically, Confirmit – as insight industry thought leaders – set up the event to ensure it stays in touch with the latest trends and developments. The aim is to reflect these learnings in the development of its products and services.
My role was to introduce some of the key opportunities and challenges facing the industry and provide a focal point for our discussions over dinner. I was ably supported by Jane Bainbridge, Head of Editorial at Impact and Research Live for the MRS, who did a fantastic job in steering the discussion over our superb four-course dinner.
We thought this blog would be helpful to share with a wider audience so that Confirmit can continue to receive feedback on trends affecting the industry. So below I am providing some of the themes around which our discussions took place.
Theme one: What role will customer insight professionals play in the new business and information era?
Recently I was privileged to be the project leader on an ESOMAR initiative that involved talking to top insight industry client team leaders about the role they saw for their insight teams in the future.
This highlighted the move away from customer insight professionals focusing primarily on risk reduction and highlighted the move towards identifying and seizing business opportunities.
As industry leaders, they saw their future as being in the vanguard of identifying strategic foresights: being the wide angle lens, those who could see the big picture in terms of customer trends and identify what this means for future growth and profitability.
Building on this theme I raised in the discussion the issue of whether we have the frameworks in place to make sense of today’s multiple sources of imperfect evidence. The issue is whether our traditional statistical concepts are now looking a little tired and whether we need to look beyond these to more holistic concepts and ideas.
Confirmit was generous enough to provide all those attending the event with a copy of my recent book, The High Performance Customer Insight Professional: How to make sense of the evidence, build the story and turn insights into action. (This is available on Amazon!) In this book, I talk about the idea of looking at our different sources of secondary, qualitative, quantitative and analytics data through the lens of evidence having Weight, Power and Direction.
Weight provides us with a way of looking at the relationship between the quantitative strength of support for a particular point of view in relation to the qualitative depth of feeling being expressed.
The Power of evidence relates to the way we now need to examine the robustness of evidence through the lens of (classic) methodological criteria, and also through the prism of what we know from practice about what works and what does not.
And Direction helps us understand the internal consistency of a particular category of data (for example a set of focus groups) in relation to the overall consistency of all categories of evidence – the secondary, qualitative, quantitative and data analytics sources.
Effectively this creates a fuzzy logic system. Put these ideas together and we arrive at the idea of evidence having high or low Strength. I also threw in the idea of whether we should now be describing the robustness of our evidence using the kinds of concepts lawyers use, such as a decision being safe.
So, the critical issue is whether we have the analytical frameworks in place to make sense of our multiple sources of evidence so that we can identify key strategic insights that will drive business growth.
Theme Two: The influence of new technology and the social media on customer decision-making
We raised the issue of whether or not the new technology and social media environment is beginning to fundamentally change the way that customers make decisions. We live in the world of constant scrolling through emails and information to quickly locate what we need and instantly decide what to do. We live in a swipe left swipe right decision culture. The question is whether this is simply refining how customers have always made decisions or creating a fundamentally different decision-making paradigm?
We, of course, have insights on many different dimensions of this issue. There are lessons from behavioral economics about cognitive biases. We understand the concepts of System 1 more intuitive, and System 2 more rational, decision-making. We know from Reversal Theory how individuals embedded in electronic communications may have more rapidly changing mood states than when engaged in face-to-face conversations. We have all of the expertise from neuroscientists telling us how screen time may affect memory and concentration. We are also learning about social media’s impact on bullying and trolling. And we certainly understand that In a straight choice between emotion and reason emotion usually wins.
So, we have a fair understanding of what is happening, but should customer insight professionals be doing more to showcase their expertise on exactly how the world of social media could be affecting customers’ decision-making.
The specific issue is whether we as an industry should be reflecting more on the social cognition process. We have long known that our attitudes are a function of our knowledge and beliefs, coupled with our perceptions of what other people think about our attitudes. Today this remains the case but, with social media, this social cognition process now involves many hundreds of people providing instant feedback in a very short space of time. Should the customer insight industry be offering more thought leadership around this phenomenon?
Theme Three: How will the arrival of artificial intelligence and AI assisted insights affect the insight industry?
Clearly, there are many exciting opportunities being opened up by AI, including the capability to analyze massive amounts of open-ended conversations and much more.
I recall reading Rom Harré’s Social Being: A theory for social psychology written in 1979, which prompted the idea that the ideal research study would include a record of every conversation any individual had with any other individual about the topic under investigation. All those years ago, this was simply an abstract theoretical idea but now we are close to achieving this!
But the arrival of AI opens up a number of challenges. Here, there is the issue of whether there is sufficient clarity around what constitutes an AI assisted insight. The start point here is to acknowledge that unfortunately there still remains a certain lack of clarity around what constitutes a conventional insight.
Some stakeholders still seem to feel that an insight is no more than simply a piece of interesting data, whereas most of us tend to work with the idea that an insight requires a psychological and behavioral dimension. The psychological dimension describes what is inside the customer’s mind – their causal motivations, attitudes, beliefs and feelings. The behavioral dimension describes what is observable – what the customer is or is not doing as a result of these causal psychological factors.
Arguably an insight needs to reveal a link between these two dimensions: causal psychological factors and behavior. An actionable insight is one that can create change or reinforce an intended customer attitude, belief or feeling. Powerful insights often reflect an unresolved tension in the mind of the customer and, as such, have the power and momentum, to drive action.
We also need to remember that powerful insights are not found, they are created through a strategic dialog – fierce conversation – between the insight professional, who knows about how the putative insight was generated, and key stakeholders who can assess the practical application of this insight. It is this strategic dialog that will help differentiate interesting observations from highly actionable insights capable of changing behavior.
Against this backdrop, with the arrival of so-called AI-assisted insights, we perhaps need some kind of taxonomy that takes us through the different ways that AI can generate insights. At one end of the spectrum, there will be simple pattern recognition albeit from large data sets. At the end, there will be insights evolving from some kind of machine learning – where we humans cannot easily understand what is going on within the black box.
This raised the issue of whether we need to start developing a checklist of questions – some kind of charter possibly – that we can ask of AI assisted/generated insights in order to test their robustness. This would also allow us to address different ethical issues, including how the AI insight was generated and also how it will be subsequently deployed. Again, a big issue and lots to discuss.
Theme Four: What will be the essence of the insight craft?
The discussion about the likely future role of AI raised the issue of what is going to be the fundamental skillset of the customer insight professional – what will be the defining essence of the customer insight industry.
It was noted that the industry was now often being described as having two distinct hemispheres: question based research and non-question based research. In addition, over the years, the industry has constantly debated whether it should focus on being more consultative in its approach or concentrate on being research experts. Some are likely to argue for showcasing our skills around clear deep thinking, problem solving, creativity, understanding people, answering business questions and essentially being evidence-based insight consultants.
But there is also the view that the core essence of what we will be doing will inevitably gravitate towards the data analytics end of the spectrum. Perhaps the sweet spot is being evidence-based experts up-to-date with the data analytics technology – but with that creative flair and spark around our unique understanding of what makes people tick and how best to solve stakeholders’ business challenges.
Theme Five: Who should we be recruiting into the customer insight industry and what should the training provision look like?
With the exciting times ahead and the opportunities to excel in an era of disruption, when it comes to our recruitment strategy for the industry, there is a strong case for recruiting individuals with a growth rather than fixed mindset. Those with a growth mindset are individuals able to inspire and help transform businesses. They will be comfortable seizing business opportunities in a timely way. Carol Dweck’s book Mindset: Changing the way you think to fulfil your potential is in my view required reading.
With regard to training, we are all clear that this now must be more learner centered, experiential, applied and practical – everything needs to be broken down into mind-sized chunks. And when it comes to teaching some of the more silky skills of being a sensemaker, storyteller and a consultant able to turn insights into action, there is a strong case for applying gamification – encouraging learning through fun, engaging games and activities.
Theme Six: What will the shape and structure of the industry look like?
Clearly the future for the insight industry is extremely positive. Organizations want to be customer-centric. They need insight professionals who can identify powerful strategic foresights and help seize winning business opportunities. However, there is the issue of whether or not the customer insight industry will simply eventually morph into a bigger data analytics function, with the customer insight role being a subset of this. The sweet spot would be to hold the industry frame together around the current insight-centric industry bodies but to recognize and embrace the fantastic contribution of data analytics specialists.
The good news is that customer insight’s ability to ask the critical why questions is going to remain crucial. In addition, insight professional’s ability to interrogate the veracity of AI generated insights will be in demand. Clearly it is important for industry bodies like the MRS and ESOMAR to continue to defend the faith and extol the virtues of integrity, transparency and accountability in evidence-based decision-making.
We look forward to hearing more from Confirmit as it continues with its thought leadership initiatives aimed at keeping Confirmit in touch with its customers, and also sharing its ideas and thinking with the industry.
David Smith
DVL Smith Ltd