I was reminded of these stats when an excellent young researcher told me he couldn’t imagine still having a job running focus groups in 10 years’ time. My instinct was that he was wrong — but I couldn’t articulate why. This piece tries to work it out.

Here are five reasons for confidence

#1 Prediction isn’t the only game in town

The ‘Behavioural Economics Revolution’ in research, for which I must bear some responsibility, has been grasped by naysayers to condemn qual. BE shows that people are unreliable at predicting their own behaviour, or stewarding their own health, wealth and happiness. QED, it is pointless to ask them what they might do. Typically, such naysayers then advocate some indirect method (i.e. ‘neuro’ techniques) or tech driven solution (e.g. Big Data). Qual, after all, is the paradigm of Small Data.

They are wrong in the conclusions they extrapolate. True, people are poor at predicting their own behaviour. This, however, does not imply that people can never do it. People get it right sometimes. It is foolish to take people’s own predictions as the gospel truth. It is not foolish to ask them and weigh the answer with caution. What is more, a BE framework provides the most robust framework for systematising that caution. More tellingly, it exposes the poverty of the question “would you do this?”

The most important question isn’t what people will do. It’s understanding how they decide so you can change what they do — and create a behaviour that may not have existed at all before. Qual remains perfect for exploring these triggers and barriers to unknown possibilities.

#2 A technique with hidden shallows

Qual owes its history to depth methodologies. From Freud onwards, motivation is taken to be ‘deep’ inside. The most telling critic of this view is Nick Chater, author of The Mind is Flat. Chater asserts we have no mental depth. Instead, we improvise our actions and reasons on-the-fly from surprisingly meagre resources.

At first blush, Chater’s view would seem devastating for qual. Again, it is, in fact, liberating. Qual is the perfect place to observe this process of improvisation. Ask a question and watch people assemble an answer. Ask another and see what changes. What is constant? What is new? What are the building blocks of these responses? If everything is on the surface, where better to see it than face-to-face?

The focus group is like a petri dish in which these improvised responses can be cultured and harvested rapidly for analysis. Once you start to see the tools people are using to improvise their answers you start to understand how to effect change.

#3 Small Data beats Big Data

Big Data is another stick used to beat the irredeemably small world of qual. The world of the web is surely just one vast write-in. For all intents and purposes it is the modern world. All answers must be there.

My esteemed colleague Rory Sutherland critiques Big Data exquisitely. Yes, Big Data (and its abuses) won Trump the Presidency. It also misguided Clinton into her loss. While Bill Clinton (his instincts honed by two victorious elections) advised campaigning in crucial swing states, the data said otherwise. As Rory notes, “Iceberg ahead” is only one data point — but it may be the one you need.

Good account planners and successful qual researchers have a talent for pointing out the iceberg.

#4 What about AI?

‘(Wo)man vs. Machine: from Competition to Collaboration’ a paper celebrated at ESOMAR (and elsewhere) makes the case for using AI in qual. As the title suggests, it does not predict AI will eliminate qual. The AI harvesting of qual, however, is limited in two ways. First, there is a lot more to qual than the words spoken. While the words remain the core of what AI analyses it will miss things. After all, qual is sometimes all about what people don’t say.

Second, the human response (your response) to a respondent is extraordinarily rich. This ‘sense’ of the room (what therapists call ‘transference’) should be cultivated in any good research and exploited in any analysis. AI isn’t there yet. People still read people in a way machines struggle to do.

#5 Telling a good story

Finally, we all know that research isn’t about the facts: it’s about the way the facts are made actionable and useful. Qual is still the best way to write a story about how people live now, and how they might live in the future. Qual researchers are storytellers, the people who make sense of the swirl of information. This should give us cause for hope because it puts us in the company of ‘Writers and authors’ Their chance of losing their job to a machine? 3.8%.