Quest for 'reality'
Is development research best served by providing realistic stimulus material and environments? Graham Booth argues that the pursuit of ‘reality’ in this context can, in fact, be positively harmful.
Do you believe that what you do reflects reality? Do you feel that the research you conduct is a fair representation of things in the real world?
I dont. And it doesnt worry me one little bit.
Is this is an irresponsible position to adopt? How does it help an industry that is striving to achieve the credibility that can get it into the boardroom and have John Humphrys take it seriously over breakfast? Surely its madness to be saying: reality doesnt matter.
First of all, let me define my terms. Im not referring to exploratory studies, where going into the real world environment can be helpful or even essential. I am thinking about development research, where the call to make qualitative research closer to reality is becoming increasingly common.
There seem to be two main ways in which people want to see more reality in development research. They either want to have stimulus material more highly finished so that it better represents what the finished XYZ would be like in reality, or to place the material in a realistic context, so that the consumer can encounter and appraise it in a way that is closer to reality.
In most cases, this desire is misguided. Lets take the use of highly finished stimulus material. Finished executions always move on considerably from the creative ideas put into research, meaning that stimulus material can never represent reality, unless it is the finished product. Furthermore, researching more finished material runs the risk of giving erroneous results.
The more finished the stimulus looks, the more consumers take it as finished material, responding to the executional elements rather than the idea. Key frames for TV ideas are usually terrible and animatics still worse, the introduction of movement furthering the misleading impression of reality. This is why, in cases where it is appropriate to the idea, my company avoids using key frames for TV advertising development and instead uses scripts or narratives.
Contrary to the gathering orthodoxy, our experience is that most consumers are adept at imagining scenarios from vividly written descriptions, and this helps creative ideas live in research.
The best approach is to accept that stimulus material cannot represent reality and concentrate on trying to represent the idea clearly, rather than the reality of its finished expression. After all, in development research we are not exploring the finished thing — we are researching the idea.
What about the desire to put stimulus material in a real context? I am sure that all researchers can give innumerable examples or this, but take the suggestion that pack concepts be researched in mocked up displays. Why do this? Is it to assess stand out?
Quite apart from the fact that you cant assess impact qualitatively, such an approach takes no account of how the consumer browses the fixture in situ, which can have a major impact on response. Added to this, and just as important, the pack will never be displayed in this way in the real world anyway, so why try to pretend that it will?
In the past we have been asked to research beer advertising concepts in a pub because it will get respondents in the right frame of mind. Perhaps this would be fair enough if they only saw TV or print advertising when in the pub, but since when has this been the case?
Not only is the pursuit of reality in development research largely a pointless endeavour, it can be positively harmful. Where it is done, it leads to an erroneous sense that the results are a better reflection of reality, which encourages people to treat the results as more reliable and definitive.
What is driving this call for more reality in development research? I believe that it stems from a desire to make qualitative research more predictive. Lets be clear about this: qualitative research is fundamentally non-predictive, and any attempt to construct it in a way that suggests it might be needs to be resisted.
So, if reality doesnt matter, what on earth are we doing in qualitative development research?
Qualitative development research exists to help us think. The purpose is to experience the people who make choices within the relevant market processing brand and communication ideas within their own frames of reference.
This touches on the point where I think reality is vital, and thats in recruitment. Our respondents must genuinely be the right people, because the whole point is to get the perspective of people who live in the market rather than speculate about it.
What people tell us (verbally, through body language or through enabling techniques) helps us think about how the ideas might be working and to develop hypotheses that we can test further. In this way, it helps us work out how the ideas function.
The brand team can see things they hadnt before, recognise new opportunities, reappraise their instincts or understand why they felt something intuitively, thus enabling them to work with it more effectively. The team can then enhance ideas, take them in new directions, retrace their steps to find new departure points and, most importantly, find ways to make ideas better.
Of course, this thinking has to be applied in the real world. But the understanding of the real world will typically be derived from other sources. Research is one source, but typically research specifically designed to understand consumer behaviour in reality, for example: factors influencing brand choice at POP, ways in which TV is consumed in home.
Clients should be using this knowledge, plus their own experience of how consumers operate in their markets and their broader business understanding, as a context within which to think about whats coming out of the development research.
Our research can only provide clues as to the possible answer, which can only be arrived at by consideration of many other pieces of information — its one piece of a jigsaw, not the whole picture.
So, if you are involved in carrying out qualitative research to help clients develop new ideas for brands and communication, I would urge you to challenge the call for something closer to reality. Point out the fallacies and dangers of attempting to build more reality into the process.
Defend the intellectual integrity of what you do and the commercial value of ideas and thought - which, incidentally, are becoming increasingly important as drivers of competitive advantage in the modern economy.
Be humble, too, in acknowledging the fact that ours is only one part of the jigsaw that the brand team has to assemble. You might find some clients dont like to hear it but, ultimately, you may be surprised to find how such an approach builds your value and earns respect.
Managing Partner, Movement
This article was first published in InBrief magazine, February 2004
Copyright © Association for Qualitative Research, 2004