The UK is the most researched nation on Earth. Some £61 per head is spent each year on surveys, polls, panels and questionnaires, more per capita than any other country, according to the Market Research Society (MRS). No wonder many Brits are suffering from survey fatigue and refuse to take part in questionnaires. This reluctance to participate is believed to have contributed to incorrect polling predictions in several elections.
The UK’s top 100 market research agencies made annual revenues of almost £3 billion in 2015, says MRS. But growth is becoming hard to find in the industry as big businesses cut their budgets. Meanwhile, a range of startup research agencies and new players offering digital methods are looking for a piece of the action.
Martin Hanscombe, a director at Kantar Worldpanel UK, says big data can offer businesses strong insights into the behaviour of consumers, but is not a direct replacement for established research methods. Kantar runs a panel of 30,000 UK shoppers, who record the shopping trips they make and the brands they buy. This data is collated to tell brands and retailers about the performance of their products. The advantage of such panels is they offer a regular and consistent analysis of behaviour, which can be compared over months and years.
But Mr Hanscombe believes digital research, such as online questionnaires and analysis of consumers’ online behaviour, is becoming more attractive to clients. “Among clients, there is an appetite for nimbleness and direct dialogue with consumers. Digital offers a direct connection, but what are sacrificed are the solidity, rigour and foundation of reliable behavioural evidence and solid comparisons versus the past,” he says.
Measuring success
A significant trend among brands over the past decade, says Mr Hanscombe, has been a shift away from prioritising market share – the amount spent on their product compared with competitors – as a measure of success. Instead, brands are putting increased focus on market penetration – the percentage of the population that buys their products.
This comes after academic analysis of shoppers’ panels showed traditional assumptions about sales growth were wrong. It had long been assumed that once consumers started buying a product they would become loyal, lifetime consumers. “It doesn’t work like that,” says Mr Hanscombe. “Even the strongest brands experience churn in people coming in and out of the brand. People shop around, so the challenge for brand owners is bringing in more people.” Brand penetration has become the key measure of success, rather than the share of market revenue, he says.
Even so, market share still maintains a grip on the expectations of many companies. The notion of market share was invented in the 1930s by AC Nielsen, the first company to offer market research and now one of the largest market research businesses in the world.
Shoppers’ panels run by companies such as Kantar and Nielsen are holding up strongly against competition from digital entrants and big data analytics. “There’s quite a lot of habit in market research about what techniques are used,” says Simon Hay, former chief executive of dunnhumby, the data company which runs Tesco Clubcard.
This, he believes, acts as a break on the adoption of new forms of research. “Once you become the lingua franca of the industry, that gives you some degree of protection from the forces and winds of change big data is throwing at the market,” says Mr Hay. While the “longitudinal” nature of panel data gives it comparability and stability, he believes the sheer volumes of data and huge sample sizes offered by big data give powerful insights into shoppers’ behaviour.
Digital disruption
An area of market research where there has been significant disruption is in the measurement of media audiences. This was once involved monitoring TV, radio and print audiences on behalf of advertisers. But with the growth in digital and mobile consumption, and an explosion of devices for consuming content, this has become far more complex. Google and Facebook account for a huge share of time spent on media, so their analysis is vital.
Brand penetration has become the key measure of success, rather than the share of market revenue
However, there has been controversy over their measurement methods, with critics accusing them of “marking their own homework” by creating their own measurements. But this is now changing and there are moves to introduce independent third-party verification of the internet giants. Even so, says Hannu Verkasalo, founder of Verto Analytics: “As long as Google and Facebook are dominating, it is very difficult for other measurement companies to challenge those tools. As long as they keep this for free, it is really destroying many businesses.”
For James Oates, marketing effectiveness director at Nielsen UK and Ireland, market research is facing an ever-expanding set of challenges as consumer behaviour changes. “There’s an evolving profile of what is going on around the consumer that makes the industry challenging and interesting,” he says.
Market research has faced some reputational challenges over the past 20 years, mainly because of its uses in politics. Richard Huntington, chairman and chief strategy officer at Saatchi & Saatchi London, says qualitative research, such as face-to-face discussions, has been damaged by the American approach of focus groups. “Qualitative research becomes bound up in the mire of spin; it is devalued and debased and seen as a means of making things blander, but working out how to manipulate people,” he says.
Meanwhile, quantitative research, based on surveys and polls, has suffered from making wrong forecasts in the 2015 general election. “We are seeing a return to old-fashioned qualitative research, with open-ended discussion of topics to yield better insight rather than the focus-group tradition of pushing a particular point of view,” says Mr Huntington.
And what of the future?
A great hope is machine-learning technology, which will analyse billions of pieces of data to make accurate predictions about expected behaviour. So perhaps market research hasn’t reached its peak quite yet.
