The simple process of asking people questions has been essential to business strategy for more than a century. What’s your favourite cereal? If money were no object, which car would you buy? Coke or Pepsi?
But new technology is presenting businesses with a huge range of alternative ways of generating consumer insight and total investment in survey research is declining.
One of the biggest flaws with survey research is that sometimes people give the wrong answers. They might be embarrassed about which snacks they give their kids or how much they drink. Or they may honestly misreport what they think they do.
Another problem is the people who agree to answer questions. The research giant Kantar, which has been experimenting with mini-surveys to encourage take-up, puts it succinctly: “It’s no surprise that consumers don’t want to spend 30 minutes completing lengthy surveys any more. And the people who do have that kind of time to spare probably aren’t the ones you’re really looking for.”
Phil Garland, a former executive with Survey Monkey, which along with Google Consumer Surveys has helped bring research within reach of much smaller businesses by making it cheaper and faster, is not hopeful that surveys as we know them will be here much longer.
“All parts of the survey world are in danger. The reality is it’s become a relic of the past for individuals to use surveys as a way to make their voices heard,” he says. “There are ‘likes’, forums, comment areas, tweets and user feedback forms to communicate directly with a company or influence their products without having to participate in surveys. The world has changed.”
For simple questions about whether people prefer this packaging or that one, or doing advertising copy testing, a cheap and cheerful survey is fine, Mr Garland says, but the pressure on the research industry to keep cutting prices means quality – and the usefulness of survey results – is threatened.
He points to high-profile failures to call elections correctly as evidence of the problems with surveys and the people whose opinions they rely on. This year’s general election, predicted by all the opinion polls as too close to call, was won by a sizeable Conservative majority that all, except the exit polls, failed to predict. The industry is currently conducting a post mortem to determine what went wrong; early evidence suggests not one single mistake, but several causes, which together led to the Labour vote being overstated.
Ben Page, Ipsos Mori’s chief executive, says these errors have little bearing on surveys conducted for businesses, which are better funded, less hurried, draw on larger samples and are interpreted by people who understand the strengths and limitations of the research they’ve commissioned.
While there’s a great deal of excitement about new approaches, businesses are not abandoning surveys
“We know that people aren’t particularly brilliant at predicting their own behaviour,” he says. The error in the predicted Labour vote, while significant to an election result, would not have a bearing on business decisions. “For many commercial decisions, if it’s that close, it’s a moot point anyway. You want to see the brown flavour is clearly ahead of the red flavour, so this level of accuracy in some ways isn’t even expected,” Mr Page says.
So surveys aren’t perfect, but what can you do? Plenty. Eye-tracking can determine which parts of a shop shelf or ad people look at. Data analysis can link location with web-browsing. Neuroscience aims to link brain activity detected by scanners with emotional response to stimuli such as logos and advertising. Wearables track consumer activity and photograph what people are looking at, focus groups gauge consumer sentiment, and researchers use ethnography – watching consumers for hours or even days as they go about their lives – to observe the detail that can help inspire new products.
One of the many new research providers on the market is ResearchBods, which uses software to track mobile use passively, and can link with diary studies that capture mood at key moments. “Behaviour-tracking is much more accurate than asking people to recall events and provides a wealth of data for researchers, as well as a means by which they can go back to consumers and ask them about specific activities,” says director Robin Hilton.
Yet while there’s a great deal of excitement about new approaches, businesses are not abandoning surveys. Almost half of all market research carried out is still survey-based, according to global research industry body ESOMAR.
Rhea Fox, head of research at eBay UK, says that between a quarter and one third of her research budget goes on surveys. “We obviously have access to a huge amount of user data, but surveys critically help us to understand the ‘why’ as well as the ‘what’,” she says.
Alison York, research director at Nickelodeon UK Research & Insights, is actually using surveys more, not less, from year to year. Surveys are used to understand media consumption, obtain feedback on new TV shows and evaluate marketing campaigns. She says: “Research really is at the centre of what we do – it informs all elements of our business – and surveys are an important part of the puzzle to provide an understanding into kids.”
At the adventure park chain GoApe, surveys of visitors done by e-mail have led the company to make specific changes to improve the customer experience. Water stations were installed, certificates developed to help visitors celebrate their achievements and park signage was improved.
The impression many people have of surveys being taken on a clipboard in a shopping mall is far from the reality of modern survey research. Surveys are increasingly being done online, are optimised for mobile, with fewer questions, and some use images or gamification to make them more engaging. Others aim to strike consumers when they’re ‘in the moment’, with a couple of questions just after making a purchase, for instance, rather than asking them to recall their feelings later.
Ipsos Mori’s Mr Page says the toolkit for business research is much more varied than it used to be with observational data, passive data and web activity. “Link that to things we can ask people about and, rather than just ask them, we can measure their galvanic response or pulse rates, we can look at how quickly they click on things to see if they’re thinking long and hard about it or is it just an implicit response?” he says.
“What’s interesting is that nothing is exterminating everything else. There’s still huge demand for face-to-face interviews, but we can get people to wear Google Glass and GoPro cameras, and film their everyday life and have that streamed back to us. We have far more channels to understand human behaviour and what people are thinking.”
Marie Wallace, global analytics strategist at IBM Ireland, is a huge advocate of the power of data analytics to generate business insight, but says it can never tell the whole story. “We keep thinking that now we have social data we believe we can analyse… but very often the data doesn’t find you the complete answer,” she says. “You have to reach out to people at times, you have to ask questions and you have to get some feedback. That isn’t ever going to go away.”
eBay uses surveys alongside focus groups, ethnography and co-creation workshops to get a rounded view of its customers. At Nickelodeon, they use smartphone media diaries, semiotics and, most recently, facial-coding, to measure kids’ reactions. “Surveys tend to give us the robustness we need to validate our qual [qualitative] findings. This also provides a more rounded picture of what is happening,” Ms York says.
ResearchBods’ Mr Hilton sees a role for surveys alongside hi-tech passive measurement. “Gone will be the long surveys of 15 minutes or more,” he concludes. “Surveys will cement themselves in gathering the attitudes and opinions of consumers around their behaviours in real time, helping to provide a much more complete understanding of consumers.”
FIVE TOP TIPS TO MASTER MARKET RESEARCH
1. Measure something meaningful, not just something you’ve measured for years and feel you need to keep doing. Trackers have value, but don’t always provide actionable insight and can instead lead to a focus on past performance at the expense of current and emerging trends.
2. Ask yourself whether you are doing research to discover something new or whether you are looking for research to validate decisions you have already made.
3. If you’re asking people questions, don’t waste their time with things that are “nice to have” but are not essential to the business question of the moment. Does the number of children they have really affect their perfume preference?
4. Don’t buy research based on price alone. A good research provider should be able to answer questions about how they recruit respondents, how they ensure panel quality, and discuss drop-out and disqualification rates.
5. Ensure that research results are presented in a way that easily communicates the key points and is accessible to people throughout an organisation. This also helps reduce the risk of duplication.
