The need to base business decisions on data not intuition

The universe is an odd place. The Casimir Effect is one of its most baffling wonders. Place two conducting plates in the vacuum of outer space and, when positioned closely in parallel, the plates pull together. The force isn’t gravity. Rather, the suction is caused by mysterious virtual particles appearing from the void, bouncing between the plates and vanishing. The Casimir Effect is proof that empty space is far from empty. It is teeming with virtual particles flashing in and out of existence.

The Casimir Effect isn’t the sort of thing you can figure out by intuition. No lay person could guess the universe possesses this property. It’s too radical. Most of what happens at the quantum level is. Hendrik Casimir published his theorem in 1948 based on the work of physicists such as Nils Bohr, who famously warned: “If you can fathom quantum mechanics without getting dizzy, you don’t get it.”

Cognitive bias

But it’s not just physics where common sense is flawed. It seems we are pretty poor at making even mundane judgments. In 1971 two academics, Daniel Kahneman and Amos Tversky, established the notion of cognitive biases. These are systematic flaws in our judgment.

Confirmation bias is the tendency to seek out information which agrees with our preferred position and to ignore contradictory information. Recency bias is the tendency to inflate the importance of the most recent piece of data. Football fans who celebrate each5.18bn estimated revenue generated by the UK market research industry in 2016 victory as though it heralds the dawn of a new era will be familiar with the concept.

Kahneman and Tversky discovered dozens of biases. The more the pair tested human intuition, the more flaws they unearthed. Nobel laureate Kahneman noted: “Even statisticians are not good statisticians.”

To survive in this perceptual warp takes a clever strategy. Since intuition is unreliable, evidence is essential. For businesses this means testing assumptions and going where the data leads.

Campaign success

Sport England offers a great example. It is under pressure to get ordinary Brits to be more active. Alas, by every measure women are more reluctant to play sport than men. But why? Maybe women don’t have the time, the money or access to the right facilities. Sport England conducted a survey to find out the truth.

Jennie Price, chief executive of Sport England, explains: “Before we began this campaign, we looked very carefully at what women were saying about why they felt sport and exercise were not for them. Some of the issues, like time and cost, were familiar, but one of the strongest themes was a fear of judgment.”

It was a revelation. Of the least active schoolgirls, 36 per cent said they feel like their body is on show during PE lessons. One in four women said they “hate the way I look when I exercise or play sport”, and 48 per cent said getting sweaty is not feminine.

Armed with these insights, Sport England worked with creative agency FCB Inferno to change perceptions. “Sweating like a pig, feeling like a fox” and “I kick balls, deal with it” were among the taglines. The ad images featured a woman putting in a gum shield, another gritting her teeth on a rowing machine.

The #ThisGirlCan videos went viral, viewed 37 million times in the first year. The press reaction was universally favourable. The Independent said: “Nothing should ever be about how to simply achieve a narrow beauty standard. But finally, it seems like more people in positions of influence are realising this.” Analysis suggests 2.8 million women took up exercise as a result of #ThisGirlCan, with 1.6 million doing it regularly.

Pitfalls of assumption

Without data we rely on guesswork. Another example: Accenture Strategy’s latest Global Consumer Pulse Research indicates that online retailers are being short-changed by an assumption about customers. Rachel Barton, managing director of advanced customer strategy at Accenture Strategy says: “Many organisations have an unconscious bias whereby they assume that their digital-savvy customers – born digital, stay digital – are the most profitable.”

Business leaders are realising they can make better decisions through using data and analytics more systematically

But it’s not true. Data shows that multichannel “experimental” customers are the most profitable. Ms Barton says the error leads to overinvestment in digital channels, costing UK businesses up to £221 billion.

There’s a long way to go. A report by PwC called Guts and Gigabytes shows when executives make major decisions, data and analytics ranks a lowly third (23 per cent) behind intuition and experience (41 per cent) and the experience of others (31 per cent).

Yann Bonduelle, PwC consulting data and analytics partner, says: “Business leaders have long used their own tried-and-trusted intuition alongside more scientific and financial factors to make decisions, and this has served them well in the past. As data become more pervasive, algorithms become more accurate and visualisation more intuitive, business leaders are realising they can make better decisions through using data and analytics more systematically.”

Intuition will always play a role. When Casimir proposed his theory, he had an existing body of knowledge and used it to develop a hypothesis, later proved by experimentation. Data alone cannot tell us which questions to ask or which ideas to test.

But it’s clear that intuition without evidence is flawed. As the great statistician W. Edwards Deming put it: “Without data you’re just another person with an opinion.”