How to measure customer experience

As competition gets ever hotter and consumers have more channels to express their likes and dislikes, the pressure on companies to know customers as individuals and measure their experience mounts. And as young people in Gen Z take to communicating by video, Snapchat and Live Messenger, the number of channels is already increasing.

3.2bnTraditional methods of measuring customer experience such as net promoter score (NPS) are still popular, and asking one question gets to customers quickly and allows the questioner to react fast.

However, it does not give detail, as Trainline, an early adopter of NPS, discovered when it used customer effort score (CES) to pinpoint what was not working. Refunds, which took 28 days, were definitely not working. It took one year to fix, but the company’s CES and NPS improved dramatically. And customers can now click on a calendar to learn when their refund is due.

“Companies do NPS to have their finger on the pulse, to react and to close the loop, but they still do customer satisfaction surveys to dig deeper to find out if is there something systemic wrong that they need to fix or improve,” says Gartner analyst Ed Thompson.

Transparency in the years ahead

A decade or more ago, customers were loyal because they had no choice. These days, not only do they have a lot of choice, but they can make decisions fast and in no time be buying someone else’s products or services.

“The future in this area is going to be transparency,” says vice president, customer experience and analytics at Capgemini, Gagandeep Gadri.

The new rules of engagement, when it comes to creating a notable experience, are empathy, serendipity, privacy, reciprocity and adaptability

“Customers can see feedback other customers make on forums and blogs, but organisations might share some of the other details they measure through buttons in store, e-mails, net promoter scores of mystery shoppers. This would complement the overall rating and balance the feedback.

“No one is doing it yet, but to differentiate themselves in the new world, organisations are going to have to stick their neck out beyond value, price and service to give a more rounded view of how they are doing customer experience.”

And the field is about to become even more of a science, according to research undertaken by Adobe and Goldsmiths, University of London, entitled The Future of Experience, which concluded that the new rules of engagement, when it comes to creating a notable experience, are empathy, serendipity, privacy, reciprocity and adaptability.

Delivering an experience

“Great marketing is now all about delivering experience; a brand is little more than the experience its customers enjoy or endure,” says Europe, Middle East and Africa (EMEA) marketing director for Adobe, John Watton.

“There is raw data, often using technology to get the right people, at the right time, on the right channel, but there is the emotional side, the connection and relationship you build, with technology giving you the chance to have a more humanised experience.”

For example, artificial intelligence will allow conversations with customers to change tone or personality because someone sounds stressed. Although this is still in its infancy, it gives the possibility of continuous improvement. Instead of tracking data to see how an organisation did, it will be able to say “every day we get better” by integrating its systems to try to learn and understand more about the success or failure of customer engagements and experiences.

how businesses collect feedback

“It will adapt in real time and over a longer tail to reflect that,” says director of innovation and senior lecturer in information management systems at Goldsmiths, Dr Chris Brauer.

And empathy is all. “We looked at it in the sense of being able to walk in someone else’s shoes and empathy is the foundation for trust,” he says. The other four dimensions of the report all lead back to empathy.

Everyman Cinemas uses NPS, but focuses on it less than talking to customers while they are on the premises, which total 17 venues and growing, and monitoring social media, particularly Facebook, Twitter and TripAdvisor.

“We respond on the hoof, rather than in a formulaic fashion and that is divvied out across the marketing team, so it is fresh,” says chief executive Crispin Lilly. “Our customers like to feel they have a relationship with us and my challenge is how to maintain that as we grow.”

Everyman’s response to feedback is quick, as its conversion of four Odeon cinemas, acquired this year, showed. The company’s cinemas are furnished with sofas and chairs, but the Esher property had sofas only, which sometimes meant people had to share a sofa with a stranger.

“We started getting feedback very quickly and over the first couple of weeks it became clear we had made a big mistake in the seating configuration. It took a couple of months to fix it,” says Mr Lilly. Everyman’s relationship with customers pays off and where people on average visit a multiplex two or three times a year, its customers visit five or six times.

Although knowing customers as individuals is a must, so is seeing the bigger picture. “Organisations need to start migrating towards employing an overarching algorithm to analyse the customer journey, including customer metadata, key buyer behaviour, purchase history, previous survey scores and customer journeys,” says director of customer analytics, EMEA, at Verint, Rachel Lane. “For this to be possible, customer data must be available in one central ‘big customer’ data hub. This is a long way off.”

Technology company Dynatrace does exactly that. It collects data from 7,500 websites worldwide, from Fortune 500 companies to startups, and stores the information in its data warehouse. This measures live both employees’ and users’ experiences of the sites, including availability, performance and customer behaviour in the light of a bad experience, how long they remain on-site and what they do immediately afterwards.

AB testing

Many businesses want to deliver a more immersive experience, and one way of doing and measuring this is AB testing. This allows companies to try out different colours, banners, workflows and design with a selection of customers to see if they buy more, stay longer, find what they want faster and come back more often.

“Small, incremental changes can make small changes to customer behaviour,” says vice president of solutions, EMEA, at Dynatrace, Michael Allen. “That is where having real-time visibility of users is critical to be a leader. Amazon is doing it all the time; on average, they make changes to production every 11.6 seconds.”

The leisure sector thrives on feedback, but because spending in those areas is discretionary, it is subject to the vagaries of confidence. Driving factors are an ageing and well-off population, and changes to working hours due to automation giving people more leisure time.

“The leisure industry has never been more focused on what it can do to build the right model for the consumer,” says head of hospitality and leisure for Barclays, Mike Saul.

“Traditional feedback is after the event and not necessarily used as a dynamic way to build out what clients might want in the future, but clients can use digital and real-time feedback to have a finger on the pulse and know what clients want,” he says.

“A top tip in [the Barclays report] The Feedback Economy is that companies have to monitor feedback and make it appropriate – a feedback request for an 18 year old and a 65 year old have to be very different,” says Mr Saul.

Banks are an excellent example of how the bank manager’s personal one-to-one relationship gave them all they needed to have a positive relationship with customers, but that got lost in the race for productivity. RBS is taking advantage of technology to turn it around.

“Insight is delivered at the point of interaction, at scale,” says leader in next best action and contextual marketing at Pegasystems, Robin Collyer. “It provides real-time measurement, adaptive models and measuring through various routes, and is continually updating and therefore into real-time analytics.

“RBS staff have been given the insight they need, so they are able to suggest the next best action to customers, who are delighted,” he says. “It is highly effective and one of the key metrics of customer satisfaction is retention.”

Measurement of customer experience by metrics has not died a death nor is it about to, but combined with feedback on social media, AB testing and other digital ruses, companies can get comprehensive real-time insight into customers’ likes and dislikes. As long as their response is equally agile, they can ensure they retain and gain customers – and that is reflected in the bottom line.



Harrah’s casinos built a data warehouse of customer information, but knew it could only see a fraction of its consumer base, such as those with loyalty cards, so it banned cash.

Before gambling begins, customers hand over cash and receive a card loaded with gambling chips, so every time a customer places a bet, plays a fruit machine and so on, Harrah’s gets a full picture of their gambling. Money from cashing in the chips also goes on to the card.

In 2010, Harrah’s worked out the burn rate of gamblers to optimise their lifetime value. The average gambler is a 35-year-old woman from Virginia who takes a long weekend in Las Vegas once a year with friends as a break from her children to see a show, gamble and have some fun. Harrah’s worked out that this type of customer might be willing to lose $500 while gambling, but if she lost a year’s housekeeping, she would be unlikely to ever return.

The company introduced Lucky Ambassadors, ladies on roller blades, who roll up to customers who are losing money beyond their perceived threshold, sympathise with the terrible luck they are having, and offer them free food and drink from trays they are carrying.

This takes them away from the table or slot machine and stops them gambling, but it also ensures they come back next year – lifetime value maximised.