Understanding big data and the cloud

Talking to a friend about this article, she asked me what “information of everything” meant. I explained that it encompassed, among other things, the automatic sharing of information between connected devices, big data, the internet of things and the cloud.

“Oh, I’ve heard of the cloud,” she tells me. “But I don’t really understand it.”

It took me to a scene in that mildly amusing rom-com Sex Tape when Jason Segel’s “husband” screams at Cameron Diaz’s “wife” after a misplaced video file finds itself unfortunately lost in the ether: “Nobody understands the cloud! It’s a mystery!”

Perhaps he’s right. But maybe we should understand the cloud. With the global cloud computing market’s estimated worth of more than $120 billion with a compound annual growth rate of 26.2 per cent a year, according to research firm MarketsandMarkets, businesses are turning their backs on their erstwhile long-term server contracts and embracing the benefits of cloud-based solutions.

Understanding the cloud

Its pay-as-you-go model helps with cash flow, inverts the cost-per-number-of-users relationship and offers all-important agility in today’s less certain working environment.

Listed as one of its Top 10 Strategic Technology Trends for 2016, Gartner defines the concept of information of everything as using smart machines “to extract greater meaning from a rapidly expanding set of sources”, networks, algorithms and devices.

Barely recognised as an algorithm for being so integral to the primary function, Amazon’s recommendations service grows its sales by 10 to 15 per cent.

Big data biggest impact

Gartner senior analyst Steve Prentice says: “Perhaps less famous, but equally valuable, would be the asset optimisation algorithms used for predictive maintenance in industrial equipment, variable-pricing algorithms in the airline industry or customer-specific offers as part of retail loyalty programmes.”

The pace of growth is such that 90 per cent of all data in existence today has been created in the past two years, according to IBM. The technology behemoth categorises big data into four Vs – volume, variety, velocity and veracity.

Each can be illustrated with some rather fantastical-sounding examples: 2.5 quintillion bytes of data are created every day; 30 billion pieces of content are shared on Facebook every month; the New York Stock Exchange captures a terabyte of trade information during every trading session; and poor data quality costs the US economy $3.1 trillion every year.

Analysing big data

But it’s important not to be blinded by figures, says Andy Lawson, UK managing director at cloud-based customer relationship management platform Salesforce.com.

Behind every one of the world’s six billion smartphones there is a customer, or potential customer, waiting to be understood

“The key thing for businesses to remember about the internet of things and the big data it creates as a result is that data can effectively power greater customer success,” he says.

Mr Lawson reminds business owners that behind every one of the world’s six billion smartphones there is a customer, or potential customer, waiting to be understood.

“The vast majority of businesses compete almost entirely on customer experience and to perfect that experience they need to be able to take advantage of the customer insight provided by the connected objects that surround us. Our hyper-connected world has become the source of a vast amount of valuable data and we’re only just waking up to the potential this data represents,” he says.

Stylists at men’s online personal shopping service The Chapar cloud-capture customer information from their favourite brand of jeans to their occupation, inviting rapport while respecting their healthy fear of the high street.

Forecast connected devices

Instead of having to pound the pavement or even stress over an online selection process, a trunk arrives full of recommended garments, which are tailored according to their preferences with each subsequent order.

Since the firm was founded in 2012, trunk retention rate has increased by 25 per cent, its customer base doubles every three months and revenue has grown 650 per cent over the past year.

Reaping the benefits of data

It’s easy to assume that this digital overhaul falls largely at the feet of denim-clad Silicon Valley types, brainstorming over a pool table.

While it undoubtedly does, Highways England is undergoing a transformation of its own and it’s somewhat more down to Earth.

Cast your mind back to 2007. The Environment Agency says more than 55,000 homes and businesses were flooded, with no early warning system for 35,000 of them. Insured losses have been estimated at nearly £3 billion with the government forecasting the cost of total flood damage could rise to £27 billion by 2080, compounded by recent torrential rain and yet more flooding.

Inspired by the Smart Cities initiative led by Innovate UK, Lancashire-based software developer InTouch is rolling out an intelligent “gully data system” to tackle the serious blockage problem threatening our roads.

Currently utilised in one million gullies around the country, the firm has developed a “smart water” flood prediction system, harnessing data from a sensor network around particular regions.

Taking weather input from the Met Office, tree listings to indicate expected “leaf litter” and traffic volumes, the data is processed by algorithm software resulting in practical information such as anticipating levels of silt blockage and the ability to treat accordingly without wasting resources.

Alan Williams, head of business solutions at InTouch, says: “The Victorian drainage system operating across much of the UK simply wasn’t designed to cope with the levels of rain we see today. If we could have predicted the effect of the floods on the road systems, the councils would have been much better equipped to tackle the problem of the blocked gullies.”

Understanding customers’ needs

Regardless of sector, those at the forefront with their digital strategy will be on average 26 per cent more profitable than their peers, according to a Capgemini paper, entitled The Digital Advantage: How digital leaders outperform their peers in every industry, which says the insurance industry should be leading the digital revolution.

James Blake, co-founder and chief executive of Hello Soda, an unstructured data analysis platform utilised by the lending, gaming, recruitment and insurance sectors, says: “Traditional data has not really changed since the 1970s. Since then we’ve had the Facebook and Google revolutions, and there is a lot more information available on individuals.”

Claiming a 62 per cent average uplift on previously “thin” credit files, Hello Soda helps businesses understand their customers’ needs through analysing their digital footprint.

“We specialise in text analytics, so understanding the context of conversations. We have an employment indicator, which will recognise the number of times someone talks about going to work,” he says.

“It’s hard to teach a machine to recognise whether you’re talking about going to work or whether it was hard work at the gym last night. But through psycholinguistics, natural language processing and Bayesian [probabilistic] belief networks, we believe we are able to make those distinctions.”