Big data comes in waves. One sector notices it, gets very excited, and then just when the hoopla has died down, another sector goes bananas for big data.
The military were early adopters, using data patterns to find terrorists in Baghdad. This spread to domestic police forces, as they identified things like which houses are likely to contain illegal immigrants, and where crime spots were emerging.
Retail got the big data bug about four years ago. The ability to suck in all data to calculate tricky concepts such as the lifetime value of a customer and best products to cross-sell proved hugely productive. Now manufacturing is seeing the potential. Big data means airlines can guess when a part is about to wear out and schedule engineers to replace it.
And project management? We seem to be in the calm before the storm. “In our experience and from what we have seen with customers, big data has yet to really take hold in project management,” says Brian Brinkmann of Logi Analytics, a self-service data analytics platform.
Time to adopt big data
Verdicts from across the big data industry echo this. Dr Mathieu d’Aquin of the Open University’s big data-focused Knowledge Media Industry, admits: “I am not personally aware of existing applications based on the meaningful use of big data in project management.”
To which there are two replies. First, there are early adopters of big data in project management. And second, the case for big data is so profound that any project manager, who wants to thrive in the next few years, ought to starting learning about it.
Here’s one example of big data delivering the goods in project management. Energy giant Centrica has three million customers. It wants to reach five million by the year-end. To do that it needs to maximise its data usage. With the right approach it can give marketers the power to see why customers defect to rivals, which campaigns work and maintenance staff can see where to focus efforts.
But as a big data novice, Centrica started out unprepared. The data was stored in a variety of incompatible silos. This made it impossible to look for correlations across the data. Centrica partnered with Hortonworks, one of the main contributors to the Hadoop technology, which is ubiquitous in big data deployments.
“Centrica had traditional data like billing and HR, which is actually quite small data,” says Andy Leaver, Hortonworks’ vice president of international operations. “Then there was data from contact centres and web interactions, and that is medium-sized data. And finally there was machine data coming from smart meters, their people on the road and other automatic sensors, which is definitely big data. In total Centrica adds 300 gigabytes of new data a day.”
Hortonworks gathered all Centrica’s data into a single place, known as the datalake. This is a 4 petabyte database. Searching technologies were added, with simple-to-use interfaces. Now 500 people can run queries and analysis on the data. They create their own reports and dashboards. Project managers in engineering are able to look at the data to see if their projects are on track. Project managers in marketing can refine customer interactions. Bulk marketing has stopped as a result.
Universal access
Pretty much anyone can now use the data. Mr Leaver says: “Non-IT people in various departments can use the data on a self-service basis. They don’t need to work with a data scientist to pore through it.”
Project managers wanting to become big data experts can take heart from this. It’s not a path to purgatory as many fear. Rami Hadadi, senior director, Europe, the Middle East and Africa, at big data software specialist Informatica, says: “Many project managers perceive rapidly evolving big data technology as too much of a technical mountain to climb. They believe they will either have to learn to code by hand or hire a data scientist at great cost in order to convert data from its raw state into actionable insight.” It’s a myth.
Project managers should learn how to use the data and foster good data etiquette
“Project managers need access to tools which simplify the process; for example, semantic search, data discovery and self-service data preparation. Rather than spending 80 per cent of the time prepping data, project managers should be taking advantage of automated solutions to free up resources, which could be used to deliver business value based on the data insight,” says Mr Hadadi.
The only merit in learning the vocabulary of big data is to demystify it. Terms such as Apache Hadoop, Pig, MapReduce and YARN get bandied about. Dr Marcello Trovati, big data programme leader at the University of Derby, says project managers should forget about going too deep.
“Paradoxically, for such a technical field, there is no need for project managers with highly technical degrees or PhDs,” he says. “Actually, it might even be counterproductive. Too much knowledge in one area might create intellectual biases, which would potentially result in pursuing specific paths because they might be in line with one’s expertise.”
Ensuring maximum return
Instead project managers should learn how to use the data and foster good data etiquette. Kim van Rooyen of big data consultancy Turner & Townsend says it’s vital for project managers to ensure they hand over data to clients post-project.
“Data should be at the heart of the contract, and there should be seamless transition between the project ending and a client taking control. A building may take three years to build, but the building will be there for 30 years. Project managers need to ensure clients get the maximum return for the life of the result by using the data,” he says.
Using past projects as benchmarks is critical. Mike Merritt-Holmes, co-founder of the Big Data Partnership, says: “Aspirationally, project managers can take in historical data from projects and over time use machine-learning to get a risk profile. You can identify flags and start to see trends.” Though he adds: “It is not mainstream yet.”
Want to improve your big data skills? Then enrol on one of the many courses available. Turner & Townsend run a distance learning course in partnership with the Royal Institute of Chartered Surveyors, called Certificate in Building Information Modelling Project Management, suitable for facilities managers, surveyors and construction project managers. Other sectors host one and two-day introductory courses. The lessons will be immediately transferable to project management.
The key is to get involved early. As Mr van Rooyen puts it: “There is a generational split, with the old guard sticking to old ways and being reluctant to embrace the digital age, and the new guard who see how data can improve delivery. It’s like Blockbuster versus Netflix. Those who use data will lead the way in the future.”
Like all those other industries before it, when project management adopts big data, it’s going to be a big deal.
CASE STUDY: SEFTON COUNCIL
Empty houses don’t pay full council tax – so occupiers often lie to councils, costing a lot of money. Sefton Metropolitan Borough Council suspected people at almost 4,900 addresses were claiming incorrectly, but carrying out physical house-to-house inspections would have been too costly. The solution? A big data number-crunching exercise. It hired big data consultancy arvato to do the job.
John Wybrant, account director at arvato, explains how data can reveal wrong-doing: “The solution was to look at credit and other financial information, such as insurance history and payment activities, and use specialist software to overlay this data with the data the council already had on each property. The result of this analysis was a ranking table, which provided us with a clearer picture to target the houses that were the most likely to be occupied.
“By using big data in this way, we were able to determine which properties had illegitimately been registered as empty, and subsequently reduce the level of council tax fraud and evasion.
“In fact, Sefton Metropolitan Borough Council was able to classify 434 houses out of 4,900 as highly likely to be occupied and a further 1,313 as reasonably likely. This categorisation helped us prioritise the subsequent physical inspections, starting with those with the highest chance of being occupied. As a result of verifying empty properties, 89 were brought back into occupancy after just the first month of the operation.”
