Breaking down Big Data Silos

Technology has traditionally been in the background – while the IT team might make the business more efficient through technology, it has also led to the view that IT is not strategic.

Throwing aside “IT crowd” stereotypes is essential, as sales, marketing and finance teams think they succeed despite IT, not because of it. Big data challenges companies’ business models and strategies with the aim of increasing revenues. Explaining why it is different is important, both for IT and for the business.

I know that many organisations argue they have been handling big data for years. In some respects that is true – individual silos of tech can be run well and create good analytics. However, the challenge is that companies can’t join that data up in the right ways to help their employees make better decisions.

Digital companies, such as Google, Amazon and Netflix, can analyse data at millisecond speeds and have automated processes in place to respond based on the customer intelligence they create. For these companies, it’s not just a case of collating information about customers. Instead it’s about being able to understand and respond to customers faster and better than competitors.

We have all seen industry sectors, such as media, entertainment and retail, seemingly change overnight as content streaming services have grown. These all-you-can-eat services rely on data to make recommendations to you, all with the view of keeping you as a customer for months, rather than one-off purchases. Enterprise companies see these digital companies as upstarts, but they also recognise how they can re-use some of these approaches for themselves.


I believe that understanding the customer requires a single view of their behaviour. While people might have had good individual sources of data, hunches and personal experience to call on in the past, this is no longer sufficient; it’s the combination of all this different data that is critical.

It’s almost impossible to see a single view of a customer when in-house data on customer, product and service transactions is spread across different systems – a nightmare scenario for large organisations that offer a variety of products, such as financial services, telecoms and utility companies.

In many insurance companies, for example, customer information systems related to general insurance products will hold entirely different data to those around pension products. Typically these systems don’t interact, so it’s not uncommon for a help desk professional to ask, “Do you have any other policies with the organisation?” This is frustrating for the customer and will not help win their loyalty.


This desire to bring in big data drives business change too. You don’t have to change everything at once and starting small with a big data project is common, as this is a good way to prove value. However, this should not be used as an excuse for change only being “skin deep”. The risk here is that companies put a few projects in place, but don’t go further than putting “lipstick on the pig” of their existing IT.

The elephant in the room for big data is always going to be the existing and overburdened legacy technology infrastructure that is in place. It’s my belief that organisations have to look closely at how to link existing systems and data to exploit new opportunities. Big organisations have been very risk averse, so this is a big cultural change and can feel more like stepping into a jungle.

Big data doesn’t happen in a silo – by integrating data projects into a coherent whole, chief information officers can support better, more effective, more beneficial big data strategies. This is part of wider transformation activities and business leaders have to break out of the mentality that exists around IT to decide where they want their businesses to be in the next decade. The question in the end has to be are you content to remain a legacy-based technology dinosaur or evolve into an integrated, responsive and data-driven mammal?