The big-data challenges facing financial services

Big data is key to the evolution of financial services, but most players in the sector have yet to come anywhere near harnessing its full potential


Many financial services firms have gained access to huge volumes of new data as a result of their digital transformations, but they’ve barely started to extract full value from the wealth of material at their disposal.

The average company collects and analyses only 24% of the operational data available to it, according to a global survey by Seagate and the International Data Corporation in 2020. There are several reasons for this, ranging from concerns about customer privacy and regulatory compliance to difficulties getting all the relevant material in the right format for crunching. 

The first big challenge facing financial services firms seeking to make more and better use of their data is “the data ecosystem, the second is talent and the third is data management”. That’s the view of Edouard Legrand, chief digital officer at BNP Paribas Asset Management. 

The elastic, almost unlimited capacity of the cloud is helping to address the first challenge. Cloud-based technology has also introduced many new capabilities to data and analytics teams. But Legrand believes that, “even if progress has been made regarding the extraction, handling and governance of data, there is always room for improvement. Most of the focus has shifted to implementing cross-functional platforms that enable everyone in the organisation to make the most of the data. Ultimately, all information should be available seamlessly, as should the tools to exploit it.” 

When Legrand talks of the talent challenge, he’s referring to the industry’s inability to attract enough people with the requisite IT skills.

Attracting talent

Nick Broughton, chief information officer at Novuna, agrees with this assessment. “Technology alone doesn’t generate value; you need data-literate people with good ideas,” he says. “Data science skills in particular are key to obtaining real insights from the wealth of data we have. Attracting, retaining and growing our internal talent pool around these new skills is an additional challenge when the demand for them in our market is so high.” 

A survey of more than 250 financial services firms in November 2021 by recruitment giant Hays revealed that 83% had struggled to recruit data scientists, even though they were typically offering annual salaries of more than £100,000 for such jobs. More than a quarter of respondents reported that they didn’t have all the skills they required to achieve their commercial objectives. 

The sector will have to become more flexible with its employment policies and practices if it wants to attract and retain the data specialists it so sorely needs. Knowing that they are in such great demand, these professionals can dictate the terms. Many prefer to work at home and not on the usual nine-to-five schedule, for instance, so it’s up to employers to make allowances for that. 

Technology alone doesn’t generate value; you need data-literate people with good ideas

Some firms are going to great lengths to establish a reliable pipeline of talent by, for instance, establishing relationships with data science communities and setting up apprenticeship programmes.

When it comes to tackling the third challenge that Legrand cites – data management – companies are working hard to put more effective governance systems in place. Their overall aim is to provide a holistic view of their products and customers using data collected and processed in real time. It’s vital to make analysis tools useable and easily accessible to everyone in the organisation who needs them, since no one wants to be calling the IT team for help every five minutes. 

One of the main opportunities arising from all this work is that it helps companies to come up with new ways of satisfying their clients, according to Broughton. 

“Creating outstanding customer experiences should be central to any data-driven initiative,” he says. 

The role of AI

Providing ever-more personalised services is one obvious area of development, but the potential of augmented advice and services, whereby data insights complement human interactions, is also exciting some companies. By combining several sources of data, for instance, they can generate investment signals and intelligence that relationship managers can use to keep their clients better informed. 

With the help of AI tech such as machine-learning systems, client-facing employees can identify patterns and trends that they would never be able to spot unaided. Intelligent tools can also help customers to better understand their financial health and the risk/return profiles of the investment opportunities available to them. 

“We see a future where clients can use tools to experiment. AI models can show them how investments could change over time, for instance,” says James Brake, interim chief data officer at Hargreaves Lansdown. “Yet we find ourselves in a world where regulation often lags behind technological innovation. With this in mind, financial services businesses must be careful in their approach to AI-enabled services, say, to preserve their clients’ trust. It’s why Hargreaves Lansdown recently created the role of head of data ethics. This helps to ensure that our approach to AI is transparent, repeatable, unbiased and able to deliver the best outcomes for clients.”

Financial services firms are already using virtual assistants and chatbots powered by natural language processing, an AI technology that’s fast becoming standard fare. Natural language processing also enables them to run automated searches of information sources ranging from news feeds to earnings reports, which will quickly identify potential problems such as profit warnings and instances of greenwashing. 

The use of alternative data sources – for instance, satellite cameras and smart sensors for climate-sensitive investments – is also likely to become a standard way to inform performance forecasts, for instance. 

Open banking with data-sharing is another exciting area to watch. 

“As the world moves towards thinking about data as a product, we need to start building our services with the data they serve in mind,” Broughton says. “Making our data products more interoperable, secure and governable will be crucial to the success of future initiatives in this field.” 

Once data becomes the product, expect to see a whole new ballgame.