How has the past year affected AI systems?
“AI uses data from patterns of behaviour to make its predictions. But there has been an abrupt change in those patterns, which may even be permanent, although that too is not yet clear, of course. This makes the data harder to work with.
“Predictive modelling is only as good as the data on which it’s built – and the events of the past year have skewed that data. We cannot rely on historical patterns in the same way as we did in 2019. If a transport-based forecast relied on the regular Monday-to-Friday commute, for example, that forecast will need radical re-engineering in the flexible future of 2021 and beyond.
“That means data scientists will have a significant challenge for some years to come, not least in ensuring that 2020-21 data doesn’t corrupt their modelling and forecasting. But, fundamentally, it also demonstrates how easy it is to think of AI as a fait accompli when in reality it can sweep things under the rug. Now is the time to engage with AI to build back stronger.”
AI that embraces uncertainty gives you more information, not less
So how can you still make use of AI?
“By embracing uncertainty. The effective use of AI hangs on two things: understanding assumptions and being transparent about what’s not known. Engaging with these intricacies will give you the best value from your AI investment while also expanding your decision-making power.
“It’s easy to see AI as a crystal ball that’s ready with clear, definitive answers. But an AI system will respond only according to the assumptions used to build it. Maybe it assumes behavioural patterns that are only rough approximations or maybe it smooths over unpredictable random effects. Either way, you need to be able to work with your AI and trust it in the face of such assumptions.
“A powerful way to gain transparency is to build AI that embraces uncertainty by calculating and reporting it clearly – using a forecast range, for example, instead of a single, more definitive number. A system built to be explicit about its uncertainty can help you to assess your risks honestly, empowering your planning and decision-making.”
What’s the point of AI if it doesn’t give you a definitive answer?
“We’re familiar, and so perhaps more comfortable, with uncertainty when it comes to weather forecasts or opinion polls during elections. Will it rain tomorrow? No one can say for certain, because we don’t have perfect data, but this doesn’t mean that ‘a 90% likelihood of showers’ isn’t useful information.
“AI that reports uncertainty can help you to understand and measure the possibility of different scenarios, even the so-called ‘black swan’ events – the outliers, such as extreme weather. By estimating the likelihood of different outcomes instead of making a single prediction, AI gives a nuanced view of what could happen, rather than a misleading guess at what will happen.”
How should you approach AI to get the most out of it?
“To obtain lasting analysis that can confidently support your critical decisions, you need to return to first principles and consider what you want to achieve. AI is no silver bullet. It’s not a product; it’s a process. By readily engaging with the process, you will learn more about your business, what’s driving it and how to make better informed decisions.
“Engagement means getting a clearer picture of what information you need in order to understand the drivers affecting your business. When your uncertainty-capable AI offers you a wide range of possibilities, it’s telling you that there’s more to learn. You may need a higher degree of detail about what is driving customer behaviour, for instance.
“Off-the-shelf or automated AI is accessible and relatively affordable, but it may provide only superficial intelligence if it’s not tailored to specific goals. It may report with confidence what should more accurately be reported with nuance. Instead of black-and-white certainty, a reliable system will report uncertainty that enables you to choose different options depending on how many shades of grey there are.”
Can you exploit the uncertainty that your AI is reporting?
“Yes, AI can work to a very fine degree of nuance if you want it to, providing that it’s based on the right questions.
“So it’s up to you to define what success looks like for your business. You need to be clear about your key performance indicators, about the questions for which you want answers. By engaging with the process that is AI, you can set more specific and sophisticated goals, identify what ‘unlikely’ looks like and decide when that matters.
“AI that embraces uncertainty gives you more information, not less. You can plan for best- and worst-case scenarios and know right away when you’re missing crucial information. All this can help to mitigate the extremes of performance that might otherwise be catastrophic. So, for example, AI can’t give a supplier of wind-generated electricity an exact wind-speed forecast for a given day, but it can predict the likely variability in speed that will help the company decide whether to put back-up systems on standby to ensure continuity of supply.”
So how do you recognise “good” when it comes to AI?
“What ‘good’ looks like depends on your goals. Maybe you need a forecast that’s as close as possible on average, or maybe you need a conservative model that shies away from over-forecasting – the consequences matter as much as the answer itself. Have honest conversations about what successful and transparent AI looks like for you. The more you engage, the more likely you are to end up with a system that pays dividends.
“Embracing uncertainty as you build your AI capability will help you to refine your business model and gain a sure long-term footing, ready to face the future with the risks and rewards firmly under your control.”
For more information or to get in touch, please visit www.smithinst.co.uk
Sponsored by
How has the past year affected AI systems?
“AI uses data from patterns of behaviour to make its predictions. But there has been an abrupt change in those patterns, which may even be permanent, although that too is not yet clear, of course. This makes the data harder to work with.
“Predictive modelling is only as good as the data on which it’s built – and the events of the past year have skewed that data. We cannot rely on historical patterns in the same way as we did in 2019. If a transport-based forecast relied on the regular Monday-to-Friday commute, for example, that forecast will need radical re-engineering in the flexible future of 2021 and beyond.
“That means data scientists will have a significant challenge for some years to come, not least in ensuring that 2020-21 data doesn’t corrupt their modelling and forecasting. But, fundamentally, it also demonstrates how easy it is to think of AI as a fait accompli when in reality it can sweep things under the rug. Now is the time to engage with AI to build back stronger.”
