Crunch time: Let AI work the numbers, but leave the emotional decisions to humans

On the surface, the accounting and finance sector appears to be a great test case for AI integration. Work revolves around numerical data, while processes are structured and rules-driven.

It also seems an ideal environment for automation, giving scope for AI to process information at speed and scale. For that reason, it’s already being used to handle some of the sector’s necessary yet repetitive and time-consuming work, such as invoice capture, month-end reporting and anomaly detection.

But as confidence in AI grows and is integrated into more accounting and finance software, there’s a risk that organizations once fearful of the technology may go to the other extreme. At that point, there’s a danger that some professionals may lean so deeply into AI that they become detached from the numbers.

That matters because the decisions that sit behind the figures and spreadsheets are rarely driven by pure logic. Financial strategy is shaped by leadership judgement, organizational priorities, risk appetite, and sometimes (although none of us likes to admit it) internal politics.

That means a surprisingly large proportion of commercial decisions contain a human component that AI cannot, and probably should not attempt to, replace.

For many finance and accounting teams, the solution is to use automation to reduce cognitive load. Individuals are then free to focus on interpretation, creativity and decision-making. It keeps humans in the process exactly where instincts and experience are most needed.

Surfacing the truth

Numbers often create a sense of objectivity and certainty, but financial reporting still involves interpretation, context and judgement. So, when accounting and finance professionals engage with AI tools and automation, it’s important they maintain a firm grasp of their data.

But there’s also a moral dimension. Most UK accounting bodies have set out ethical codes and guidance on deploying AI in ways that preserve human oversight and authority, and protect sensitive data.

It’s an important consideration because, for the first time, we’re not simply asking technology to deliver calculations, as we might with a spreadsheet or a calculator. We’re asking for opinions and interpretations, while simultaneously hiding much of the methodology that gets us to the answer.

In some ways, it’s like comparing classic and modern cars. Open the bonnet of a car from the 80s or before, and you can see all the engine parts. You could buy a manual and (if you were brave) carry out a few repairs yourself. Today, if you open the bonnet of a new car, much of what you’re looking at is sealed, covered or controlled by software.

Sure, there are signs to add windscreen wash, but that’s about it. For everything else, we’ve handed control to diagnostic machines, telemetry and professional mechanics. It’s great not to get our hands dirty, but not so wonderful if the car breaks down miles from home.

The problem isn’t that the technology in new cars is bad. It’s that the further we move from understanding the systems we rely on, the harder it becomes to challenge them when something appears to be wrong.

A question of trust

For the same reason, finance leaders still need visibility into how AI-generated outputs are reached and the confidence to challenge them when necessary. That’s important because AI output is ultimately based on prediction, and prediction is not the same as verified truth.

Nevertheless, we tend to trust it—usually because it’s saving us time and feels authoritative. Published research supports this view: A study by KPMG and the University of Melbourne found that two-thirds of workers do not evaluate AI outputs for accuracy, despite more than half reporting that they had made AI-related mistakes in their work.

At the same time, research from Glean found that in one month, 77% of UK workers had to correct AI-generated work. Together, the findings suggest that while AI is becoming an increasingly trusted part of the workplace, human oversight and validation remain essential.

Keeping score 

Output confidence scoring could be one solution. Rather than trusting every AI-generated output, organizations can implement processes in which the technology itself flags lower-confidence outputs for human review before action is taken. This creates a better balance between automation and oversight. It also gives finance teams clearer visibility into the reliability and completeness of AI-generated outputs. 

In effect, confidence scoring creates an early warning system, helping organizations understand what AI is recommending and, importantly, how much trust they should place in those recommendations.

A word of warning: The responsibility for checking outputs should not sit exclusively with junior staff. It’s true that they need to learn the manual calculations and processes sitting behind automated systems so they can properly challenge the results.

However, the same principle applies to seniors. Their experience should not distance them from the logic behind the numbers. If anything, AI makes that visibility even more important.

There is also another, longer-term consideration. Every finance team has people who instinctively know when something doesn’t look right. They’ve spent years working with the data and understanding the processes behind it.

Some of that knowledge is written down, but much of it is gained from experience and built over time. If organizations become too detached from the systems and calculations underpinning their reporting, they risk losing some of the institutional memory that helps them challenge anomalies and spot potential issues.

The reality beneath the numbers

Productivity gains from AI and automation are creating more space for higher-value thinking rather than removing people from the process. The information surfaced by AI can ultimately help leaders make better decisions by revealing connections, patterns and insights that may previously have remained hidden within the data.

In many ways, this is not new: Leaders have always relied on summaries, dashboards, and reporting layers to help them make decisions. Indeed, the purpose of month-end reporting and executive summaries created by accounting and finance teams was always designed to deliver that information in an easy, digestible way.

But AI dramatically widens the gap between decision-makers and the operational reality beneath the numbers. Before AI, executives could ask their finance team to explain the rationale on which they based their reports. Even after the move from spreadsheets to software, those teams remained closely involved in creating the reports and could usually trace how the conclusions were reached.

With that in mind, the finance teams that benefit most from AI will likely be the ones that strike the right balance between automation and oversight—the ones that can delegate repetitive, manual processing tasks but don’t lose visibility into the data. If organizations surrender that understanding completely, then commercial judgement and entrepreneurial instinct may be blunted or misdirected.

Organizations may, therefore, ultimately prefer not to hand complete control to AI-enabled systems and instead focus on people-led automation. The goal should be confidence in transparent AI-supported workflows, not unswerving reliance on more opaque automated outputs. That stops AI from acting as a replacement for human judgement and transforms it into an extension of human capability.

We've featured the best business intelligence platform.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit