Artificial intelligence (AI) is no longer a future consideration. It is already embedded in many of the software applications businesses use every day and is increasingly being used by employees to draft reports, analyse data, prepare forecasts, summarise contracts and automate routine tasks.

While AI offers significant productivity benefits, its rapid adoption is creating new risks. In many organisations, the use of AI is advancing faster than the policies, controls and governance frameworks needed to manage it.

Finance functions are natural candidates for early AI adoption because many of their activities involve analysing information, identifying patterns and producing reports. This also means that finance functions may be exposed to greater risks if AI is used incorrectly, particularly where decisions rely upon financial information, forecasts or accounting judgements.

The question for business owners and directors is therefore no longer whether their organisation will use AI. It is whether they know where and how it is being used, and whether appropriate controls are in place.

The risks may not be obvious

AI can produce responses that appear authoritative and convincing but are incomplete, inaccurate or simply wrong. For example, an AI tool may produce a seemingly reasonable cash flow forecast, financial analysis or proposed accounting treatment while overlooking a critical assumption or regulatory requirement. An incorrect output could compromise the reliability of financial information underpinning important business and financial decisions. The consequences may extend beyond the numbers to regulatory exposure, reputational damage and reduced stakeholder confidence.

There are also significant privacy and confidentiality risks. Employees may upload customer information, payroll data, contracts, financial results or other sensitive information into publicly available AI tools without understanding how that information will be stored or used.

Other potential risks include:

• decisions being made without adequate human review;
• an inability to explain, validate or reproduce an AI-generated analysis;
• reliance on outdated, incomplete or biased information;
• critical business processes becoming dependent on a third-party AI provider;
• inadequate records of how a calculation or conclusion was reached; and
• existing approval and segregation-of-duties controls being bypassed.

These risks do not necessarily arise because employees are acting irresponsibly. In many cases, they simply have not been given clear guidance about what is and is not permitted. Directors and management remain responsible for decisions made within the organisation, including decisions informed by AI. The use of AI does not remove existing obligations relating to governance, risk management, privacy, cybersecurity or financial reporting.

Start by understanding current use

Before developing a detailed AI strategy, businesses should first establish how AI is already being used. Some basic questions include:

1. Which AI tools are employees currently using?
2. What information is being entered into those tools?
3. Which business processes and decisions are being influenced by AI-generated output?
4. Who reviews that output before it is relied upon?
5. Are AI tools embedded in existing accounting, payroll, customer relationship management or operational systems?
6. What would happen if a critical AI tool became unavailable or produced an incorrect result?

An AI register can provide a useful starting point. Businesses are often surprised by how many AI-enabled tools already exist within their operations. An AI register should record each AI system or tool, how it is used, the information it accesses, the person responsible for it and the level of risk associated with its use.

Not every use of AI requires the same level of control. Using AI to prepare a first draft of an internal email is very different from using it to generate a cash flow forecast, assess a customer’s creditworthiness or recommend supplier payments. Governance should be proportionate to the potential consequences if something goes wrong.

What does good AI governance look like?

The Australian Government’s Guidance for AI Adoption identifies six essential practices: assigning accountability, understanding impacts, managing risks, sharing essential information, testing and monitoring, and maintaining human control.

Importantly, AI governance does not require an entirely new approach to risk management. Many of the same principles already applied to financial controls, delegations, information security and operational risk can also be applied to AI.

For most businesses, a practical AI governance framework should include the following.

Assign responsibility

A senior person should be accountable for AI governance across the organisation. Responsibility should also be assigned for each significant AI system or use case.

Establish a clear policy

Employees need practical guidance on approved tools, acceptable uses, prohibited information and when human review is required. A policy that is too broad or restrictive may simply drive AI use underground.

Protect sensitive information

Personal, commercially sensitive or confidential information should not be entered into an AI tool unless the organisation understands and has approved how the information will be stored, processed and protected.

Maintain human oversight

AI should support rather than replace professional judgement. Material calculations, reports, accounting conclusions and business decisions should remain subject to appropriate review and approval.

Preserve existing controls

AI should not be allowed to circumvent established delegations, approval limits or segregation of duties. For example, a tool that prepares a journal entry, changes supplier details or recommends a payment should not also be able to approve that action.

Test and monitor

An AI system should be tested before it is introduced and monitored after implementation. Businesses should consider whether outputs remain accurate, whether the system’s behaviour has changed and whether new risks have emerged.

Govern AI without preventing innovation

The objective should not be to prevent the use of AI. An outright prohibition may result in employees using unapproved tools without management’s knowledge, creating even greater risk.

AI can be a valuable business tool, but it should not become an uncontrolled decision-maker. Organisations remain accountable for their information, systems and decisions, regardless of whether AI was involved. Good governance allows businesses to access the benefits of AI while managing the associated risks in a controlled and defensible manner.

Contact Pilot

AI governance is a rapidly developing area. If you would like assistance assessing how AI is being used within your organisation, identifying the associated risks or developing practical policies and controls, contact our Risk Advisory specialists, Terence Brueton, Chris King or Daniel Gill on (07) 3023 1300.