Skip to content

AI Governance in Practice: Why Oversight Is Not the Same as Accountability

Most organisations adopted AI faster than any way to defend what it produces. Real governance is not a reviewer at the end of the process — it is the evidence that lets someone reconstruct…

Overview

Most organisations have adopted AI faster than they have adopted any way to defend what it produces. The tools are in the building, the pilots have run, and somewhere a model is already drafting text that will inform a decision with real consequences — a supplier assessment, a compliance summary, a board paper. What is usually missing is not capability. It is the ability to answer a simple question six months later: on what basis was this concluded, and who is answerable for it?

That question is the whole of AI governance. Policies, principles and committees are preparation for being able to answer it. They are not a substitute for it.

Oversight Is Not the Same as Accountability

The most common control in use today is the human reviewer. A person is placed at the end of the process, asked to check the output, and a box is marked complete. This is described as keeping a human in the loop.

It is worth being honest about what that control actually achieves. A reviewer who receives a finished document — with no record of what evidence it drew on, which version of which standard applied, or what the model was asked — is not reviewing it. They are agreeing with it. They have no basis on which to disagree, and no way to demonstrate afterwards that they examined anything at all.

Oversight without evidence transfers accountability to the reviewer without giving them the means to exercise it. That is worse than having no control, because it produces a signature that looks like assurance.

Evidence Is the Unit of Governance

Governance is not settled at the moment of approval. It is settled later, when someone asks you to reconstruct the decision.

That reframes what needs to exist. Not a policy stating that AI output must be reviewed, but a record showing which inputs were accepted, which were rejected and for what reason, which version of which source governed the work, and what changed between the first draft and the approved version. If that record is assembled after the fact, it is a reconstruction, and it will be treated as one. If it accumulates as the work happens and cannot be altered once set, it is evidence.

The practical test is simple. Take a completed piece of work and try to answer, without asking the people who produced it: what did this rely on, and has any of it changed since? If answering requires a conversation, the process is not governed.

Separation of Duties Applies Here Too

Finance and audit settled this question a long time ago. The person who prepares work should not be the only person who approves it, and the person who approves it should not be the person who releases it. The principle is not about distrust. It is about making an error visible to someone who has no stake in it not having occurred.

AI-assisted work has quietly abandoned this. One person often prompts the model, edits the output, decides it is sound, and sends it. The speed is the attraction. The absence of a second pair of eyes is the cost, and it is rarely priced.

Reintroducing separation is unglamorous and effective: distinct responsibility for authorship, for independent quality review, and for release, each recorded, each held by a different person. It adds hours to a workflow and removes an entire category of failure.

Analysis Can Be Delegated. Judgment Cannot.

There is a meaningful difference between using AI to analyse evidence and using AI to reach the conclusion.

The first is a strong use of the technology. Models read widely, find what is missing, test a document against a standard, and surface what a person should look at more closely. The second is where the difficulty begins, because a conclusion carries responsibility — and responsibility cannot be held by a system that cannot be questioned, cannot be sanctioned, and will not be in the room when the outcome is examined.

That distinction is worth designing into the process rather than leaving to individual discretion. Let the model do the analysis. Let a named person write the conclusion, and say so on the record.

Governance Has to Outlive the Deliverable

Most governance stops at the moment of delivery. The report is approved, it is released, and the file closes. Whether the recommendation worked becomes a separate conversation, usually held nowhere.

This is the largest gap, and closing it would do more for the credibility of AI-assisted work than any policy document. If a piece of work claims an outcome, that claim should be written down before delivery, measured afterwards against the same definition, and confirmed or disputed by the party who received it. A claim that cannot be tested is not a governed claim. It is a marketing sentence.

Where to Start

Not with a framework. Frameworks are written to be complete, and completeness is the reason they are so rarely implemented.

Start instead with one workflow that already matters — one producing something a regulator, a client or a board could reasonably ask you to justify. Make its inputs explicit. Record what was accepted and what was not. Give authorship, review and release to different people. Then write down, before delivery, what the work is expected to change, and return later to see whether it did.

That is a smaller undertaking than most governance programmes, and it produces something they generally do not: a piece of work you can actually defend.

Continue the conversation

What could this insight unlock for your organization?

Talk with our team