The governance gap in enterprise AI deployments

Most organisations already govern their information reasonably well. Access to customer data follows roles. The document system enforces classification labels. Email is archived and discoverable. Ask the security team who opened a sensitive file last Tuesday and they can usually tell you.

Then someone pastes that file into an AI assistant, and most of those controls stop at the browser tab. The DLP policy may not see the prompt. The classification label doesn’t travel with the text. The conversation sits in the vendor’s system on the vendor’s retention schedule, and nobody inside the organisation can search it.

That is the governance gap. It isn’t caused by AI being unusually dangerous. It exists because the tool was adopted as a productivity app rather than as an information system, so it never went through the questions every other system has to answer.

How the gap tends to surface

Usually it’s one of three moments. These are illustrations, not case studies.

An auditor asks for an inventory of every system that processes personal information. The AI tool has to go on the list, and the follow-up questions (retention, access, logs) have no good answers.

A staff member pastes a client’s financial details into a chatbot to tidy up a letter, then mentions it in passing. Now someone has to decide whether this is a data breach, and the organisation can’t say what was sent or where it went.

A large client sends a supplier security questionnaire with a new section on AI. You’re asked which tools staff use for client work and how the client’s information is protected in them. “We have a policy” won’t satisfy a procurement team that asks for evidence.

Apply the questions you already ask of other systems

Governing an AI tool isn’t a new discipline. It’s the existing one applied consistently.

Access first. If an analyst can’t open client financial records in the core system, they shouldn’t be able to reach the same records through an AI assistant that’s connected to a document store. The current ISM says this directly for AI applications: ISM-2093 expects role-based access controls that restrict access to sensitive data.

Classification next. Each conversation should carry a classification, set by the user or by policy, and the tool should behave differently depending on it. A CONFIDENTIAL or OFFICIAL: Sensitive conversation should get stricter handling than a PUBLIC one, such as redaction, blocked features or a narrower set of models.

Then the audit trail. Record who sent each prompt, when, which model answered and what classification applied. Decide deliberately whether to keep full prompt text or a hash. Full text is better for investigations, but it creates a new store of sensitive data that needs protecting in its own right.

Retention and deletion. AI conversations should follow your retention schedule, not the vendor’s default. If your policy says certain records are destroyed on request, the tool needs to be able to do that. The ISM now has a control for exactly this (ISM-2123: prompts and outputs are securely deleted when a chat session is removed).

Finally, incidents. Add AI scenarios to the playbook: sensitive data in a prompt, prompt injection through an uploaded document, a compromised account bulk-querying the assistant. Each of these needs the audit trail above to investigate.

Why the audit trail matters to regulators

Two obligations make the point concrete.

Under the Notifiable Data Breaches scheme, an entity that suspects an eligible data breach must take all reasonable steps to complete an assessment within 30 days, and then notify affected individuals and the OAIC as soon as practicable if it is one. Doing that assessment means knowing whose personal information was in the prompt. If nobody can see the prompt, the assessment is guesswork.

For APRA-regulated entities, CPS 234 requires mechanisms to detect and respond to information security incidents in a timely manner (paragraph 23) and notification to APRA within 72 hours of a material incident (paragraph 35). It’s hard to show either for a tool that produces no records.

A policy that can be enforced

An AI usage policy doesn’t have to be long. The ISM has included a control for one since December 2025 (ISM-2074), and it’s a sensible baseline outside government too. At a minimum it should answer:

• which AI tools are approved, and for which categories of information
• what is explicitly excluded (for example, health information, privileged material, OFFICIAL: Sensitive in public tools)
• how users classify a conversation, and what happens if they don’t
• how long conversations and logs are kept, where, and who owns them
• who can read the logs, and who reviews unusual use
• which AI events count as a security incident and who gets told
• when the policy is reviewed

The harder part is enforcement. APRA’s 30 April 2026 letter to industry on AI put it plainly: in many entities, preventative controls were lacking, with entities relying primarily on policy direction or after-the-fact checks rather than enforceable technical restrictions. That observation was about banks, insurers and super funds, but it describes most organisations. A policy that depends entirely on staff remembering it won’t hold up when someone asks for evidence.

Technical enforcement means classification applied in the tool, access rules enforced by the system rather than by etiquette, and logs generated automatically and sent somewhere your security team already looks.

Start narrow

The gap doesn’t need closing in one go. Find the use that carries the most risk, usually anything involving personal information, client data or privileged material, and put that into a governed tool first, with classification and logging switched on. Widen it once the controls have been shown to work.

K9 AI was built around this pattern. Every conversation carries a classification that sets how it’s handled, including whether sensitive details are redacted before any model sees them (automatic redaction applies at the stricter levels; the default level scans without redacting, and admins can change that). Every prompt is logged with user, time, IP, model and classification, retention is set by your administrator from 30 days to 7 years or until deleted, and legal holds can be placed on an organisation, user or conversation.

Frequently asked questions

What is the minimum AI governance an organisation should have?

A written AI usage policy (the ISM has expected one since December 2025, under ISM-2074), an inventory of approved tools and use cases, technical controls that match the policy, and logs your security team can actually query.

Should AI chats be kept under our records retention schedule?

Treat prompts and outputs as business information and decide their retention deliberately. If the tool keeps them on its own schedule, or can't delete them on request, that is a gap to close before rollout.