← The notebookAI & governance

AI Governance in Practice: Start with the Consequence, Not the Technology (Part 2 of 6)

By Chun Wong10 min read
Chun Wong Senior Information Security Leader | Cyber Risk & GRC | ISO 27001, SOC 2 | Privacy & AI Governance | Financial Services & Fintech

In the first article in this series, AI Governance: The Board-Level Security Imperative, I argued that AI governance is no longer simply a technology, legal or policy issue. It is now a board-level concern because AI affects data, decision-making, operational resilience, regulatory exposure and trust.

This instalment moves from the board-level case to the practical question: how should organisations distinguish between routine AI use and the use cases that require stronger scrutiny, evidence and oversight?

The answer starts with risk-based tiering and with judging the consequence of the use case, not simply the technology behind it.

My AI GRC journey may look quite different from yours, and I’d value hearing about your experience. What has worked, what has not and what have you learned along the way? Please share your successes, misses and lessons in the comments.

The Core Point

Not every AI use case deserves the same level of scrutiny.

A drafting assistant working with public information should not face the same governance burden as a system influencing underwriting, recruitment, customer eligibility, regulatory reporting or a critical operational process.

The challenge is to apply enough control to manage the risk without creating a process that people avoid.

If governance is too light, material use cases can reach production without proper scrutiny. If it is too heavy, teams will either give up or find a way around it.

Risk-based tiering separates routine, low-impact use from AI that could materially affect people, sensitive data, regulated processes or critical services.

The aim is straightforward:

  • Low-risk uses should move quickly.

  • Higher-risk uses should receive meaningful scrutiny.

  • Some uses should not proceed at all.

Risk-Based Tiering: The Practical Middle Ground

That leads to the practical question:

How do you govern a growing number of AI use cases without creating unmanaged risk or slowing adoption to a standstill?

Treating every use case in the same way does not work.

A tool that improves the wording of an internal document is not equivalent to a system that influences whether someone receives a product, whether a claim is paid or whether an important case is escalated.

The technology may be similar.

The consequence is not.

Done properly, risk-based tiering turns AI governance from a policy document into something people can actually use.

Start with the Consequence, Not the Technology

AI assessments often start with the model or provider.

  • Which model is it?

  • Is it generative AI?

  • Is it hosted internally?

  • Is it supplied by a major cloud provider?

  • Is it accessed through an API, agent or AI gateway?

Those questions matter. But they do not tell you, on their own, whether the use case is high risk.

Start with what the AI is being asked to do.

  • What decisions does it influence?

  • What data can it access?

  • Who might be affected?

  • What happens if the output is wrong, biased, manipulated, unavailable or disclosed to the wrong person?

A familiar model used in a high-impact process may create far more risk than a newer model being used for a low-risk administrative task.

Likewise, choosing a reputable provider does not remove the need to understand how the service handles data, retains prompts, uses submitted information or manages sub-processors.

The technology matters.

The consequence matters more.

What Should Determine the Tier?

A useful assessment does not need to be complicated, but it does need to consider the right things.

Purpose and decision influence

Start with the role the AI is actually playing.

  • Is it generating ideas?

  • Drafting content?

  • Summarising records?

  • Recommending an action?

  • Ranking people, cases, transactions or alerts?

  • Approving, rejecting, pricing or determining an outcome?

The closer the system gets to making, or materially shaping, a decision, the stronger the governance needs to be.

A system that drafts an internal summary is not equivalent to one influencing whether a candidate is shortlisted, a customer is approved or a claim is paid.

Data sensitivity

Next, look at the data.

  • What goes into the system?

  • What can it access?

  • Where is the data processed?

Could the provider retain it, use it for training or disclose it to sub-processors?

Relevant distinctions include:

  • public information

  • internal business information

  • commercially confidential information

  • personal and special category data

  • customer, patient or policyholder information

  • privileged material

  • regulated records

  • credentials or security data

The same tool may be low risk in one context and high risk in another simply because the data changes.

People and organisations affected

AI risk is not just about the system. It is also about those affected by its output.

  • Could the use case affect a customer, employee, candidate, claimant, investor, supplier or vulnerable person?

  • Could it lead to unfair treatment, financial loss, reduced access or reputational harm?

  • Would the affected person know that AI influenced the outcome?

  • Could they challenge it?

The greater the potential impact, the stronger the requirements for human review, transparency, testing and escalation.

Autonomy and system access

AI may support a human decision, recommend an outcome or act automatically.

Those are not the same.

Simply saying there is “a human in the loop” is not enough.

The reviewer needs enough information, time, competence and authority to challenge the output. If the human routinely accepts the recommendation without scrutiny, the control exists on paper rather than in practice.

For agentic or tool-enabled systems, the assessment should also consider what the AI can actually do:

  • Which systems can it access?

  • Which permissions does it inherit?

  • What actions can it execute?

  • How are those actions constrained and monitored?

  • Could manipulated inputs alter its behaviour?

The more autonomous the system, the stronger the control requirements.

Scale, dependency and reversibility

Scale matters.

A small pilot may create limited exposure. The same capability deployed across thousands of users, customers or transactions may create material risk.

Even a modest error rate becomes significant when repeated at volume.

Operational dependency matters too.

A use case may become high risk because an important service depends on it, even if it does not make a consequential decision.

Ask:

  • Would an error be detected quickly?

  • Can the decision or action be reversed?

  • Is there an audit trail?

  • Would the organisation know if the model’s behaviour changed?

  • Is there a workable fallback if the service fails?

The harder a failure is to detect, reverse or recover from, the higher the risk.

Regulatory and contractual context

The same use case may carry different obligations depending on the sector, jurisdiction and contractual environment.

Relevant considerations may include data protection, consumer protection, equality obligations, employment law, operational resilience, outsourcing, model risk and sector-specific regulation.

Contracts may also restrict model training, sub-processors, processing location, confidentiality, retention and incident notification.

This is why AI governance cannot sit entirely within technology or security.

Legal, privacy, compliance, risk, procurement and the business all have a role.

A Practical Four-Tier Model

Organisations do not need a highly sophisticated scoring engine before they can start governing AI sensibly.

A four-tier model is often enough.

These tiers are illustrative rather than absolute. The same capability may sit in different tiers depending on the data involved, the decisions it influences, its autonomy, its scale and the regulatory or contractual context.

Tier 1: Low Risk

Low-impact uses involving non-sensitive information and little or no influence over material decisions.

Examples include:

  • drafting generic internal communications

  • summarising public information

  • generating meeting agendas

  • brainstorming non-confidential content

  • improving grammar or formatting

Typical controls should include approved tools, basic user guidance, restrictions on sensitive data and human review before use.

These use cases should move quickly through a lightweight route.

Requiring committee approval would add little value and would probably encourage people to bypass the process.

Tier 2: Moderate Risk

Uses involving internal information, business activity or outputs that may be used externally.

Examples include:

  • summarising internal documents

  • drafting customer communications

  • assisting with software development

  • supporting security or compliance analysis

  • generating management information

Controls may include documented ownership, data classification review, supplier assessment, output validation, access restrictions and periodic reassessment.

These use cases should be reviewed, but the process should remain pragmatic and time-bound.

Tier 3: High Risk

Uses that could materially affect people, regulated processes, financial outcomes or critical services.

Examples include:

  • underwriting, pricing or claims support

  • recruitment, eligibility or fraud decisions

  • regulated or consequential analytics

  • systems with significant autonomy

  • AI components on which critical services depend on

Controls should normally include named senior ownership, formal multidisciplinary assessment, testing, meaningful human oversight, monitoring, fallback arrangements, incident response and approval before production use.

A provider’s claim that a product is secure, responsible or enterprise-ready does not remove the organisation’s accountability.

Tier 4: Restricted or Prohibited

Uses where the risk is unacceptable, the legal or ethical basis is unclear or meaningful control cannot be demonstrated.

Examples include:

  • placing credentials or highly sensitive data into unapproved tools

  • making material decisions without accountable human oversight

  • covert monitoring or profiling without a lawful, ethical and proportionate basis

  • using customer data to train models without authority

  • deploying systems where material failure cannot be detected or reversed

  • using AI to circumvent legal, regulatory or contractual requirements

These boundaries need to be clear.

Employees are more likely to follow rules they understand than broad instructions telling them to “use AI responsibly”.

Tiering Must Trigger Action

A risk label on its own does not manage anything.

Higher tiers require stronger evidence that the controls are operating, not simply that they have been designed.

The value lies in what the tier triggers.

Each tier should lead to a clear pathway covering:

  • who can approve the use case

  • which functions need to be involved

  • what evidence is required

  • what data may be used

  • what testing is needed

  • what human oversight applies

  • what monitoring is expected

  • when the use case must be reassessed

Where possible, this should be built into existing processes such as procurement, supplier due diligence, privacy assessment, security risk assessment, architecture review, change management and incident management.

The aim is integration, not duplication.

Risk tiering is also not a one-time exercise.

A pilot may begin with public data and a small group of users. It may later connect to internal systems, enter a customer journey or begin influencing operational decisions.

At that point, the original assessment may no longer be valid.

Material changes in data, autonomy, scale, jurisdiction, provider or integration should trigger reassessment.

The tier should follow the use case as it evolves.

Avoid False Precision

It is easy to over-engineer this.

A detailed scoring model can help with consistency, but it can also create false confidence.

A use case scoring 48 rather than 51 is not necessarily meaningfully safer.

Scoring should support judgement, not replace it.

Clear escalation rules are often more valuable than an elaborate formula that few people understand.

Several failure modes appear repeatedly:

  • classifying risk according to the tool rather than the use case

  • treating human review as a universal answer

  • focusing only on data leakage while ignoring decision quality, bias, resilience and accountability

  • assessing the use case once and failing to revisit it

  • creating a process so burdensome that teams route around it

  • allowing AI into production before ownership, monitoring and evidence are clear

The objective is not maximum control.

It is effective control.

What the Board Needs to See

Boards do not need a catalogue of every low-risk productivity use case.

They need visibility of the material risk position.

Useful reporting might include:

  • high-risk use cases and trends

  • material AI dependencies

  • significant control gaps

  • incidents and near misses

  • third-party concentration

  • overdue decisions or unclear accountability

The board should also understand whether ownership is effective.

A high-risk AI register with no accountable business owner is not evidence of governance.

It is evidence that a gap has been identified.

That gap still needs to be closed.

The CISO and the Wider C-Suite

Risk tiering is cross-functional, but someone has to make it operational.

From a GRC perspective, the CISO, or equivalent security leader, has an important role in ensuring that the wider C-suite understands the risks, obligations and accountability model.

That does not mean the CISO should own every AI decision.

Business leaders should remain accountable for the processes and outcomes within their areas. Legal, privacy, compliance, technology, data and risk each bring necessary expertise.

The CISO’s role is often to connect those disciplines, translate the risks and make sure the control model works in practice.

There is also a commercial dimension.

Customers, investors, business partners and regulators increasingly want assurance over AI use, data handling, model providers, monitoring and contractual controls.

A clear risk-tiering model helps the organisation answer those questions consistently.

Not every organisation has complete or mature C-suite coverage. I have seen NEDs provide necessary challenge where executive ownership was unclear or still developing.

That can help in the short term, but it should not become a substitute for accountable executive ownership.

Proportionate Governance Is the Goal

  • AI governance should make safe adoption easier, not harder.

  • Low-risk uses should move quickly within clear guardrails.

  • Moderate-risk uses should receive proportionate review.

  • High-risk uses should face meaningful challenge, evidence and oversight.

  • Restricted uses need clear boundaries.

The test is not whether the organisation has produced the most detailed framework. It is whether people understand it, use it and make defensible decisions.

A practical risk-tiering model should help answer five questions:

  1. What is the AI being used to do?

  2. What data and people are affected?

  3. What happens if it fails?

  4. What controls are proportionate?

  5. Who is accountable for the outcome?

That is how AI governance moves from policy into practice.

The next article in this series will look at security as a commercial enabler: why the people signing the cheques need to understand not only what the investment protects, but how it supports trust, growth and the bottom line.

Original LinkedIn publication ↗

Personal commentary. Views are the author’s own.