Normalisation of deviance erodes oversight effectiveness when systems operate without incident for extended periods. Automation provides a vigilance baseline that does not degrade, while human oversight focuses on judgement activities. Portfolio scaling requires shared infrastructure, tiered oversight intensity, centralised governance with distributed execution, and portfolio compliance dashboards.
High-risk AI systems may operate for years, and maintaining effective oversight over these timescales is a significant organisational challenge.
High-risk AI systems may operate for years, and maintaining effective oversight over these timescales is a significant organisational challenge. Research in safety-critical industries including aviation, nuclear power, and healthcare consistently demonstrates that human vigilance degrades over time, particularly when the system operates reliably and incidents are rare.
Normalisation of deviance occurs when a system operates within expected parameters for months or years and the organisation's sensitivity to anomalies diminishes. Thresholds originally set to trigger urgent review become tolerated as normal. Minor non-conformities that would have prompted immediate action in the system's first quarter of operation are accepted as routine. This phenomenon is well-documented in safety-critical industries, and AI operational oversight is susceptible to the same dynamic.
Four countermeasures address the degradation of oversight effectiveness over time.
Four countermeasures address the degradation of oversight effectiveness over time. Personnel rotation on a 6 to 12 month cycle is the primary structural countermeasure. A new person brings a fresh perspective: they notice anomalies the previous person had normalised, ask questions about processes the previous person had stopped questioning, and identify documentation gaps the previous person had worked around. The AI Governance Lead plans the rotation schedule in advance with a handover period including knowledge transfer and a documented checklist.
Threshold drift checks address the gradual relaxation of operational thresholds. Over time, teams may informally adjust alert thresholds upward to reduce alert volume, and each adjustment is individually reasonable but the cumulative effect decreases sensitivity to genuine problems. Quarterly threshold drift checks compare current operational thresholds in monitoring configuration, CI pipeline gates, and alert rules against the values documented in the . Any discrepancy must be either reverted to the documented threshold or formally approved by updating the AISDP with the new threshold and the rationale.
An organisation's first high-risk AI system receives intensive attention.
An organisation's first high-risk AI system receives intensive attention. By the time the portfolio reaches ten or twenty systems, the oversight framework must scale or it will fail. The governance model designed for a single system will not survive a portfolio.
Shared infrastructure is the foundation. Monitoring infrastructure, evidence repositories, document management systems, and are designed as shared services supporting multiple systems. Shared monitoring with multi-tenant configuration allows a single team to oversee all systems from a unified dashboard, with per-system metric labels enabling both aggregate views and per-system drill-down. Shared infrastructure also enables cross-system analysis detecting patterns such as a common vulnerability across multiple systems using the same GPAI model.
Quarterly. Compare current operational thresholds in monitoring configuration, CI pipeline gates, and alert rules against the values documented in the AISDP.
Personnel not involved in daily operations review monitoring data, evidence, the non-conformity register, and meeting minutes with no prior context. They often reveal systemic issues the operational team has normalised.
A finding in one system, such as a monitoring gap that was exploited, can be applied as a preventive check across all other systems through standardised processes and cross-system analysis.
Over months and years, thresholds are tolerated, minor non-conformities accepted, and sensitivity to anomalies diminishes, requiring active countermeasures.
Automation provides continuous threshold monitoring, compliance checks, and anomaly detection that do not degrade, freeing humans for judgement-intensive tasks.
Through shared monitoring infrastructure, tiered oversight intensity, centralised governance with distributed execution, and portfolio compliance dashboards.
Beyond three to five AI systems. Dedicated tooling with shared dashboards and standardised processes becomes necessary.
Oversight fatigue manifests through gradual erosion of governance effectiveness. The risk register review becomes a formality. Monitoring dashboards are glanced at rather than analysed. The quarterly governance meeting shortens from two hours to thirty minutes. Each individual erosion is minor; cumulatively, they create gaps that are invisible from the inside because the people closest to the system have normalised them.
Fresh eyes reviews bring personnel not involved in daily operations into periodic deep-dives. An internal auditor, a team member from a different system, or an external consultant reviews monitoring data, the evidence repository, the non-conformity register, and governance meeting minutes with no prior context. Their findings often reveal systemic issues that the operational team has normalised.
Automation of oversight reduces dependence on sustained human vigilance. Automated threshold monitoring, automated compliance checks verifying the deployed system matches the AISDP-documented configuration, and automated anomaly detection provide a baseline that does not degrade over time. Human oversight should focus on activities requiring judgement: interpreting anomalies, making governance decisions, and assessing whether real-world impact aligns with intended purpose.
Budget and staffing continuity is essential. Oversight resources are vulnerable to budget pressure, particularly when the system operates without incident. The annual oversight cost is treated by the AI Governance Lead as a committed operational expense, not a discretionary allocation. If the budget is reduced, the impact on oversight capability is assessed and any resulting compliance risk documented in the risk register.
Tiered oversight recognises that not all high-risk systems require the same intensity. A credit scoring system affecting millions of consumers warrants more intensive oversight than an internal document classification system. Higher-tier systems receive more frequent reviews, dedicated personnel, and more granular monitoring. Lower-tier systems receive scheduled reviews, shared personnel, and standard monitoring configurations. The AI Governance Lead documents tier assignments and reviews them annually.
Centralised governance with distributed execution ensures consistent standards while keeping operational knowledge local. The AI Governance Lead provides central coordination: maintaining the portfolio-level risk register, ensuring consistent standards, and reporting to executive leadership. Day-to-day oversight execution is distributed to the teams closest to each system.
Portfolio-level reporting gives executive leadership a consolidated view: how many systems are deployed, their aggregate compliance status, approaching review deadlines, highest residual risks, and resource constraints. Standardised processes reduce per-system governance overhead through common AISDP templates, evidence taxonomies, non-conformity workflows, and assessment checklists. For one to three systems portfolio management can be manual; beyond three to five systems, a compliance management platform becomes justified.
CTO of Standard Intelligence. Leads platform engineering and contributes to the PIG series technical content.
