No single risk identification technique captures every class of risk that an AI system can present. Article 9 of the EU AI Act demands a risk management system that identifies known and reasonably foreseeable risks, which requires combining five complementary methods. This page covers the methodology, sequencing, and practical tooling for each method, from component-level FMEA through to forward-looking horizon scanning.
Risk identification for AI systems requires a multi-method approach because no single technique captures every class of risk.
Risk identification for AI systems requires a multi-method approach because no single technique captures every class of risk. The recommended approach combines five complementary methods: Failure Mode and Effects Analysis using the IEC 60812 framework, structured stakeholder consultation with affected persons and domain experts, regulatory gap analysis mapping the system against Articles 8 through 15, adversarial red-teaming through simulated attacks, and horizon scanning from external sources including the OECD AI Policy Observatory and the AI Incident Database.
Each method covers blind spots the others miss. FMEA is systematic and granular but misses systemic and emergent risks. Stakeholder consultation surfaces experiential risks but is subjective and recruitment-dependent. Regulatory gap analysis provides comprehensive legal coverage but identifies gaps rather than magnitudes. Red-teaming reveals exploitable weaknesses but is resource-intensive with incomplete coverage. Horizon scanning is forward-looking but speculative and requires curation. Omitting any one method creates a gap in the risk register. The practical difficulty is scoping each method appropriately for the system's complexity and risk tier, not running all five without consuming disproportionate time. For the broader risk management framework, see .
Failure Mode and Effects Analysis is the workhorse of AI risk identification.
Failure Mode and Effects Analysis is the workhorse of AI risk identification. For each system component (data pipeline, feature engineering, model inference, post-processing, user interface), the team enumerates the ways the component can fail, the effects of each failure, and the severity of those effects. In an AI context, failure modes extend beyond traditional software failures to include data drift, concept drift, adversarial manipulation, distributional shift in input data, label noise propagation, and emergent biases.
The practical approach is to start with the system's architecture diagram and walk through each component left to right, asking three questions at each node: what can go wrong with the data entering this component, what can go wrong with the processing within this component, and what can go wrong with the output leaving this component? Each failure mode receives a Risk Priority Number based on severity (one to ten), occurrence (one to ten), and detectability (one to ten). RPNs above a defined threshold (often 100 on a 1,000-point scale, though this is system-specific) trigger mandatory mitigation. For FMEA, dedicated software such as Relyence or Jama Connect provides structured worksheets with automatic RPN calculation and reporting. Smaller teams can use spreadsheet templates; the essential fields are component, failure mode, effect, severity, occurrence, detectability, RPN, and planned mitigation. The completed worksheet is a Module 6 evidence artefact.
Risks to fundamental rights are frequently invisible from within the engineering team.
Risks to fundamental rights are frequently invisible from within the engineering team. Structured consultation with deployers, affected persons, civil society representatives, domain experts, and legal advisors surfaces risks that technical analysis alone will miss. The AI System Assessor documents the consultation, including attendees, the questions posed, and the responses received.
Consultations should be scheduled at multiple points in the development lifecycle, not confined to a single pre-deployment review. Stakeholder consultation requires participation from deployers who know how the system actually behaves in their operational context, affected persons or their representatives, domain experts, and civil society organisations. Each stakeholder concern is traced to a risk register entry or a documented rationale for exclusion. For the methodology on structured stakeholder engagement in the context of fundamental rights, see .
The team systematically maps the system's characteristics against every obligation in Articles 8 through 15, identifying areas where current design, testing, or operational practice falls short.
The team systematically maps the system's characteristics against every obligation in Articles 8 through 15, identifying areas where current design, testing, or operational practice falls short. Each gap becomes a risk entry in the register. The gap analysis should also consider the Master Compliance Questionnaire, working through each applicable question to ensure comprehensive coverage. Credo AI's policy engine can automate the Article-to-control mapping and flag unmatched requirements. The Master Compliance Questionnaire provides the structured walkthrough for manual execution.
The Technical SME leads both FMEA and regulatory gap analysis, as these two methods produce the bulk of the initial risk register before other methods are applied.
A dedicated adversarial testing programme subjects the system to deliberate misuse scenarios, input manipulation, data poisoning attempts, and social engineering of the human oversight layer.
A dedicated adversarial testing programme subjects the system to deliberate misuse scenarios, input manipulation, data poisoning attempts, and social engineering of the human oversight layer. Red-team exercises are conducted with personnel who were not involved in the system's development; this independence is the single most important requirement for the method's effectiveness, ensuring fresh perspectives and reduced confirmation bias.
MITRE ATLAS provides the threat taxonomy, cataloguing known adversarial techniques against AI systems (evasion, poisoning, extraction, inference) with real-world case studies. The red team works through the ATLAS matrix, assessing which techniques are applicable and attempting to execute them in a controlled environment. Microsoft's PyRIT (Python Risk Identification Toolkit) automates portions of this for LLM-based systems, generating adversarial prompts, testing for jailbreak vulnerabilities, and measuring resilience to prompt injection. For non-LLM systems, the red team manually crafts adversarial inputs such as perturbed images for vision models or manipulated feature values for tabular models. Each finding becomes a risk entry recording the attack description, severity, and required mitigation. For how these risks are then scored, see .
The team reviews incidents, enforcement actions, and published risk assessments from comparable AI systems in the same or adjacent domains.
The team reviews incidents, enforcement actions, and published risk assessments from comparable AI systems in the same or adjacent domains. A recruitment screening system provider, for example, should review enforcement actions against algorithmic hiring tools, published bias audits of similar systems, and academic literature on fairness in automated selection. This method identifies risks that the organisation might not have encountered internally.
Three curated sources provide the foundation: the OECD AI Policy Observatory for regulatory developments, the Stanford HAI tracker for policy and research, and the AI Incident Database (maintained by the Responsible AI Collaborative) for real-world incident scenarios. Horizon scanning can be delegated to a compliance analyst, but findings must be reviewed by the Technical SME and AI Governance Lead. The AI System Assessor performs horizon scanning at least quarterly and before every risk register review. For the iterative review cadence that incorporates horizon scanning findings, see .
The EU AI Act's risk framework addresses health, safety, and fundamental rights.
The EU AI Act's risk framework addresses health, safety, and fundamental rights. Reputational risk is not explicitly within the Act's scope, yet it is among the most consequential risks an organisation faces when deploying AI systems. Reputational harm can materialise faster and more destructively than regulatory enforcement, and it affects multiple stakeholder groups in distinct ways.
Five categories of reputational risk merit assessment. Customer reputational risk arises when deployers discover unfair, biased, or opaque outcomes; customer attrition in AI-dependent services tends to be abrupt, and a single well-publicised incident can trigger mass contract termination. Market reputational risk stems from disproportionate media attention to AI fairness incidents, extending beyond direct customers to potential customers, partners, and broader market perception. Regulatory reputational risk is amplified for organisations among the first to receive enforcement actions under the Act. Shareholder and investor reputational risk affects ESG ratings and cost of capital; for publicly listed organisations, the share price impact of an AI incident can dwarf the direct regulatory costs. Employee reputational risk affects recruitment and retention in a competitive labour market, though this risk is frequently underestimated because it is slow-moving and difficult to quantify.
The AI System Assessor completes FMEA and regulatory gap analysis first, as they produce the bulk of the initial risk register.
The AI System Assessor completes FMEA and regulatory gap analysis first, as they produce the bulk of the initial risk register. Red-teaming follows, testing the system against the threat landscape that FMEA may not anticipate. Stakeholder consultation runs in parallel with technical methods, scheduled at multiple points in the development lifecycle. Horizon scanning is performed at least quarterly and before every risk register review.
For all five methods, the AI Governance Lead reviews the consolidated risk register for completeness before the assessment proceeds to risk scoring. Each red-team finding becomes a risk entry. Each stakeholder concern is traced to a risk register entry or a documented rationale for exclusion. The consolidated register forms the input to the scoring and evaluation process described in .
Each method covers blind spots the others miss, so omitting any one creates a gap. Scope each method proportionally to the system's complexity and risk tier. FMEA and regulatory gap analysis form the foundation; red-teaming, stakeholder consultation, and horizon scanning follow.
MITRE ATLAS provides the threat taxonomy. Microsoft PyRIT automates adversarial prompt generation, jailbreak testing, and prompt injection resilience for LLM-based systems. For non-LLM systems, red teams manually craft adversarial inputs such as perturbed images or manipulated feature values.
Assess reputational risk for each identified technical, fairness, and compliance risk across five factors: probability of public discovery, narrative severity, stakeholder sensitivity, containment capability, and likely duration. Document mitigations alongside the technical risk mitigations in the AISDP.
The RPN is the product of severity, occurrence, and detectability scores, each rated one to ten. RPNs above the defined threshold (often 100 on a 1,000-point scale, though system-specific) trigger mandatory mitigation. The threshold must be documented and consistently applied.
Each technique covers blind spots the others miss. FMEA catches component failures but misses systemic risks, stakeholder consultation surfaces experiential risks invisible to engineers, and red-teaming reveals exploitable weaknesses that structured analysis overlooks.
Walk through each component using IEC 60812, enumerating AI-specific failure modes including data drift, adversarial manipulation, and bias. Each failure mode receives a Risk Priority Number from severity, occurrence, and detectability scores.
Deliberate misuse scenarios, input manipulation, and data poisoning attempts using the MITRE ATLAS threat taxonomy, conducted by personnel independent from the development team. Microsoft PyRIT automates portions for LLM-based systems.
Reviewing incidents, enforcement actions, and published assessments from comparable AI systems using three curated sources: OECD AI Policy Observatory, Stanford HAI tracker, and the AI Incident Database.
Reputational harm can materialise faster and more destructively than regulatory enforcement, affecting customers, market perception, regulatory standing, investors, and employee recruitment across five distinct categories.
The AI System Assessor assesses reputational risk for each identified technical, fairness, and compliance risk across five factors: probability of public discovery, narrative severity, stakeholder sensitivity, containment capability, and likely duration. Reputational risk mitigations include proactive transparency measures such as publishing model cards, fairness reports, and audit summaries, alongside crisis communication planning, deployer and affected-person notification procedures, and engagement with civil society and academic reviewers.
CTO of Standard Intelligence. Leads platform engineering and contributes to the PIG series technical content.
Risk Management System
Accuracy, Robustness, Cybersecurity
Technical Documentation Contents
