SLScott's Lab Learn

Domain 3 · 16% of the exam · about 8 questions

3.1AI Risk Assessment and Threat Management

AI risk assessment frames a system's context, measures its risks with repeatable tests and decides how to treat them. Threat catalogs such as OWASP and MITRE ATLAS name what attackers do, and monitoring keeps the picture current after go-live.

See this section as a mind map

What the blueprint lists for this section (3)
  • Risk Assessment Frameworks and Methodologies
  • Threat Identification and Vulnerabilities
  • Risk Monitoring and Reporting

3.1 · lesson 1 of 10

AI RMF MAP: framing AI risk

MAP is the know-what-you-are-dealing-with step of the NIST AI Risk Management Framework. Before anyone tests a model, the team writes down what the system is for, who it touches, what it depends on and what could go wrong. Think of it as a site survey before construction: you cannot price the risk of a building nobody has measured.

MAP builds context in five categories, and its output feeds an initial go or no-go decision. Only a go moves the system on to MEASURE and MANAGE.

Key points

  • The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) was released on 26 January 2023 with four functions: GOVERN, MAP, MEASURE and MANAGE.
  • NIST states that AI RMF 1.0 is being revised under the White House AI Action Plan; version 1.0 remains the published framework as of November 2026.
  • MAP has five categories: context (MAP 1), categorization (MAP 2), capabilities, benefits and costs (MAP 3), component and third-party risks (MAP 4), and impacts (MAP 5).
  • MAP 1.5 requires organizational risk tolerances to be determined and documented, which gives later measurements a yardstick.
  • MAP 4.1 covers the legal and technology risks of components, including third-party data or software and possible infringement of others' intellectual property.
  • MAP 5.1 records the likelihood and magnitude of each impact, beneficial or harmful, using past uses, public incident reports and feedback from outside the build team.
  • After MAP, there should be enough context for an initial go or no-go decision on whether to design, develop or deploy the AI system.

Exam tip

If a question asks where intended purpose, risk tolerance or affected stakeholders get documented, the answer is MAP. A common distractor is MEASURE, which tests risks that MAP already identified.

Source: NIST AI RMF Playbook: MAP function

3.1 · lesson 2 of 10

AI RMF MEASURE: testing AI risk

MEASURE turns the risks found in MAP into evidence: test results, metrics and tracked trends. It runs before release and keeps running in production. Like a vehicle inspection, it checks the car against a known standard and then repeats the check on a schedule.

MEASURE takes the risks MAP found, tests them and tracks them over time, and its results feed MANAGE. MEASURE 4 closes the loop by checking that the measurements still work.

Key points

  • MEASURE uses quantitative, qualitative or mixed-method tools to analyze, assess, benchmark and monitor AI risk, drawing on MAP and informing MANAGE.
  • MEASURE has four categories: methods and metrics (MEASURE 1), trustworthy characteristics (MEASURE 2), tracking risks over time (MEASURE 3), and feedback on measurement efficacy (MEASURE 4).
  • MEASURE 1.1 starts with the most significant risks and requires documenting any risks or trustworthiness characteristics that will not or cannot be measured.
  • MEASURE 1.3 calls for internal experts who were not front-line developers, or independent assessors, to take part in regular assessments.
  • MEASURE 2 evaluates trustworthy characteristics, for example MEASURE 2.7 for security and resilience, with test sets, metrics and tools documented under MEASURE 2.1.
  • Test, evaluation, verification and validation (TEVV) should happen before deployment and regularly in operation, with results formally documented.
  • MEASURE 3.3 sets up ways for end users and affected communities to report problems and appeal outcomes, and feeds those reports into evaluation metrics.

Exam tip

Independence is the keyword. When a question asks how to reduce bias or conflicts of interest in an assessment, choose independent assessors or non-developer experts (MEASURE 1.3), not the build team.

Source: NIST AI RMF Playbook: MEASURE function

3.1 · lesson 3 of 10

ISO/IEC 23894 AI risk process

ISO/IEC 23894 is guidance on running risk management for AI, not a standard you certify against. It reuses the ISO 31000 structure that many risk teams already follow and adds AI-specific advice at each step. If a company already runs ISO 31000, 23894 explains what changes when the system learns from data.

The ISO/IEC 23894 process mirrors ISO 31000: set scope and criteria, assess in three steps, then treat, monitor and report. Communication and consultation run alongside every step.

Key points

  • ISO/IEC 23894:2023, Artificial intelligence, Guidance on risk management, was published in February 2023 by ISO/IEC JTC 1/SC 42, the joint AI committee.
  • It is meant to be used with ISO 31000:2018 and mirrors its clause structure: Clause 4 principles, Clause 5 framework and Clause 6 processes.
  • The Clause 6 process covers communication and consultation, scope, context and criteria, risk assessment, risk treatment, monitoring and review, and recording and reporting.
  • Risk assessment (clause 6.4) runs in three steps: risk identification, then risk analysis, then risk evaluation.
  • Annex A lists common AI-related objectives, Annex B lists AI risk sources, and Annex C maps risk management processes to the AI system life cycle.
  • Following ISO 31000, risk is a deviation from objectives that can be positive or negative, unlike some safety standards where risk means harm only.
  • Certification happens against ISO/IEC 42001, whose clauses 6.1.2 and 6.1.3 require AI risk assessment and AI risk treatment.

Exam tip

Know the pairings: 23894 is AI risk guidance built on ISO 31000, 42001 is the certifiable management system, and 42005 covers impact assessment. A question about what a vendor gets certified against wants 42001.

Source: ISO/IEC 23894:2023 Guidance on AI risk management

3.1 · lesson 4 of 10

AI impact assessments and FRIA

A risk assessment asks what could go wrong for the organization; an impact assessment asks what the AI could do to people and society. ISO/IEC 42005 gives the method, and the EU AI Act makes a fundamental rights impact assessment mandatory for certain deployers. It is the AI cousin of a privacy impact assessment.

ISO/IEC 42005:2025EU AI Act FRIA, Article 27
Who and when
  • Voluntary guidance
  • Any developer, provider or user
  • Across the whole life cycle
  • Legal duty for deployers
  • Public bodies and public services
  • Credit scoring, life and health insurance
What it covers
  • Impacts on people
  • Individuals, groups and society
  • Pairs with 42001 and 23894
  • Six required elements
  • Use, affected groups and harms
  • Oversight, remedies, notify regulator
ISO/IEC 42005 is voluntary guidance any organization can use, while the EU AI Act FRIA is a legal duty for specific deployers of high-risk systems. Both look at effects on people, not only on the organization.

Key points

  • ISO/IEC 42005:2025, AI system impact assessment, was published in May 2025 and guides assessing how AI systems and foreseeable uses affect individuals, groups and society.
  • ISO positions 42005 as complementing ISO/IEC 42001, ISO/IEC 38507 and ISO/IEC 23894, with assessments repeated across the life cycle and updated as needed.
  • ISO/IEC 42001 requires an AI system impact assessment as a planning requirement (clause 6.1.4) and again during operation (clause 8.4).
  • EU AI Act Article 27 requires a fundamental rights impact assessment (FRIA) before first use by public bodies, private providers of public services, and credit or life and health insurance deployers.
  • A FRIA describes the deployer's processes, the period and frequency of use, affected groups, specific risks of harm, human oversight measures, and steps if risks materialize.
  • FRIA duties sit with the high-risk rules, which apply to Annex III systems from 2 December 2027 under Regulation (EU) 2026/1744, the Digital Omnibus on AI.
  • The deployer notifies the market surveillance authority of the results, and since the Digital Omnibus may cross-reference an existing data protection impact assessment (DPIA).

Exam tip

Watch the role: the FRIA is a deployer duty, not a provider duty. If the scenario is a city council or a bank scoring credit, the deployer completes the FRIA before first use.

Source: ISO/IEC 42005:2025 AI system impact assessment

Once risks are found and measured, someone has to decide which ones are acceptable. That decision compares each risk's likelihood and severity with the tolerance leadership set in advance, then picks a response. It is the same choice an insurer makes: avoid it, reduce it, share it or live with it.

Risks inside tolerance can be accepted and monitored. Risks outside it are mitigated, transferred or avoided, or accepted only through a recorded, time-limited exception, and residual risk is documented either way.

Key points

  • NIST AI Risk Management Framework (AI RMF) MANAGE 1 prioritizes risks from MAP and MEASURE, starting with MANAGE 1.1: does the system meet its purpose, and should it proceed.
  • MANAGE 1.3 lists four responses to high-priority AI risks: mitigating, transferring, avoiding or accepting.
  • MANAGE 1.4 documents negative residual risk, defined as the sum of all unmitigated risks, for downstream acquirers and end users.
  • Risk tolerance is set before assessment (MAP 1.5), so a risk above tolerance needs treatment or a documented, approved exception.
  • EC-Council's ADG (Adopt, Defend, Govern) framework control MC-2 classifies each AI system by data sensitivity, autonomy, external exposure, harm potential and business criticality.
  • In ADG, the AI Governance Council sets go or no-go thresholds and owns the Governance Exception Register, which records justification, risk acceptance and expiry.
  • ADG maps MC-2 risk classification to EU AI Act Articles 6 and 7, the NIST MAP function and ISO/IEC 42001 clause 6.1.2.

Exam tip

Risk acceptance is a governance decision with a named owner and an expiry date, never an engineer's call. Transferring risk to a vendor by contract does not transfer accountability for how the AI is used.

Source: NIST AI RMF Playbook: MANAGE function

3.1 · lesson 6 of 10

OWASP Top 10 for LLMs 2026

The OWASP Top 10 for LLM Applications is the community's ranked list of the most critical security risks in apps built on large language models. The 2026 edition reshuffled the order by weighing real incident data alongside practitioner votes. A red team and a release gate can both work from it as a shared checklist.

The ten 2026 entries grouped by where they strike: what goes in, what the model learned from, what comes out, and what leaks or runs up cost. The grouping is a memory aid, not an OWASP category.

Key points

  • The OWASP Top 10 for Large Language Model (LLM) Applications 2026 was published by the OWASP GenAI Security Project in August 2026, replacing the 2025 list.
  • The 2026 order: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Excessive Agency, LLM04 Supply Chain, LLM05 Data and Model Poisoning.
  • Continued: LLM06 Unbounded Consumption, LLM07 Misinformation, LLM08 Hidden Context Exposure, LLM09 Vector and Embedding Weaknesses, LLM10 Improper Output Handling.
  • System Prompt Leakage (LLM07:2025) was renamed and broadened into Hidden Context Exposure (LLM08:2026); the ten risk areas otherwise stayed the same.
  • Excessive Agency rose to third and Improper Output Handling fell from fifth to tenth; the ranking weighted the community vote 75% and incident data 25%.
  • LLM08 says to assume hidden context such as system prompts is discoverable, so it must never hold credentials or serve as the only authorization boundary.
  • The list covers the model as a component; once a model acts with tools and memory, OWASP points to its separate Top 10 for Agentic Applications.

Exam tip

Some exam items still quote 2025 numbering, so anchor on the risk names, not only the numbers. Any answer that stores an application programming interface (API) key in the system prompt is wrong under Hidden Context Exposure.

Source: OWASP GenAI LLM Top 10 2026

When an AI system stops only answering and starts acting, with tools, memory and other agents, new failure modes appear. OWASP's Top 10 for Agentic Applications names them. Think of the difference between a chatbot that gives bad advice and an assistant that actually wires the money.

LLM Top 10 2026

  • Model as a component
  • LLM08 Hidden Context Exposure
  • LLM07 Misinformation

Shared ground

  • LLM01 and ASI01 injection
  • LLM04 and ASI04 supply chain
  • LLM05 and ASI06 poisoning
  • LLM03 maps to ASI02, 03, 08

Agentic Top 10 2026

  • Model as an actor
  • ASI07 Inter-Agent Communication
  • ASI09 Human-Agent Trust
  • ASI10 Rogue Agents
OWASP splits the territory: the LLM list owns the model as a component, and the Agentic list owns the model as an actor with tools, memory and peers. Several risks have a counterpart on each list.

Key points

  • The OWASP Top 10 for Agentic Applications 2026 was released on 9 December 2025 by the OWASP GenAI Security Project, using IDs ASI01 to ASI10.
  • ASI01 Agent Goal Hijack, ASI02 Tool Misuse and Exploitation, ASI03 Identity and Privilege Abuse, ASI04 Agentic Supply Chain Vulnerabilities, ASI05 Unexpected Code Execution.
  • ASI06 Memory and Context Poisoning, ASI07 Insecure Inter-Agent Communication, ASI08 Cascading Failures, ASI09 Human-Agent Trust Exploitation, ASI10 Rogue Agents.
  • ASI04 covers poisoned runtime components such as Model Context Protocol (MCP) servers and tool registries, which the LLM list leaves out of LLM04 Supply Chain.
  • The LLM Top 10 2026 notes that Excessive Agency shows up in agents as ASI02 Tool Misuse, ASI03 Identity and Privilege Abuse and ASI08 Cascading Failures.
  • OWASP's earlier Agentic AI Threats and Mitigations guide uses a separate 15-threat taxonomy (T1 Memory Poisoning, T2 Tool Misuse and so on), not the Top 10 numbering.
  • EC-Council's ADG (Adopt, Defend, Govern) framework routes agentic threats to its Agentic Hardening overlay and to MC-7 Tool and MCP Register, MC-8 Runtime Monitoring and MC-9 Incident Response.

Exam tip

If an agent follows hidden instructions in a document and changes its objective, the answer is ASI01 Agent Goal Hijack, the agentic face of prompt injection.

Source: OWASP Top 10 for Agentic Applications 2026

3.1 · lesson 8 of 10

MITRE ATLAS adversary matrix

MITRE ATLAS is the AI counterpart of MITRE ATT&CK: a public knowledge base of how real adversaries attack AI-enabled systems, organized as tactics (the why) and techniques (the how). Defenders use it to plan red-team tests and detections, and risk teams use it to show a threat is real rather than hypothetical.

A simplified attack path through 7 of the 16 ATLAS tactics. AI Model Access and AI Attack Adaptation are unique to ATLAS, and the other tactics are adapted from ATT&CK.

Key points

  • MITRE ATLAS stands for Adversarial Threat Landscape for AI Systems; it is modeled on MITRE ATT&CK and grounded in real-world attacks and red-team demonstrations.
  • ATLAS release 2026.10 (28 October 2026) holds 16 tactics, 129 techniques, 92 sub-techniques, 44 mitigations and 77 case studies.
  • Since May 2026, ATLAS content uses year-and-month versions such as 2026.10, so older references like v5.x or 84 techniques are out of date.
  • Two tactics are unique to ATLAS: AI Model Access and AI Attack Adaptation, which was called AI Attack Staging until release 2026.08.
  • Example techniques: LLM Prompt Injection (AML.T0051), Training Data Poisoning (AML.T0020), Supply Chain Compromise (AML.T0010), Replicate AI Model Behavior (AML.T0005).
  • Mitigations include Verify AI Artifacts (AML.M0014) and Maintain an AI Bill of Materials (AML.M0023).
  • Each technique is tagged by platform: Predictive AI, Generative AI, Agentic AI or Enterprise.

Exam tip

OWASP lists vulnerability categories to fix; ATLAS describes adversary behavior to detect and emulate. A question about planning a realistic AI red-team scenario points to ATLAS.

Source: MITRE ATLAS

Most AI attacks fall into a few families. Poisoning corrupts what the model learns, injection and evasion trick it at run time, and extraction or inference attacks steal the model or its training data. NIST's adversarial machine learning taxonomy gives each a standard name, so risk registers and red-team reports speak the same language.

Attack on the data or modelAttack through queries
Integrity
  • Poisoning
  • Data, backdoor and model poisoning
  • Split-view and frontrunning
  • Evasion and injection
  • Crafted inputs at inference
  • Direct and indirect prompt injection
Confidentiality
  • Intellectual property theft
  • Stolen weights or datasets
  • Compromised model repository
  • Extraction and inference
  • Replicate model behavior
  • Membership inference, inversion
Sort an attack by what it breaks and how it gets there. Poisoning and evasion break integrity, while theft, extraction and inference attacks break the confidentiality of the model or its training data.

Key points

  • NIST AI 100-2 E2025 (March 2025) is the adversarial machine learning (AML) taxonomy; attacker goals are availability breakdown, integrity violation, privacy compromise, and misuse violations for generative AI.
  • Data poisoning alters training or fine-tuning data to cause targeted errors, bias or backdoors activated by specific inputs, and the effect can persist after the bad data is gone.
  • Split-view poisoning buys expired domains that curated datasets point to; frontrunning poisoning edits crowd-sourced pages just before a scheduled snapshot.
  • Direct prompt injection comes from the user; indirect prompt injection hides instructions in content the model reads, such as web pages, documents or tool outputs.
  • Evasion attacks craft inputs at inference time so a trained model misclassifies them, without changing the model itself.
  • Model extraction replicates a model's behavior through queries; membership inference and model inversion reveal or reconstruct training data.
  • Draft NIST SP 2400-102e2026, released on 9 November 2026 with comments due 18 December 2026, is the proposed successor to AI 100-2 E2025.

Exam tip

Match the attack to the life-cycle stage: poisoning at training, evasion and prompt injection at inference, extraction and inference attacks through query access. Governance answers for poisoning favor provenance and data controls.

Source: NIST AI 100-2 E2025 Adversarial Machine Learning

3.1 · lesson 10 of 10

AI risk monitoring and reporting

An AI risk assessment starts going stale the day the model ships, because data drifts, users find new uses and attackers adapt. Monitoring watches for those changes against agreed thresholds, and reporting moves the signal to the people who must act, up to the board and, for serious incidents, the regulator.

Under EU AI Act Articles 26 and 73 the deployer tells the provider first, and the provider reports to the market surveillance authority on a deadline set by severity. Inside the organization, ADG routes the same event to the AI Governance Council.

Key points

  • NIST AI Risk Management Framework (AI RMF) MEASURE 3 tracks existing, unanticipated and emergent risks; MANAGE 4.1 adds post-deployment monitoring with appeal, override and decommissioning.
  • EC-Council's ADG (Adopt, Defend, Govern) control MC-8 monitors production for abuse, drift, data leakage, unsafe actions, bias emergence and configuration drift, with alert thresholds and response service levels.
  • ADG control MC-10 requires governance reviews with exception handling and board reporting for high-risk systems.
  • ADG measures in three tiers: control effectiveness metrics, outcome metrics such as mean time to detect, and board-level indicators such as AI risk posture.
  • EU AI Act Article 72 post-market monitoring and Article 73 incident reporting attach to high-risk systems, so in practice they follow the high-risk dates, not a separate fixed date.
  • Article 73 serious incident reports go to the market surveillance authority within 15 days, 2 days for widespread infringement or critical infrastructure, and 10 days after a death.
  • A deployer that identifies a serious incident must immediately inform the provider first, then the importer or distributor and the market surveillance authority (Article 26(5)).

Exam tip

Reporting scales with the audience: operational metrics for owners, outcome metrics for risk committees, a few aggregated indicators for the board. The 15, 10 and 2 day Article 73 deadlines are frequent distractor swaps.

Source: EU AI Act Article 73: Reporting of serious incidents

Finished AI Risk Assessment and Threats?