Domain 1: AI Technology Ecosystem and Ethical Considerations

CRAGE Domain 1 Study Guide: AI Technology Ecosystem and Ethical Considerations

Section 1: Detailed Domain Synthesis

1. Fundamentals of AI and the Technology Ecosystem

Defining AI Systems and Core Inference

An Artificial Intelligence (AI) system is defined fundamentally by its ability to infer how to process inputs to produce predictions, content, recommendations, or decisions, operating with varying degrees of autonomy. Unlike traditional software that strictly executes fixed rules pre-written line-by-line by human programmers, an AI system derives its models or outputs from data.

• OECD Definition: The Organisation for Economic Co-operation and Development defines an AI system as a machine-based system that, for explicit or implicit objectives, infers from received input how to generate outputs. In November 2023, the OECD explicitly added "content" to this output list to encompass generative AI technologies.
• EU AI Act (Regulation (EU) 2024/1689): Article 3(1) aligns directly with the OECD definition while adding that systems exhibit varying levels of autonomy and potential adaptiveness post-deployment. Recital 12 emphasizes that the capability to infer is the defining characteristic of an AI system, explicitly excluding systems based solely on human-defined rules for automated operations. Inference techniques include machine learning (learning from data) and logic- and knowledge-based approaches (reasoning from encoded knowledge).
• NIST AI Risk Management Framework (AI RMF): Adapts its definition from the 2019 OECD Recommendation and ISO/IEC 22989:2022 (the standard for AI concepts and terminology).
• Adopt, Defend, Govern (ADG) Framework: Encompasses all AI systems—from classical machine learning to diffusion, multi-modal, and composite systems—noting that a model is only one element of a full AI system.

Machine Learning and Generative AI Paradigm

Machine learning (ML) establishes capability through data analysis rather than hard-coded instruction.

• Predictive AI (PredAI) vs. Generative AI (GenAI): According to NIST AI 100-2 E2025, predictive AI sorts or scores data (e.g., assessing loan risk), whereas generative AI synthesizes novel content (text, images, audio, code). Most industrial AI deployments remain predictive.
• Learning Paradigms:
  • Supervised Learning: Trains on labeled datasets via classification (predicting discrete values) or regression (predicting continuous values).
  • Unsupervised Learning: Identifies patterns in unlabeled data.
  • Semi-Supervised Learning: Combines a small volume of labeled data with large volumes of unlabeled data.
  • Reinforcement Learning: Employs an agent that learns an optimal policy by maximizing cumulative reward signals within an environment.
• Foundation Models & Adaptation: Foundation models are large-scale general-purpose models pre-trained on massive datasets via self-supervised learning. They are typically aligned using Supervised Fine-Tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF). Downstream adaptation includes fine-tuning, retrieval-augmented generation (RAG) applications, or wrapper agents. Flaws in a shared foundation model present systemic supply chain risks across all downstream applications.
• General-Purpose AI (GPAI) Models: The EU AI Act defines a GPAI model as one demonstrating significant generality capable of competently performing a broad spectrum of distinct tasks, capable of integration into varied downstream systems.
• AI Agents: Under the ADG framework, an AI agent is an autonomous system capable of planning, executing multi-step tasks, invoking external tools, and taking direct actions on behalf of a user or enterprise.

Contextual Risk and Use Cases

Governance evaluates the context of use and degree of operational autonomy rather than the underlying algorithm alone. Identical algorithms yield vastly different risk profiles depending on application.

• EU AI Act Categorizations:
  • Banned Practices (Article 5): Prohibits unacceptable risks such as social scoring and manipulative techniques intended to cause significant harm (enforceable since February 2, 2025).
  • High-Risk Areas (Annex III): Includes biometrics, critical infrastructure, education/vocational training, employment/worker management, essential private/public services access, law enforcement, migration/asylum/border control, and administration of justice/democratic processes.
  • Regulatory Timeline: Under the Digital Omnibus on AI (Regulation (EU) 2026/1744), enforcement of Annex III high-risk obligations is set for December 2, 2027 (deferred from August 2, 2026).
• ADG Autonomy Tiers & Harm Taxonomy: Human-in-the-loop (HITL): e.g., An AI copilot drafting an email for human review.Human-on-the-loop (HOTL): e.g., An automated trading agent acting autonomously within pre-set position bounds.Human-out-of-the-loop (HOOTL): e.g., An overnight data-pipeline execution agent reporting summary results in the morning.Harm Types: Distinguishes between Operational Harm (e.g., a copilot fabricating a corporate refund policy) and Societal Harm (e.g., an automated underwriting model systematically denying credit to specific protected demographics).
  1. Human-in-the-loop (HITL): e.g., An AI copilot drafting an email for human review.
  2. Human-on-the-loop (HOTL): e.g., An automated trading agent acting autonomously within pre-set position bounds.
  3. Human-out-of-the-loop (HOOTL): e.g., An overnight data-pipeline execution agent reporting summary results in the morning.
  4. Harm Types: Distinguishes between Operational Harm (e.g., a copilot fabricating a corporate refund policy) and Societal Harm (e.g., an automated underwriting model systematically denying credit to specific protected demographics).

AI Development and Lifecycle Operations

AI lifecycle operations are iterative and dynamic, requiring continuous Test, Evaluation, Verification, and Validation (TEVV) rather than a single pre-launch gate. Maintenance is more intensive than standard software due to continuous data, model, and concept drift.

• NIST AI RMF Lifecycle: Comprises six core stages: Plan and Design, Collect and Process Data, Build and Use Model, Verify and Validate, Deploy and Use, and Operate and Monitor, accompanied by an overarching Use or Impacted by dimension assessing societal/human impacts. Risk management should begin during Plan and Design.
• OECD Lifecycle: Incorporates an explicit Retire or Decommission stage and emphasizes non-sequential, iterative workflows.
• Standards & Controls:
  • ISO/IEC 5338:2023: Establishes standard AI system lifecycle processes by adapting ISO/IEC/IEEE 15288 and 12207 with AI concepts from ISO/IEC 22989 and 23053.
  • ADG Minimum Control 5 (MC-5): Mandates formal, governed change control with logged approvals for any adjustments to prompts, tool boundaries, underlying models, or retrieval data sources.

Technology Stack Governance and Shared Responsibility

An AI system consists of a multi-tiered ecosystem.

• ADG Technology Layers: Comprises model gateways, retrieval and memory components, tool registries, guardrails/harm detection, telemetry/replay engines, training/evaluation pipelines, and input validation checks.
• Model Context Protocol (MCP): An open-source standard enabling AI applications to interface with external databases, software tools, and enterprise workflows. Managed as data access pathways under ADG governance.
• Nine ADG Governance Surfaces: Model, Prompt, Context, Tools, Orchestration, Identity, Safety Layer, Telemetry, and Learning Loop.
• Shared Responsibility Models:

Accountability for system outcomes remains entirely with the deploying organization, regardless of vendor delegation. ADG Minimum Control 12 (MC-12) mandates documented responsibility boundaries and formal vendor due diligence.

Organizational Readiness and Governance Maturity

Establishing baseline organizational readiness is necessary before scaling AI solutions.

• AI Literacy Requirements: EU AI Act Article 4 requires providers and deployers to take measures supporting AI literacy across staff (enforceable since February 2, 2025; revised under Regulation (EU) 2026/1744 on July 27, 2026, to clarify that no specific competency level is legally guaranteed). Article 26 mandates targeted training for personnel overseeing high-risk systems.
• NIST GOVERN and MAP Functions: GOVERN 1.6 mandates an enterprise AI inventory; GOVERN 2.1 assigns explicit roles/responsibilities; GOVERN 2.2 mandates risk training; MAP 1.3/1.5 document organizational goals and risk tolerances.
• ADG Implementation Roadmap (24-Month Strategy):
  • Foundation Phase (Months 0–3): Establishes AI inventory, assigns system owners, classifies system criticality, charters the AI Governance Council (covering MC-1 through MC-3).
  • Control Deployment (Months 3–9)
  • Agentic Readiness (Months 9–15)
  • Maturity & Continuous Assurance (Months 15–24)

2. AI Ethics, Frameworks, and Responsible Execution

Core Trustworthy Characteristics and Trade-offs

Ethical AI governance expands evaluation beyond legal compliance to encompass technical robustness and moral soundness.

• EU AI HLEG Framework: Trustworthy AI requires three concurrent pillars: it must be lawful, ethical, and robust. Meeting statutory obligations alone is insufficient.
• NIST Trustworthy AI Characteristics: Valid and Reliable (the foundational prerequisite for all other characteristics)SafeSecure and ResilientAccountable and Transparent (cross-cutting; applies across all dimensions)Explainable and InterpretablePrivacy-EnhancedFair with Harmful Bias Managed
  1. Valid and Reliable (the foundational prerequisite for all other characteristics)
  2. Safe
  3. Secure and Resilient
  4. Accountable and Transparent (cross-cutting; applies across all dimensions)
  5. Explainable and Interpretable
  6. Privacy-Enhanced
  7. Fair with Harmful Bias Managed
• Distinctions in NIST Terminology:
  • Transparency: Describes what happened within the system.
  • Explainability: Explains how a specific decision or output was generated.
  • Interpretability: Clarifies why the output occurred and what it means to the user contextually.
• Systemic Trade-offs: Optimizing one characteristic may diminish another (e.g., high predictive accuracy vs. model interpretability, or strict differential privacy techniques vs. accuracy on sparse demographic data). NIST mandates that trade-offs be resolved transparently and justified within context.

Key Ethical Challenges and Vulnerabilities

• NIST SP 1270 Bias Taxonomy: Categorizes AI bias into three distinct origins, all of which can manifest without explicit intent or prejudice:Systemic Bias: Stemming from historical or institutional societal norms and practices.Computational and Statistical Bias: Arising from unrepresentative sampling, algorithmic design, or dataset flaws.Human-Cognitive Bias: Resulting from human decision-makers' cognitive heuristics during system design, training, or operational use.
  1. Systemic Bias: Stemming from historical or institutional societal norms and practices.
  2. Computational and Statistical Bias: Arising from unrepresentative sampling, algorithmic design, or dataset flaws.
  3. Human-Cognitive Bias: Resulting from human decision-makers' cognitive heuristics during system design, training, or operational use.
• AI Opacity & Hallucination: Opacity hinders redressing unfair decisions. NIST AI 600-1 formally designates confidently stated false outputs as confabulation (commonly termed hallucination), listing it among 12 key generative AI risk vectors.
• Automation Bias: Over-reliance on automated outputs. EU AI Act Article 14(4)(b) mandates that individuals providing human oversight for high-risk AI must actively guard against automation bias.
• Deepfakes: Under EU AI Act Article 50(4), deployers must explicitly disclose AI-generated or manipulated deepfake content (enforceable starting August 2, 2026).
• ADG Harm Taxonomy: Technical (security vulnerabilities), Societal (bias and fundamental rights violations), Operational (reliability/accuracy failures), and Systemic (emergent AI-to-AI hazards, such as agent collusion).

International AI Governance Initiatives

[ Value & Guideline Standards ]
            /                                   \
   OECD AI Principles                    UNESCO Ethics Rec.
 (47 Adherents, 2019/2024)             (193 States, RAM/EIA Tools)
           |                                     |
           +-----------------+-------------------+
                             |
                   [ Non-Binding Codes ]
                             |
                   G7 Hiroshima Code (2030)
                             |
                [ Legally Binding Frameworks ]
            /                                   \
   EU AI Act (Reg 2024/1689)             CoE Framework Convention
 (Direct EU Regulation)                 (First Binding Treaty, Sept 2024)

• OECD AI Principles: Adopted May 2019, updated May 2024 (47 adherent nations). First intergovernmental standard. Contains 5 values-based principles (inclusive growth/well-being, human rights/democratic values, transparency/explainability, robustness/safety, accountability) and 5 policy recommendations.
• UNESCO Recommendation on the Ethics of AI: Adopted November 2021 by 193 Member States. First global ethics standard. Features 4 core values, 10 principles, and 11 policy areas. Supplies two core execution tools: the Readiness Assessment Methodology (RAM) for national evaluations and the Ethical Impact Assessment (EIA) for individual AI projects.
• EU AI HLEG & ALTAI: Published Ethics Guidelines in April 2019 (7 key requirements), formalized into the Assessment List for Trustworthy AI (ALTAI) in July 2020.
• G7 Hiroshima Code of Conduct: Established October 2023 as a voluntary framework for advanced AI developers.
• Council of Europe (CoE) Framework Convention on AI: Opened for signature September 5, 2024; represents the first legally binding international treaty on AI, human rights, democracy, and the rule of law.

US Department of Defense (DoD) Ethical Principles

Adopted in February 2020 for combat and non-combat applications:

1. Responsible: Personnel exercise appropriate judgment and care while remaining responsible for development, deployment, and operational use.
2. Equitable: Deliberate affirmative steps are taken to minimize unintended bias.
3. Traceable: Methodologies, data sources, design procedures, and documentation are transparent and auditable so relevant personnel understand the technology.
4. Reliable: Capabilities have explicit, well-defined applications, undergoing rigorous testing and validation across their life cycle within those parameters.
5. Governable: Systems retain the capability to detect and avoid unintended consequences, featuring mechanism controls to disengage or deactivate deployed systems exhibiting unintended behavior (e.g., kill switches).

The DoD expanded these principles via six Responsible AI (RAI) tenets in 2021 (governance, warfighter trust, product lifecycle, requirements validation, RAI ecosystem, workforce). A January 2026 strategy memo reframed RAI execution around objectively truthful models and lawful utility.

3. Lifecycle-Embedded Responsible Execution

Lifecycle Integration and the RE3 Model

Responsible AI execution requires embedding controls across all lifecycle phases rather than performing isolated pre-launch reviews.

• EC-Council RE3 Trust Model: Framework defining enterprise AI as Responsible, Ethical, Efficient, and Explainable. Responsible execution dictates that every automated outcome traces directly to an accountable human decision-maker.
• ADG Separation of Duties (MC-3): Mandates strict structural isolation between deployment ownership, technical security validation, and formal governance sign-off authority.
• ISO/IEC 42001:2023: The international management system standard establishing certifiable operational criteria for an AI Management System (AIMS).

Phase-by-Phase Governance Requirements

Phase 1: Problem Definition & Data Processing

• Problem Definition: Conduct impact assessments (ISO/IEC 42005:2025) evaluating socio-technical effects on individuals and groups. Validate intended purpose (NIST MAP 1.1) and systematically evaluate non-AI alternatives (NIST MANAGE 2.1).
• Data Collection & Quality: EU AI Act Article 10(3) requires high-risk training, validation, and testing data to be relevant, representative, complete, and free of errors.
• Bias Mitigation Data Exception: Under Article 4a (added via Digital Omnibus Regulation (EU) 2026/1744), processing special categories of personal data is strictly permitted solely for detecting and correcting systemic bias, subject to explicit security safeguards and immediate deletion post-mitigation.

Phase 2: Design, Development & Testing

• Design Oversight: Integrate human oversight mechanisms directly into model architecture (EU AI Act Article 14; NIST MAP 3.5).
• TEVV & Evaluation: NIST MEASURE 2.1/2.3 requires documenting metrics, test sets, and performance within deployment-like environments.
• ADG Minimum Controls:
  • MC-11: Requires fairness and bias testing using representative evaluation datasets and recognized statistical metrics.
  • MC-4: Mandates validation of performance across all four harm classes (Technical, Societal, Operational, Systemic), accompanied by a signed authorization report prior to production release.

Phase 3: Deployment, Monitoring & Maintenance

• Deployment Sign-off: Enforce formal ADG Deploy & Authorize workflows requiring explicit governance approval, verified deployment readiness, and validated rollback routines.
• Transparency Mandates: EU AI Act Article 50 obligates deployers to inform natural persons whenever they interact directly with an AI system (enforceable August 2, 2026).
• Continuous Monitoring: ADG MC-8 mandates real-time telemetry tracking for abuse, performance drift, data exfiltration, safety breaches, configuration drift, and emergent bias. High-risk systems require post-market monitoring plans (EU AI Act Article 72) and formal serious incident reporting (Article 73).

Phase 4: Documentation, Continual Improvement & Retirement

• Auditability: EU AI Act Articles 11, 12, and 13 mandate technical documentation, automated event logging, and clear operational instructions for downstream deployers. ADG Telemetry surfaces guarantee that model inferences remain fully replayable and auditable.
• Decommissioning: Safely phase out systems without raising secondary security or operational risks (NIST GOVERN 1.7 / MANAGE 2.4). Preserve all audit logs, clear system records, update the inventory, and integrate operational lessons back into enterprise procurement policies.

Sustainability, Accessibility, and Social Responsibility

• Environmental Footprint: The International Energy Agency (IEA) reports data center power consumption grew 17% in 2025 and projects demand to expand from 485 TWh to roughly 950 TWh by 2030 (~3% of global electricity demand). Despite these macro demands, efficiency per individual AI task improves by at least an order of magnitude annually.
• Regulatory & Framework Standards: EU AI Act Annex XI requires GPAI model developers to document estimated energy consumption. Article 95 encourages codes of conduct prioritizing energy-efficient architectures, team diversity, digital accessibility, and workplace equity.

Section 2: Short-Answer Quiz

1. What specific operational characteristic distinguishes an AI system from traditional software according to EU AI Act Recital 12 and the OECD definition?
2. Under the Digital Omnibus on AI (Regulation (EU) 2026/1744), what is the updated enforcement date for Annex III high-risk AI obligations?
3. How does NIST SP 1270 categorize the three origins of bias in AI systems?
4. What are the three core elements required for Trustworthy AI according to the EU High-Level Expert Group (AI HLEG)?
5. In the ADG framework's shared responsibility model for Software as a Service (SaaS) or embedded AI, which governance surfaces remain under the exclusive control of the deploying organization?
6. Which international AI instrument represents the first legally binding international treaty on artificial intelligence?
7. What are the five ethical AI principles established by the US Department of Defense (DoD) in February 2020?
8. Under EU AI Act Article 4a (introduced via the Digital Omnibus), under what specific condition is processing special category personal data legally permitted during model development?
9. How does NIST distinguish between transparency, explainability, and interpretability within its trustworthy AI framework?
10. What specific requirement does ADG Minimum Control 3 (MC-3) enforce to maintain structural accountability within AI governance?

Section 3: Quiz Answer Key

1. Answer: The primary distinguishing characteristic of an AI system is its capability to infer how to generate outputs (such as predictions, content, recommendations, or decisions) from inputs or data. Unlike standard software that executes explicit rules defined entirely by human developers, AI systems derive operational logic or models autonomously. Rule-based systems executing hard-coded operations are explicitly excluded from this definition.
2. Answer: Under Regulation (EU) 2026/1744 (the Digital Omnibus on AI), the enforcement date for Annex III high-risk AI obligations was deferred to December 2, 2027. This updates the original timeline previously set for August 2, 2026. The shift provides deployers and providers additional time to comply with high-risk system obligations.
3. Answer: NIST SP 1270 categorizes AI bias into three distinct categories: systemic bias, computational and statistical bias, and human-cognitive bias. Systemic bias arises from historical or institutional societal norms, computational bias stems from unrepresentative sampling or algorithmic flaws, and human-cognitive bias originates from cognitive heuristics applied during development or operation. All three can occur without explicit intent or prejudice.
4. Answer: The EU High-Level Expert Group (AI HLEG) specifies that Trustworthy AI must simultaneously satisfy three fundamental requirements: it must be lawful, ethical, and robust. Achieving legal compliance alone is insufficient if the system lacks ethical alignment or technical/operational robustness. These three elements form the foundational criteria for trustworthy system deployment.
5. Answer: When deploying SaaS or embedded AI systems, the organization retains direct ownership of Context, Identity, Telemetry, and its overarching Governance Policy. The vendor assumes operational responsibility for the underlying model, orchestration layer, tools, runtime environment, and learning loop. Despite this vendor operational shift, overall legal and operational accountability for system usage remains entirely with the deploying organization.
6. Answer: The Council of Europe Framework Convention on AI is the first legally binding international treaty governing artificial intelligence, human rights, democracy, and the rule of law. Opened for signature on September 5, 2024, it differs from non-binding recommendations published by bodies such as the OECD or UNESCO.
7. Answer: The five US Department of Defense ethical AI principles are Responsible, Equitable, Traceable, Reliable, and Governable. Responsible mandates human oversight and accountability; Equitable requires deliberate steps to minimize unintended bias; Traceable requires auditable methodologies and data context; Reliable demands testing across explicit use cases; and Governable ensures systems can be disengaged or deactivated if unintended behaviors manifest.
8. Answer: Processing special categories of personal data under Article 4a is permitted strictly when it is strictly necessary for the detection and correction of systemic bias in AI models. This processing is subject to rigorous technical safeguards to ensure privacy protections. Furthermore, the sensitive data must be permanently deleted immediately after the bias detection and correction process is completed.
9. Answer: NIST defines transparency as describing what happened within an AI system, including its data sources and design architecture. Explainability addresses how a specific output or decision was generated by the model. Interpretability clarifies why the output occurred and what it contextually means to the human operator or affected end user.
10. Answer: ADG Minimum Control 3 (MC-3) enforces separation of duties to prevent concentration of authority in single roles or teams. It explicitly isolates deployment ownership, security validation responsibilities, and governance approval authority across separate entities. This structural boundary guarantees that no single team can build, validate, and unilaterally deploy an AI system into production without independent oversight.

Section 4: Essay Practice Questions

1. Compare and contrast the scope, legal nature, and operational implementation mechanisms of the OECD AI Principles, the UNESCO Recommendation on the Ethics of AI, and the Council of Europe Framework Convention on AI.
2. Explain how the ADG framework's shared responsibility model reallocates control surfaces across Homegrown, Foundation Model API, and SaaS deployment architectures. Analyze how organizational accountability remains constant despite these technical shifts.
3. Analyze the technical and ethical trade-offs between predictive accuracy, model interpretability, and privacy preservation in high-risk AI deployments. Detail how governance frameworks recommend resolving these competing priorities.
4. Detail the four harm classes defined in the ADG framework (Technical, Societal, Operational, Systemic). Provide hypothetical failure scenarios for each class within an autonomous AI agent context, and outline the specific controls required to mitigate them across the AI lifecycle.
5. Discuss the regulatory evolution of AI literacy, transparency, and human oversight requirements under the EU AI Act (including modifications made via the Digital Omnibus on AI, Regulation (EU) 2026/1744). What concrete processes must an enterprise implement to maintain compliance through 2027?

Section 5: Comprehensive Glossary