Competence, Roles and the AI Architect

The operating model of Chapter 12 names teams and gates, and the forums of Chapter 13 name the bodies that decide; this chapter names the people in them. Large organisations adopting AI discover that the scarce resource is not compute or budget but competence: the ability to decide which model is good enough, to read a model card critically, to design an oversight path a caseworker will use, and to tell a supervisory authority what the system does. This chapter defines the AI architect and the delivery and assurance roles around it, sets out the AI literacy obligation of Article 4 of Regulation (EU) 2024/1689, the AI Act, and offers a competence framework as code before asking whether to hire or to upskill.

Role map placing the AI architect between the architecture roles, the delivery roles, the assurance roles and human oversight staff

Figure 14.1: The AI role map. The AI architect translates between the architecture disciplines and the delivery, assurance and oversight roles (the table below).

Group Roles What the AI architect provides
Architecture roles Enterprise, solution, data and security architect (Input to the AI architect)
Delivery roles AI engineer; ML engineer; data steward Decision records and boundaries; evaluation strategy; data requirements
Assurance roles Model risk manager; AI compliance officer; data protection officer Evidence; obligation mapping; impact assessment input
Oversight Human oversight staff Oversight path

Table for Figure 14.1: Roles around the AI architect.

The AI architect

Every organisation that has industrialised AI has someone doing the AI architect's job, because AI systems cut across the boundaries of the existing architecture roles. The enterprise architect owns the target landscape, the solution architect one system, the data architect the data estate and the security architect the threat model. An AI system needs all four and adds a fifth concern none of them owns: the model as a derived, probabilistic, degrading component whose behaviour must be evaluated rather than specified.

What the role adds

The AI architect owns the architectural treatment of that fifth concern. In practice this means owning the AI Decision Records for a portfolio of use cases, applying the organisation's reference architecture to each, defining the evaluation strategy that decides whether a model is acceptable, contributing technical input to risk classification (Chapter 8), and designing the human oversight path that Article 14 of the AI Act requires providers to build into high-risk systems and Article 26(2) requires deployers to staff. The AI architect also holds the three tensions of Chapter 1, speed against control, autonomy against accountability and innovation against sovereignty, and makes them explicit in each decision rather than leaving them to whoever shouts loudest. This chapter treats the role as a governance position, accountable for the decisions rather than for the tooling.

Huyen (2025) describes AI engineering as the discipline of building applications on foundation models, in which evaluation, prompt design, retrieval and model choice matter more than training (68, Huyen, 2025). The AI architect works one level above this, deciding the application's shape, boundaries, controls and the evidence it must produce, and is accountable for the practice of AI Architecture, as Chapter 1 defined it, in a given portfolio.

Relationship to the other architecture roles

The AI architect does not replace the established roles: they keep their domains and the AI architect owns the seams between them.

Role Keeps responsibility for Relies on the AI architect for
Enterprise architect Capability map, target landscape Which capabilities AI changes; AI principles in the framework
Solution architect Design of the embedding application Model selection, evaluation criteria, oversight design, AI Decision Records
Data architect Data products, contracts, lineage Training and retrieval data requirements, drift signals
Security architect Threat model, identity, zero trust AI-specific threats, guardrail policy, agent permissions
Platform architect Internal developer platform Gateway, registry, evaluation service requirements

In a small organisation one person covers the AI and solution architect roles for a use case; in a large one the AI architect sits in the hub of Chapter 12's model, works in the enabling team and reviews at the readiness gate. Architecture as Code (Chapter 18) described the competency development loop for infrastructure specialists, and the AI architect follows the same loop with a different syllabus (66, Nordqvist, 2025).

Delivery and assurance roles

An industrialised programme needs roles that build and roles that verify, and it needs them to be distinct. Separation of duties is a regulatory expectation in financial services (3, European Union, 2022) and a sound practice everywhere, because the person who tuned a model to pass an evaluation is the wrong person to certify that it passed.

Roles that build

The AI engineer builds applications on foundation models: retrieval pipelines, prompt configurations, tool integrations, evaluation harnesses and guardrails, a discipline Huyen (2025) distinguishes from machine learning engineering (68, Huyen, 2025). The ML engineer trains, fine-tunes and serves models and understands why a model's metrics moved; Huyen (2022) set out this role's iterative lifecycle from data to monitoring (67, Huyen, 2022). Most large organisations need many more AI engineers than ML engineers, and a job advertisement should not confuse them. The data steward, usually a business role, owns the quality, lineage and lawful basis of the data that feeds both.

Roles that verify

The model risk manager, a role borrowed from banking, owns the risk classification, risk register, evaluation thresholds and independent validation. The AI compliance officer, where the role exists, maps the AI Act and related instruments to controls and evidence, maintains the AI inventory and prepares for conformity assessment and market surveillance. The data protection officer keeps the independence the GDPR guarantees and owns the data protection impact assessment. All three sit outside the delivery line, as Chapter 12 argued, and consume evidence from pipelines rather than produce it.

Human oversight staff

The role most often forgotten is the one the law names most explicitly. Article 26(2) of the AI Act requires deployers of high-risk systems to assign human oversight to natural persons who have the necessary competence, training and authority, and the support to exercise it (1, European Union, 2024). Article 14 requires providers to design the system so that those persons can understand its capabilities and limitations, remain aware of the tendency to over-rely on its output, interpret it correctly, decide not to use it, and intervene or stop it. They should not need to be AI specialists, but they need role-specific competence, documented authority to override and an interface that makes overriding easy.

New and changing roles

Four roles have appeared that most job catalogues do not yet contain, each because a usage mode created work no existing role owned. The AI system owner is the accountable person for one system in use, described in the worked example below; the deployer duties of Article 26 give the role legal content. The agent operator runs Mode 5 agents in production, between site reliability engineering and human oversight: the operator works the approval queue for actions above the agent's threshold, reviews overreach signals such as calls to tools outside the agent's manifest, and may revoke the agent's credentials, which cuts off its further actions because it acts only through its own scoped identity. An agent with write access to production needs someone whose job is to stop it, not someone who stops it when they happen to notice.

The skills curator owns a governed catalogue of the reviewed instructions and scripts that agents and agentic work tools may load (105, Agent Skills, 2026), reviewing submissions for quality, data classification and permissions and retiring orphaned skills; without this part-time role a catalogue fills with abandoned and overlapping skills within a year. The evaluation engineer maintains the evaluation sets, graders and thresholds that gate each release and runs red-team exercises; once a programme has several systems the role separates from the AI engineer, for the separation-of-duties reason above, and reports to model risk or the platform team.

Worked example: AI system owner role description

Illustrative example. The AI system owner is, in Chapter 12's terms, the accountable executive for the use case or a senior manager to whom that executive delegates day-to-day ownership in writing. Nordvik Region keeps the role description below in the architecture repository, where the inventory tooling checks that every registered system has a named owner.

# Role description: AI system owner
Organisation: Nordvik Region | Role ID: ROLE-AI-OWNER | Version: 2026.3
Instance: Benefits-triage system (INV-0042), high-risk, Annex III point 5(a)
Holder: Head of Benefits Unit | Reports to: Director of Social Services

## Accountabilities
1. Approve the intended purpose and prohibited uses in the AI Decision Record.
2. Accept or reject the system at the readiness review and each major update.
3. Ensure human oversight is assigned to caseworker team leaders with the
   competence, training and authority Article 26(2) requires.
4. Ensure risks and serious incidents are reported to the provider and the
   market surveillance authority (Article 26(5)).
5. Sponsor the DPIA and the fundamental rights impact assessment (Article 27).

## Decision rights
- May suspend the system at any time; team leaders may suspend it for their team.
- May not change evaluation thresholds; these are set by model risk.

## Does not
- Configure the platform, write prompts or build integrations.

## Competence and literacy (AI Act Article 4)
- Practitioner level in human_oversight_design and eu_ai_regulation.
- Refresh every 12 months and at each major provider update.

Each accountability that comes from a deployer obligation names its article, so the readiness review and an auditor can follow the obligation to a person. The "may not" and "does not" lines matter as much as the duties, because they keep the owner out of the build and the threshold setting.

AI literacy as an obligation

Article 4 of the AI Act, as amended by the Digital Omnibus on AI, Regulation (EU) 2026/1744, requires providers and deployers to take measures to support the development of AI literacy among their staff and other persons dealing with AI systems on their behalf, without guaranteeing any specific level for any individual, taking into account their technical knowledge, experience, education and training, the context of use and the persons on whom the systems are used (27, European Union, 2026). The 2024 text required measures to ensure a sufficient level (1, European Union, 2024). The obligation still applies to every AI system and has applied since February 2025, and the Commission confirms that no particular governance structure or certificate is required (133, European Commission, 2026).

The amendment changes less in architecture than it appears to. It lowers the bar for a generic programme but leaves untouched the staffing duty of Article 26(2) and the design duty of Article 14, so for high-risk systems competence must still be shown and Article 4 becomes the floor rather than the main lever. An organisation that dismantles role-specific records because the general duty softened cannot evidence the oversight duties that did not.

The architectural reading is that literacy is role-specific and must be evidenced: a caseworker who overrides a system needs to know where it fails, an AI engineer needs to understand prompt injection, and a board member needs to know what a deployer is accountable for, so one generic e-learning module satisfies none of them. Literacy therefore belongs in the same repository as the roles: each role carries a literacy requirement, each oversight officer has a training record, and the production readiness review checks that the record is current before a high-risk system goes live. For public authorities the duty is sharper, because the literacy of officials who decide about citizens is part of the answer to a complaint or an appeal.

Management-body competence

Two instruments carry competence duties sharper than Article 4 that reach the top of the organisation. Article 20(2) of the NIS 2 Directive requires members of the management bodies of essential and important entities to follow training so that they can identify risks and assess cybersecurity risk-management practices (4, European Union, 2022). Article 5(4) of DORA requires members of a financial entity's management body to keep sufficient knowledge and skills to understand and assess ICT risk (3, European Union, 2022), and Solvency II adds fit-and-proper requirements for those who run an insurer or hold key functions (184, European Union, 2009). None of these provisions mentions AI, but AI providers and agents are ICT risks. The framework therefore needs a board-level learning path, half a day a year on the AI portfolio, its highest-risk systems and its last incidents, with the record kept as evidence: the forums of Chapter 13 decide, and this path makes sure their members can.

Across the usage modes

What literacy means under Article 4 depends on the usage modes of Chapter 2, which differ in what the user must understand and what the team running the mode must master.

Usage mode Literacy the users need Competence the operating team needs
Mode 0 Ambient AI Recognise AI-generated summaries; check the source before relying on them Discovery of supplier-enabled features, awareness material
Mode 1 Conversational assistants What may be entered; output can be fluent and wrong Acceptable-use policy, data classification, tenant configuration
Mode 2 Embedded workplace assistants The assistant reaches everything the user can reach Permission hygiene, sensitivity labelling, audit log review
Mode 3 Agentic work tools Read each proposed command before approving; recognise injected instructions Sandboxing, permission scoping, secret handling
Mode 4 Built AI applications Intended purpose, limitations and override procedure (oversight staff) Evaluation, retrieval and prompting, guardrails
Mode 5 Autonomous production agents What the agent may decide alone and how to stop it Permission manifests, monitoring of delegated actions, kill-switch drills

Mode 0 is the case most literacy programmes miss: nobody chose the tool, so nobody is trained for it, yet staff read AI search summaries and automatic meeting notes every day. In Modes 1 to 3 the organisation is a deployer of a product someone else built, so literacy concerns users and the people who configure the product. In Modes 4 and 5 it becomes the provider whenever it develops an AI system and puts it into service under its own name, within the meaning of Article 3(3) of the AI Act, and the competence requirement moves into the delivery and operations teams.

The competence curve is therefore steep: Modes 0 to 2 ask for critical reading from many people, Mode 3 is the first in which a person approves actions a tool then executes and so needs engineering judgement, and Modes 4 and 5 add skills few people need but no course can supply. The readiness review should treat a missing operator or evaluation engineer for a higher mode as a blocking finding.

Worked example: individual competence and literacy record

Illustrative example. The record below belongs to a caseworker team leader in Nordvik Region who oversees the benefits-triage system. The learning management system writes it and the readiness check reads it; it holds what the person completed and demonstrated and no scores beyond what the check needs, which keeps the processing of employee data proportionate under the GDPR.

# competence/records/nr-emp-20417.yaml
competence_record:
  person_ref: NR-EMP-20417
  framework_version: 2026.2
  roles:
    - role: human_oversight_officer
      system: INV-0042 benefits-triage
      authority: may override any ranking and suspend the system for own team
      regulatory_reference: AI Act Article 26(2)
  literacy:
    reference: AI Act Article 4
    level_demonstrated: practitioner
  competences:
    - area: system_limitations
      evidence: quiz on instructions for use, provider version 2.3
    - area: override_procedure
      evidence: sandbox exercise, 3 of 3 seeded wrong rankings overridden
      assessor: senior caseworker
    - area: automation_bias_awareness
      evidence: case review workshop
  validity:
    valid_until: 2027-09-05
    revalidate_on: [provider major version, change of override interface]
  readiness_check: {status: current, last_checked: 2026-10-30}

The record links each competence to evidence of practice rather than to course attendance alone. The revalidate_on triggers tie literacy to changes in the system, so a major provider update expires the record and prompts retraining.

Competence frameworks and learning paths

A competence framework turns the roles above into something that can be assessed, planned and reported. Ours follows the four levels of Architecture as Code (Chapter 18), awareness, practitioner, advanced and expert, and attaches to each role its competences and literacy requirement. As a YAML file, a learning management system can import it, a readiness check can query it, and a change to it is reviewed like any other. The excerpt records for each role the usage modes it covers, so that adopting a new mode raises the question of who holds the roles it needs.

# competence/ai-competence-framework.yaml
framework:
  version: 2026.2
  levels: [awareness, practitioner, advanced, expert]
  literacy_reference: AI Act Article 4
  roles:
    - role: ai_architect
      modes: [3, 4, 5]
      competences:
        - {area: reference_architecture, required_level: expert}
        - {area: evaluation_strategy, required_level: advanced}
        - {area: human_oversight_design, required_level: advanced}
      literacy_requirement: advanced
    - role: ai_engineer
      modes: [3, 4, 5]
      competences:
        - {area: retrieval_and_prompting, required_level: advanced}
        - {area: evaluation_engineering, required_level: advanced}
      literacy_requirement: practitioner
    - role: human_oversight_officer
      modes: [4, 5]
      competences:
        - {area: system_limitations, required_level: practitioner}
        - {area: override_procedure, required_level: practitioner}
        - {area: automation_bias_awareness, required_level: practitioner}
      evidence_required_before_go_live: true
      learning_path: paths/oversight-officer.md
    - role: agent_operator
      modes: [5]
      competences:
        - {area: delegated_action_monitoring, required_level: advanced}
        - {area: agent_incident_response, required_level: advanced}
  assessment: {cadence_months: 12, method: practical_demonstration_and_review}

Learning paths that use the platform

The most effective learning paths are built on the platform's golden paths. An AI engineer learns evaluation engineering by writing the evaluation set for a sandbox use case and running it through the real evaluation service; an AI architect learns risk classification by classifying a real intake record and defending it to the model risk manager. These are the ingredients Architecture as Code (Chapter 18) recommended for infrastructure competence, with a new syllabus.

Nordvik Region's path for oversight officers, paths/oversight-officer.md, illustrates the pattern. It takes about nine hours over three weeks in five modules: what the system does and does not do, read from the provider's instructions for use under Article 13; automation bias and when to doubt the rank; override and suspension on the real interface, in a sandbox tenant with a model configured to produce seeded wrong rankings; reporting an observation as an incident; and informing applicants that they are subject to the system, under Article 26(11). Each module ends in a demonstration that writes an entry to the competence record. It is practical because Article 14 describes oversight in terms of abilities, which only a demonstration can evidence. The framework also gives the enabling team of Chapter 12 its exit criterion: an engagement ends when the team has people at the required level for the roles its use case needs.

Career paths and assessment

A framework that only measures people reads as a compliance exercise; it earns its keep when it also shows where they can go. Three paths need deliberate design: a specialist ladder, so that experts progress without line management; moves between building and verifying, recorded with a cooling-off period of, for example, twelve months before the person validates a system they helped to build; and a path for operational staff. Caseworkers who become skilled oversight officers need that skill recognised, or oversight is felt as a burden added to a full caseload.

Assessment is by practical demonstration reviewed by a more experienced colleague. Results are personal data that could be used to judge performance, so their purpose, retention and access must be fixed before the first assessment, in many countries with employee representatives.

Hiring versus upskilling

The market for AI competence is thin and expensive, and public authorities compete in it at a pay disadvantage. The strategic answer is to upskill more than to hire, for three reasons. Existing staff know the domain, the data and the politics; most roles above are existing roles with an AI syllabus added; and the enabling team is itself an upskilling engine, since Skelton and Pais (2019) describe facilitating as the mode in which capability is transferred rather than delivered (69, Skelton & Pais, 2019).

Hiring is for the scarce specialisms that upskilling cannot supply fast enough: inference engineering for self-hosted models, fine-tuning, AI security research and, in the first year, the AI architect who seeds the framework. Consultants can fill these if the engagement leaves competence behind, and public administrations can share specialists through the shared-service bodies of Chapter 12. Forsgren, Humble and Kim (2018) found that high-performing organisations invest in continuous learning rather than one-off training (70, Forsgren et al., 2018); for AI, where models, threats and law all move within a year, that finding is a planning assumption. A useful rule for groups is that competences to build and run the shared platform sit centrally, while competences to oversee a system in a market, and to explain it in that market's language and to its supervisor, sit with the entity that uses it.

Nordvik Region and Hansa Mutual

Nordvik Region's examples serve one purpose: that the staff who oversee the benefits-triage system can understand, doubt and override its ranking. Its constraint is capacity within public-sector rules. It hires the AI architect within fixed pay scales, buys inference and evaluation skills through framework agreements, makes unit heads system owners and upskills caseworkers for oversight. With no Mode 5 agents and Mode 3 confined to a small platform team, it needs no agent operators and only a part-time skills curator. Its Article 4 programme (27, European Union, 2026) reaches about 6,000 employees in one language, and its records must withstand public-access requests and appeals.

Hansa Mutual's problem is breadth and variation. Its engineers need deep skills for Modes 3 and 5, its claims handlers need oversight skills, and its literacy programme runs in German, Dutch, Swedish, Polish, Norwegian and English. The group owns the framework and platform roles, the claims-triage agent's operator rota sits at the Polish service centre, and each entity localises training. Article 4 binds the EU entities, while Norway applies it as group policy until EEA incorporation (187, EFTA, n.d.) and the UK and Swiss entities as policy under their own regimes. Training records are personal data, and national rules on employee data under Article 88 of the GDPR (2, European Union, 2016) give one record schema a different access policy in each country. Nordvik must build scarce skills inside public-sector constraints; Hansa must deliver consistent skills across languages, entities and labour regimes.

Summary

The AI architect owns the seams between the architecture roles, between building and verifying, and between the system and the people who oversee it. Around it, a programme needs builders and, outside the delivery line, verifiers. The most overlooked role is the human oversight officer whom Article 26(2) of the AI Act requires deployers to assign, and new roles follow the usage modes: the AI system owner, the agent operator, the skills curator and the evaluation engineer.

Article 4, as amended by the Digital Omnibus on AI, now asks for measures that support AI literacy rather than ensure it. That lowers the bar for a generic programme but not for oversight, which Articles 14 and 26(2) still govern, while NIS 2, DORA and Solvency II add duties for management bodies. A competence framework in YAML makes all of this checkable at the readiness gate. Upskilling the people who already know the domain is the strategy; hiring is for scarce specialisms and the first AI architect.

Competent people in well-defined roles still have to choose to use the systems they are given. Chapter 15 therefore turns to adoption: how diffusion of innovation and Kotter's model of leading change apply to AI, how employee consultation becomes a design review, and how an adoption dashboard turns use into evidence.

Sources

  1. Huyen, C. (2025). AI Engineering: Building Applications with Foundation Models. Sebastopol, CA: O'Reilly Media. ISBN 978-1-0981-6630-4. (68, Huyen, 2025)
  2. Nordqvist, G. (2025). Architecture as Code. Companion volume to this book, published at aac.geon.se. (66, Nordqvist, 2025)
  3. Huyen, C. (2022). Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications. Sebastopol, CA: O'Reilly Media. ISBN 978-1-0981-0796-3. (67, Huyen, 2022)
  4. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
  5. European Parliament and Council of the European Union. Regulation (EU) 2026/1744 amending Regulations (EU) 2024/1689, (EU) 2018/1139 and (EU) 2023/1230 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI). Official Journal of the European Union, 24 July 2026. (27, European Union, 2026)
  6. European Commission (2026). AI literacy: questions and answers. Shaping Europe's digital future, updated 20 October 2026. (133, European Commission, 2026)
  7. European Parliament and Council of the European Union. Directive (EU) 2022/2555 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive). Official Journal of the European Union, 27 December 2022. (4, European Union, 2022)
  8. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union, 27 December 2022. (3, European Union, 2022)
  9. European Parliament and Council of the European Union (2009). Directive 2009/138/EC on the taking-up and pursuit of the business of Insurance and Reinsurance (Solvency II) (recast). Official Journal of the European Union, L 335, 17 December 2009. (184, European Union, 2009)
  10. Skelton, M. & Pais, M. (2019). Team Topologies: Organizing Business and Technology Teams for Fast Flow. Portland, OR: IT Revolution Press. ISBN 978-1-942788-81-2. (69, Skelton & Pais, 2019)
  11. Forsgren, N., Humble, J. & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. Portland, OR: IT Revolution Press. ISBN 978-1-942788-33-1. (70, Forsgren et al., 2018)
  12. European Parliament and Council of the European Union. Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (2, European Union, 2016)
  13. Agent Skills (2026). Agent Skills Overview and Specification. agentskills.io. (105, Agent Skills, 2026)