Anti-Patterns

This chapter catalogues the ways in which enterprise AI programmes fail. Chapter 18 showed five composite cases that worked, at least eventually, and Chapter 19 one real incident in which several controls failed at once; this chapter is about the far larger population of programmes that fail more quietly. Each anti-pattern is described through its symptom and cause, its consequence and its remedy, so that you can recognise it early, name it without causing offence, and point to the chapter that holds the corrective practice. The catalogue extends the anti-patterns in Architecture as Code (Chapter 26B) (66, Nordqvist, 2025), many of which reappear here in AI-specific form.

Figure 20.1: The anti-pattern map links the three clusters to the lifecycle stages where symptoms first appear. The table below places each of the fifteen anti-patterns.

Anti-pattern	Cluster	First visible at
Pilot purgatory; shadow AI	Portfolio and strategy	Intake
Model as the product; vendor-defined architecture	Portfolio and strategy	Design
Centre of excellence that says no; accountability left to supplier	Governance and compliance	Intake
Data protection as afterthought; platform left to others	Governance and compliance	Design
Governance theatre; compliance by checkbox	Governance and compliance	Assure
Unbounded agents; prompt sprawl	Engineering and operations	Build
Evaluation by demo	Engineering and operations	Assure
GPU hoarding	Engineering and operations	Operate
The model nobody retires	Engineering and operations	Retire

Table for Figure 20.1: The fifteen anti-patterns by cluster and stage.

How to read this catalogue

An anti-pattern is not a one-off mistake but a solution that looks reasonable, is adopted repeatedly, and produces the same bad outcome every time. Sculley and colleagues (48, Sculley et al., 2015) described the first generation for machine learning as hidden technical debt such as entanglement and pipeline jungles. Paleyes, Urma and Lawrence (55, Paleyes et al., 2022) found the same deployment obstacles recurring across case studies. The anti-patterns here are their organisational and architectural descendants, grouped as portfolio, governance and engineering anti-patterns.

Across the usage modes

Several anti-patterns are mode errors: a system in one usage mode of Chapter 2 is governed as if it belonged to another.

Usage mode	Characteristic anti-pattern	Remedy
Mode 0 Ambient AI	Supplier-enabled features and automatic meeting notes stay outside the inventory	Discover and register; accept, configure or disable per feature (Chapter 9)
Mode 1 Conversational assistants	Prohibiting use without a sanctioned alternative, which produces shadow AI	Approved service behind the gateway, plus Article 4 literacy (Chapter 14)
Mode 2 Embedded workplace assistants	Treating the assistant as harmless although it acts with the user's full permissions	Permission and labelling clean-up before enablement; audit logging
Mode 3 Agentic work tools	Granting a tool that runs commands the trust appropriate to a chat assistant	Sandboxed execution, scoped credentials, approval for irreversible actions
Mode 4 Built AI applications	Shipping without an AI Decision Record (AIDR), so purpose and duties are never settled	AIDR at intake, evaluation gates before release (Chapters 9 and 13)
Mode 5 Autonomous production agents	Promoting a Mode 4 application by widening its tools, without a new decision	Permission manifest, kill switch, re-classification (Chapters 8 and 13)

Risk follows what the system can do, not how it was acquired: the Mode 3 and Mode 5 errors are forms of the unbounded agent, and the Mode 0 and Mode 1 errors are the two faces of shadow AI.

Portfolio and strategy anti-patterns

Pilot purgatory

Symptom. Dozens of AI pilots over two or three years, none in production, because pilots are funded as experiments with no production budget, platform or owner.

Consequence. The organisation pays for learning repeatedly and books none of the value.

Remedy. Fund the platform, run as a product by a platform team in the sense of Skelton and Pais (69, Skelton & Pais, 2019), before funding the fifth pilot. A golden path, a supported route to production with platform services, gates and evidence in place, is the way out; Chapter 16 shows how to promote or retire pilots.

Shadow AI

Symptom. Staff use consumer AI services with corporate data, business units switch on AI features in software-as-a-service products, and developers call foundation model APIs from personal accounts, because unsanctioned use has been forbidden without a faster alternative.

Consequence. Personal data and trade secrets leave without a lawful basis or a contract. Under the AI Act, Regulation (EU) 2024/1689 (1, European Union, 2024), the organisation may have become a deployer of systems it has never assessed, with Article 26 obligations it cannot discharge.

Remedy. Make the sanctioned path faster than the shadow path: an AI gateway with approved models and logging, the inventory of Chapter 9, and telemetry that finds unregistered use so that it can be onboarded rather than punished. Shadow AI is a demand signal; read it.

The model-as-the-product fallacy

Symptom. The roadmap revolves around which foundation model to license, while retrieval, guardrails, oversight and monitoring are treated as plumbing.

Consequence. Zaharia and colleagues (52, Zaharia et al., 2024) observed that state-of-the-art results increasingly come from compound systems rather than single models; programmes that optimise the model and neglect the system ship brittle products whose failures cannot be diagnosed.

Remedy. Architect, approve and document the system, not the model. Because the model is one replaceable component, the use-case approval, documentation and evaluation must cover the whole, and a model change becomes a governed change rather than a new project.

Vendor-defined architecture

Symptom. The AI architecture is a vendor's diagram with the logo changed, because the vendor arrived with a complete story when the organisation had none and architecture was consulted after signature.

Consequence. Exit becomes impractical and sovereignty requirements are met only as far as the vendor chose. For a financial institution under DORA, Regulation (EU) 2022/2554 (3, European Union, 2022), a vendor that has quietly come to support critical or important functions leaves the institution with contractual and exit obligations under Articles 28 to 30 that nobody has documented.

Remedy. Write the architecture first and procure against it, using the decision on model origin, hosting and exit of Chapter 6 and the vendor assessment and exit strategies of Chapter 7.

Governance and compliance anti-patterns

Governance theatre

Symptom. An AI ethics board meets quarterly with a charter and minutes but has never stopped, altered or retired a system, because by the time an initiative reaches it the system is live and the sponsor is senior. It was designed as a forum, with no gate in the pipeline and no authority that bites.

Consequence. Risk accumulates invisibly, and when an incident exposes it the board becomes a liability, because it shows that the organisation knew governance was needed and chose a performance of it.

Remedy. Convert the board's intentions into policy as code and pipeline gates, as Chapter 9 describes. A board that reads evidence generated by the system is useful; a board that reads slides prepared for it is theatre.

Compliance by checkbox

Symptom. The AI Act readiness programme is a spreadsheet of articles with a status column, owned by a function without engineering capacity, and no row links to a control, a test or an artefact.

Consequence. Article 9 of the AI Act requires risk management as a continuous, iterative process throughout the lifecycle of a high-risk AI system, which a spreadsheet updated before each audit is not. When a market surveillance authority asks for the technical documentation of Article 11 and Annex IV, the checkbox has no evidence behind it.

Remedy. Map obligations to controls, controls to evidence and evidence to pipeline stages, as Appendix A shows, and generate the documentation from the repository, as Chapter 11 describes. Hansa Mutual's first spreadsheet, for example, marked the Article 26(6) logging obligation for its underwriting-support model "Compliant: logging enabled"; as a mapping, the obligation is compliant only while a retention test passes and a daily check confirms that log volume matches inference volume, because retention is worthless if logs are silently missing.

Data protection as an afterthought

Symptom. The data protection officer learns about the AI system at the go-live review and is asked to approve a data protection impact assessment (DPIA) within a week, because personal data was treated as an input to be fetched rather than a legal relationship to be designed.

Consequence. Training sets contain personal data without a lawful basis under Article 6 of the GDPR, Regulation (EU) 2016/679 (2, European Union, 2016); special-category data under Article 9 leaks into embeddings; and decisions with legal effect are automated in ways that Article 22 restricts.

Remedy. Express lawful basis, purpose limitation and data minimisation as constraints in data contracts, the machine-readable agreements under which a system receives data. Run the DPIA alongside the AI Decision Record from the first design discussion, and for public authorities run the fundamental rights impact assessment (FRIA) of Article 27 of the AI Act in the same cadence.

The centre of excellence that excels at saying no

Symptom. The AI centre of excellence approves almost nothing; business units stop asking, and shadow AI grows in proportion to its rigour. It is accountable for risk but not for value.

Consequence. The organisation loses the benefits of AI and keeps the risks, which migrate to the shadow. Forsgren, Humble and Kim (70, Forsgren et al., 2018) showed that heavyweight external approval correlates with worse delivery performance and no better stability.

Remedy. Rebuild the centre as a platform team and an enabling team in the sense of Chapter 12, with golden paths that encode the criteria so that compliant initiatives pass without a meeting, and measure it on the throughput of safe deployments.

The platform as someone else's problem

Symptom. The governance function writes policies while IT runs the AI platform under a separate roadmap. When the board requires that high-risk systems retain oversight decisions for the period archives law requires, nobody asks whether the platform can do it, and evidence requests are later answered by hand.

Consequence. Every governance mechanism in this book depends on the platform: policy as code, evidence and the stop authority all run through it. A governance programme that does not shape the platform governs by assertion, and supervisors notice, because the evidence they ask for, from the logs of Articles 12 and 26 of the AI Act to DORA's ICT risk records, comes from systems the governance function never specified.

Remedy. Treat the platform's governance capabilities as requirements the governance function owns and the platform team delivers: a standing item on the platform roadmap, gates for the control functions in the design authority's decisions (Chapter 13), and the evidence requirements of Chapter 11 in the platform's acceptance criteria.

Accountability delegated to the supplier

Symptom. Asked how a procured high-risk system meets the AI Act, the business owner produces the vendor's declaration of conformity. Nobody has assigned oversight staff or set up monitoring, because provider and deployer duties were conflated.

Consequence. Article 26 of the AI Act places obligations on deployers that no contract can discharge: use according to the instructions, competent human oversight, monitoring, log retention and, for employers, informing workers' representatives (1, European Union, 2024). Under Article 28 of DORA, financial entities likewise remain fully responsible for compliance when they use ICT third-party service providers (3, European Union, 2022).

Remedy. Split the obligations in the procurement file between provider and deployer, assign each deployer obligation to a named role before signature (Chapters 7 and 12), and require the provider to supply what the deployer needs, from logs to change notifications.

Engineering and operations anti-patterns

Unbounded agents

Symptom. An agent built to answer questions about invoices acquires the ability to issue credit notes because the tool was available. Nobody can say what it is allowed to do, because the prototype's broad credentials were never narrowed and its instructions serve as the security boundary.

Consequence. Indirect prompt injection through a retrieved document (51, Greshake et al., 2023) can direct the agent to take actions no human authorised, while the audit trail shows a legitimate service account at work. The guidance from Anthropic (53, Anthropic, 2024) to prefer simple, bounded workflows until autonomy is demonstrably needed is ignored at the prototype stage.

Remedy. Bounded autonomy in the sense of Chapter 4 (Principle 4): a permission manifest in version control, a tool allow-list, per-action authorisation under the invoking user's identity, approval gates for irreversible actions, and spend limits enforced outside the model.

Worked example: agent permission set, before and after

Illustrative example. Hansa Mutual's invoice-enquiry agent answers supplier questions about payment status. The prototype ran with the first configuration below; production runs with the second.

# agents/invoice-enquiry.yaml (prototype)
agent: invoice-enquiry
credentials: svc-finance-integration     # shared service account
tools: ["*"]                            # everything the finance API exposes
instructions: Only answer questions. Never issue credit notes.

# agents/invoice-enquiry.yaml (production)
agent: invoice-enquiry
version: 1.3.0
owner: Accounts Payable Platform
identity:
  principal: spn-invoice-enquiry
  acts_on_behalf_of: delegated-user       # supplier's portal identity
tools:
  allow:
    - name: get_invoice_status
      constraints: {supplier_id: "$caller.supplier_id"}
    - name: create_query_ticket
      approval: none                      # reversible, routed to a person
  deny_by_default: true
budgets:
  max_tool_calls_per_session: 12
  max_cost_eur_per_day: 40
kill_switch: feature-flag://agents/invoice-enquiry/enabled

The prototype's only boundary was a sentence in the instructions, which a retrieved document can contradict. The production manifest removes the credit-note tool altogether, scopes every call to the calling supplier's own records, and enforces budgets in the runtime.

Evaluation by demo

Symptom. The system was approved because the sponsor typed six questions into it during a steering meeting and liked the answers. Plausible output is persuasive, and there is no evaluation set or definition of acceptable quality.

Consequence. Regressions ship undetected when the model or prompt changes, quality disputes become arguments about anecdotes, and the accuracy, robustness and cybersecurity obligations of Article 15 of the AI Act cannot be evidenced.

Remedy. Build an evaluation pyramid of governed golden datasets, behavioural tests, red teaming and sampled human review, with acceptance criteria as code and evaluation gates in the promotion pipeline; the governance point is who sets the bar: model risk or the system owner, approved at the tier Chapter 13 assigns, never the sponsor who wants the system live.

Prompt sprawl

Symptom. The same instruction text exists in forty variants across notebooks, code, configuration files and a shared document. In Hansa Mutual's claims department, for example, the instruction for drafting replies existed in two services with a one-word difference nobody had noticed.

Consequence. Behaviour cannot be reproduced, evaluated or rolled back, and the technical documentation required by Annex IV of the AI Act cannot describe the system's instructions because nobody can enumerate them.

Remedy. Prompts are configuration: they live in version control in a registry that names an owner, reviewers, an evaluation suite and the version each consumer pins, so that a change cannot reach production without passing evaluation and can be traced to an approval.

GPU hoarding

Symptom. Business units reserve accelerator capacity against future need; utilisation is low, because idle capacity costs the holder nothing, yet new initiatives are told none is available.

Consequence. The organisation pays for compute it does not use and denies it to initiatives that would. Public bodies with annual budget cycles are especially exposed, because capacity bought in one year cannot be returned in the next.

Remedy. Treat capacity as a shared platform resource with chargeback, scheduling and routing to smaller models or shared endpoints, and expose utilisation to the steering committee of Chapter 16.

The model nobody retires

Symptom. A model trained four years ago still scores applications. Its training data predates a change in law, its pipeline contractor has left, and its monitoring disappeared in a tooling migration, and no retirement criterion was ever defined.

Consequence. Concept drift accumulates unmeasured. The system may now be a high-risk AI system under Annex III of the AI Act without anyone having classified it, and its failure is discovered by the people it affects.

Remedy. Every AI Decision Record states retirement criteria, and every inventory entry (Chapter 9) has an owner and a review date. Drift monitoring feeds the risk re-evaluation of Chapter 8, and the portfolio register of Chapter 16 treats retirement as a first-class outcome.

Detecting anti-patterns as code

Most anti-patterns leave traces in the inventory, model registry, gateway logs and pipeline history. Architecture fitness functions, in the sense of Architecture as Code (Chapter 13) (66, Nordqvist, 2025), can query those traces and raise a signal before a steering committee has to. In the suite below, each function has a data source, a threshold and the chapter holding the remedy, and a scheduled job writes results to the evidence store of Chapter 11.

# anti-pattern-fitness-functions.yaml
apiVersion: ai-architecture.example.org/v1
kind: FitnessFunctionSuite
metadata:
  name: anti-pattern-signals
  owner: chief-architect-office
  schedule: "0 6 * * 1"
functions:
  - id: pilot-purgatory
    source: portfolio_register
    query: "count(use_cases where stage == 'pilot' and age_days > 365)"
    threshold: { max: 3 }
    remedy_chapter: 16
  - id: shadow-ai
    source: network_telemetry
    query: "count(distinct ai_endpoints not in ai_inventory)"
    threshold: { max: 0 }
    remedy_chapter: 9
  - id: unbounded-agents
    source: model_registry
    query: "count(agents where permission_manifest is null)"
    threshold: { max: 0 }
    remedy_chapter: 13

The thresholds are illustrative. A signal that fires is not an accusation but an invitation to look, which is all that early detection ever offers.

Nordvik Region and Hansa Mutual

Nordvik Region is most exposed to the anti-patterns of a cautious buyer: pilot purgatory, vendor-defined architecture, shadow AI when approved tools arrive slowly, and the model nobody retires, because retiring a procured system requires a new procurement; its decommissioning practice (AIDR-0018) exists for that reason. Governance theatre is a particular risk in public bodies, where documents are produced for scrutiny rather than for control.

Hansa Mutual is exposed to the anti-patterns of a builder, as the examples above show. Its multinational footprint adds one that Nordvik cannot have: treating the group policy as the law in every market, so that an EU control is assumed to cover the United Kingdom and Swiss entities, or the Norwegian entity before an obligation is incorporated into the EEA Agreement. Nordvik's anti-patterns come from doing too little too slowly; Hansa's come from doing much, quickly, in one shape for many jurisdictions.

Summary

Portfolio anti-patterns arise when urgency or vendors decide what the organisation builds, governance anti-patterns when accountability is separated from the means to act, and engineering anti-patterns when prototyping habits survive into production.

The remedies share a shape: each replaces a human intention with a codified control: a permission manifest instead of a promise, an evaluation gate instead of a demonstration. That is the thesis of this book applied to failure: probabilistic components cannot be governed by inspection, only by instrumentation. The European context sharpens every failure, for example because public authorities' impact assessments and algorithm registers turn hidden anti-patterns into published ones.

Chapter 21 turns the catalogue inside out, organising the practices that prevent these failures by lifecycle stage and by the three tensions of Chapter 1.

Sources

1. European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union. (1, European Union, 2024)
2. European Parliament and Council of the European Union (2016). Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data (General Data Protection Regulation). Official Journal of the European Union. (2, European Union, 2016)
3. European Parliament and Council of the European Union (2022). Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union. (3, European Union, 2022)
4. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F. & Dennison, D. (2015). Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28. (48, Sculley et al., 2015)
5. Zaharia, M., Khattab, O., Chen, L., Davis, J. Q., Miller, H., Potts, C., Zou, J., Carbin, M., Frankle, J., Rao, N. & Ghodsi, A. (2024). The Shift from Models to Compound AI Systems. Berkeley Artificial Intelligence Research blog. (52, Zaharia et al., 2024)
6. Anthropic (2024). Building effective AI agents. Anthropic research blog. (53, Anthropic, 2024)
7. Paleyes, A., Urma, R.-G. & Lawrence, N. D. (2022). Challenges in Deploying Machine Learning: A Survey of Case Studies. ACM Computing Surveys, 55(6). (55, Paleyes et al., 2022)
8. Nordqvist, G. Architecture as Code. Companion volume to this book, published at aac.geon.se. (66, Nordqvist, 2025)
9. Forsgren, N., Humble, J. & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press. (70, Forsgren et al., 2018)
10. Skelton, M. & Pais, M. (2019). Team Topologies: Organizing Business and Technology Teams for Fast Flow. Portland, OR: IT Revolution Press. ISBN 978-1-942788-81-2. (69, Skelton & Pais, 2019)
11. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T. & Fritz, M. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec '23). (51, Greshake et al., 2023)