Generative AI in Banking: From Experiment to Core Infrastructure

This post is a written companion to a presentation I gave at a 2025 conference on Generative AI applied to banking. Downoad the presentation in PDF and PowerPoint.

The talk — “Generative AI in Banking”, a fifteen-slide, in-depth analysis prepared for technology directors of European banks — walked through where the economic value sits, which use cases are already live in EU institutions, how to architect for it, and what the EU AI Act demands of every deployment. This article covers the same ground in long form; if you attended the session, consider this the extended version of the deck.

The economic case: a productivity opportunity the size of a business line

The McKinsey Global Institute estimates that generative AI could add $200–340 billion of annual value to the global banking sector if use cases were fully implemented — equivalent to 2.8–4.7% of total industry revenues and 9–15% of operating profits, largely through increased productivity. Banking ranks among the sectors with the largest opportunity relative to revenue. By segment, the largest absolute gains land in corporate banking (~$56 billion per year) and retail banking (~$54 billion per year).

Where does that value concentrate? Across 63 use cases in 16 business functions, roughly 75% of the total sits in four areas, all of them core to banking:

Customer operations — chat and voice assistants, agent support, call summarisation
Marketing and sales — personalised offers, content generation, next-best-action
Software engineering — code generation, testing, documentation, legacy migration
R&D — research synthesis, product design, knowledge discovery

Two-thirds of senior digital and analytics leaders at a 2023 McKinsey forum said generative AI will fundamentally change how they do business. The first pilots target productivity because banking economics are under pressure — but the same models can reshape distribution and service models.

Europe is not waiting: generative AI is the current wave

AI in euro area banking is nothing new — earlier waves built fraud detection and credit scoring. Generative AI is the present wave, and supervisory data shows it is already entering production:

~90% of significant euro area banks already use AI technologies (ECB, 2025 data). Fraud and cybercrime detection leads (used by more than half), followed by marketing (~50%), chatbots (40%) and credit scoring (30%).
~40% of EU banks already use general-purpose AI, led by customer support and internal-process optimisation (EBA RAQ, autumn 2024).
55% of surveyed banks use general-purpose or agentic AI in consumer-facing processes (EBA sample, 2025).

Three domains already deliver measurable value

My presentation deliberately grounded its use cases in what the EBA actually observes in EU banks — not a vendor wish list. Three domains stand out.

1. Customer operations — the first production beachhead.
Volumes are high, text-heavy, and quality is measurable. Generative AI assistants resolve customer queries and answer employees’ questions on internal policies; models transcribe and summarise contact-centre audio and draft next-best responses for operators. Resolution rate, handle time and customer satisfaction give a clean before/after read — rare for AI projects. And because routine queries sit outside regulated decision-making, early deployments rarely trigger high-risk obligations.

2. Software engineering — your own developers are the fastest payback.
Banks use general-purpose AI to generate code from natural language, detect errors, support testing, convert between languages and migrate legacy systems — a structural problem in European core banking estates. Coding assistants can run against internal repositories under the bank’s own access controls, so no customer data leaves the perimeter. It also compounds: faster delivery accelerates every other AI initiative on the roadmap. For context, significant euro area banks invested over in digital technologies including AI in 2025 (~1.3% of tangible assets, ECB).

3. Risk, compliance and legal — the quiet revolution is in documents.
Risk and legal teams are text factories. Observed uses include monitoring legal and regulatory change, summarising court rulings, assessing contractual clauses, drafting regulatory reports and preparing audits. One boundary matters: drafting and summarising is low-risk assistance; deciding is not. Under the EU AI Act, AI used to evaluate creditworthiness or set credit scores is (Annex III), carrying conformity, data-governance, human-oversight and logging duties.

Architecture: one governed gateway, many models

The genera reference architecture has four layers: an experience layer (customer channels and employee desktops), a GenAI gateway — model routing, prompt and output guardrails, RAG orchestration over verified sources, identity and consent checks, and full logging for audit and AI Act record-keeping — a model layer, and a governed data-and-knowledge layer. Governance wraps every layer: model risk management, AI Act conformity, DORA vendor oversight and GDPR data protection.

On sourcing, the EBA finds that most EU banks deploy general-purpose AI through third-party cloud APIs (flexibility and scale), a lower proportion run open-source models on-premises (maximum control, more skills required), and only a small number build proprietary models — in-house effort concentrates on use cases tied to proprietary knowledge such as credit scoring and customer analytics. Most banks combine one to three approaches, with RAG and fine-tuning adapting models to specific applications.

Regulation has a calendar — plan backwards from 2026

The EU AI Act (Regulation (EU) 2024/1689) phases in over three years:

Aug 2024 — in force; governance structures set up. Start your use-case inventory.
Feb 2025 — prohibitions on manipulative practices apply, plus the AI-literacy duty for staff.
Aug 2025 — obligations for general-purpose AI models apply.
Aug 2026 — high-risk obligations apply, covering creditworthiness assessment and credit scoring: data governance, human oversight, logging, conformity.
Aug 2027 — extended deadlines for AI embedded in regulated products conclude the phase-in.

Penalties reach €35 million or 7% of global turnover. Alongside it, DORA (in force since 17 January 2025) applies ICT risk management, incident reporting and third-party register duties to AI vendors and cloud model APIs alike. The practical rule: classify every use case by risk tier before it ships — retrofitting conformity is slower and more expensive than building it into the gateway from day one.

The supervisors’ observed challenges map directly onto platform decisions:

Risk (as observed by the EBA)	Control (built into the platform)
Hallucinations and inaccurate output	Ground every answer via RAG over verified sources; block free-form generation in customer flows; human review for consequential replies
Input-data quality, governance and consent	Curated, versioned, quality-checked corpora; GDPR consent and purpose limitation enforced at the data layer
Reliance on third parties	Model routing as insurance: at least two qualified models per critical flow; DORA register, exit plans, portability tests
Opaque logic	Log prompts, sources and outputs; generate customer-facing reason statements from the same record

The roadmap: foundation, scale, industrialise

A pragmatic 24-month sequence is the following:

Foundation (months 0–6) — gateway MVP with routing, guardrails and logging; a full use-case inventory classified by AI Act risk tier; the first domain live (developer coding assistants, with a measured baseline).
Scale (months 6–18) — three use cases in production across service, engineering and document work; a RAG platform with curated corpora; a second qualified model per critical flow.
Industrialise (months 18–24) — high-risk conformity proven before August 2026; model validation, monitoring and retraining as routine; unit economics tracked (cost per resolved contact, per delivered feature).

Investment posture is measured, not blank-cheque: more than half of respondent EU banks plan to invest up to 0.25% of equity in general-purpose AI; around 21% plan above that (EBA RAQ, autumn 2024).

What to do on Monday

Three priorities close the presentation, and they work just as well here:

Stand up the GenAI gateway. Routing, guardrails, RAG and logging as shared infrastructure — not rebuilt per use case.
Pick three measurable use cases. Start where the EBA sees production today: customer support, coding, document work.
Classify before you build. AI Act risk tiering in the intake process; high-risk conformity ready before August 2026.

Build the platform once. Then scale use cases into it — with the AI Act’s clock as your planning horizon.

artificial intelligence conference Generative AI key note speaker

Generative AI in Banking: From Experiment to Core Infrastructure​