Model Risk Management Will Fail on Agentic AI

Model risk management is being imported wholesale into artificial intelligence governance via the Office of the Superintendent of Financial Institutions Guideline E-23 effective May 1, 2027, but the import will fail on a specific mechanism. Traditional model risk frameworks assume a model is a versioned artefact with a validation gate, while agentic systems are continuous behavioural surfaces. The result will be a wave of compliant-looking documentation for systems that were never actually validated, and the failures will show up as post-deployment incidents in financial firms.

What most people think

Many practitioners and risk leaders believe that extending proven model risk management discipline to artificial intelligence and machine learning is the safest available path. The reasoning is that banks are well positioned because they already run mature model risk management programmes for credit scoring, market risk, and liquidity models. Under this view, artificial intelligence is just another asset class that needs to be plugged into the existing three lines of defence, subjected to independent validation, and entered into the model inventory with a designated owner and a periodic review schedule.

What the data shows

Our live incident database, which tracks reported artificial intelligence failures from public news, recorded 1426 stories in the last 180 days. In the most recent 45 days, governance stories rose to 215 compared to 176 in the prior 45 days, while compliance stories dropped from 89 to 49. Within our incident data, multi-agent risks accounted for 112 stories in 180 days, making it the second-highest coverage category after privacy compromise. At the same time, capability and robustness risks registered zero stories in the same window.

When we look at threat intelligence from our sister platform ThreatClaw, a different picture of risk emerges. Attack techniques with documented real-world case studies from the MITRE ATLAS framework, such as artificial intelligence agent tool invocation with 15 documented real cases, are almost never mentioned in mainstream news coverage. While governance teams focus on policy compliance and static inventories, operational security teams are tracking active agent tool misuse, prompt injection, and unauthorized data access.

Why this happens

Model risk validation works by freezing a model version, testing it against a defined input distribution, and signing off. An agent that calls tools, retrieves live data from external APIs, and chains decisions has no stable input distribution and no freeze point. When you validate an agentic system using traditional model risk management, you are validating a snapshot in time that the deployed system no longer matches. The prompt changes, the retrieval index updates, and the tool permissions expand, but the validation certificate remains valid on paper. This produces documentation that is complete and misleading at the same time.

Governance decides what an artificial intelligence agent is allowed to do, but operational reality often diverges sharply from policy. As discussed in the ThreatClaw article Why Your AI Projects Need Their Own Rulebook at https://www.threatclaw.ai/blog/why-your-ai-projects-need-their-own-rulebook, static policies fail when applied to dynamic systems. To bridge this gap, teams often rely on tools like argus.threatclaw.ai, which records every trace an artificial intelligence application produces and scans it for prompt injection and data leaks, including attacks hidden inside retrieved documents and tool results rather than what the user typed.

The best argument against this

Banks do not treat agents as static spreadsheets, and risk teams already know how to handle dynamic pricing models and algorithmic trading systems that update frequently. Therefore, the argument goes, model risk management functions can simply adapt their validation methodologies to include continuous monitoring, automated regression testing, and dynamic inventories without breaking the underlying framework. This objection is correct in theory, but it underestimates the operational friction of banking institutions. Most model risk management teams are structured around annual validation cycles and static artifacts. Forcing them to validate continuous behavioural surfaces requires a complete redesign of how validation teams operate, which most institutions are not staffed or budgeted to do before May 2027.

What I think happens next

By December 2027, at least one Canadian federally regulated financial institution will disclose in a public filing, supervisory letter summary, or published finding that its artificial intelligence model inventory or validation approach was insufficient for agentic or continuously updated systems, or the regulator will publish guidance clarifying that validation expectations apply differently to such systems. What would prove this wrong is if the regulator publishes no clarification on scope for agentic systems and no Canadian federally regulated financial institution discloses a model inventory or validation gap for such systems by December 2027.

What to do about it

• Separate your artificial intelligence inventory into frozen-artefact models and continuous-behaviour systems, and apply completely different validation gates to each.
• Define a re-validation trigger for agentic systems based on tool-set changes, retrieval-source changes, or prompt updates, rather than relying on a calendar schedule.
• Log every production agent action with the specific version of the prompt, tool set, and retrieval index in force at that exact moment so that post-incident reconstruction is actually possible.
• Connect your governance oversight to runtime observation platforms like argus.threatclaw.ai to verify what your agents actually did in production against what the model risk framework assumed they would do.

More from our platforms

These sister platforms cover the parts of this problem that sit outside governance.

• Argus (argus.threatclaw.ai) records every trace an AI application produces and scans it for prompt injection, jailbreaks and data leaks, including the attacks hidden inside retrieved documents and tool results rather than in what the user typed. Governance decides what an AI agent is allowed to do. Argus shows what it actually did.
• ThreatClaw (www.threatclaw.ai) tracks the threat side of the same systems: 22 live intelligence feeds, exploitation predicted before it is officially confirmed, threat actor profiles, and detection rules you can deploy straight away. A control is only as good as the threat it is sized against.

Related reading:

• Why Your AI Projects Need Their Own Rulebook on ThreatClaw
• When Hackers Let Your AI Train Itself Into Failure on ThreatClaw