The EU AI Act's incident reporting rule will miss the attacks that matter most

The EU AI Act's serious-incident reporting duty, due to apply from 2 December 2027, will produce a wave of under-reported incidents. Not because firms hide them, but because the duty's trigger is 'serious' while the dominant real-world attack classes generate outcomes that are individually minor and only become serious in aggregate.

What most people think

The consensus is that Article 73 will improve visibility. Once firms are legally required to notify regulators of serious AI incidents, the argument goes, the fog lifts. The main risk is non-compliance or weak enforcement, so the policy conversation focuses on deadlines, templates and penalties. This is a reasonable view. Reporting duties have worked before in aviation, medical devices and financial services. The assumption is that the incidents that matter are the ones big enough to notice, and that a legal duty will surface them.

What the data shows

Our live incident database, which tracks reported AI failures from public news, tells a different story. In the last 180 days we logged 1,500 stories. The severity mix over the last 45 days is 52 critical, 353 major and 13 minor. The feed is overwhelmingly 'major', meaning sub-serious, events. A per-event severity gate would discard almost all of it.

The attack classes with the most documented real-world cases in MITRE ATLAS barely appear in the news at all. LLM Prompt Crafting (AML.T0065) has 22 documented real cases. Evade AI Model (AML.T0015) has 18. Neither appears in our 180-day news window. Meanwhile the catalogue keeps growing: AI Agent Tool Invocation (AML.T0053) has 15 documented cases, Indirect prompting (AML.T0051.001) has 13, and User Harm (AML.T0048.003) has 12. None of these are rare techniques. They are simply not the kind of event that generates a headline or, by extension, a serious-incident notification.

Attention is already drifting away from the reporting machinery. Governance category stories fell from 231 to 168 in the last 45 days, a drop of 63. Compliance stories fell from 67 to 32. The governance world is looking elsewhere before the duty even starts.

Why this happens

Serious-incident reporting keys on a single event crossing a harm threshold. The highest-volume documented attack techniques do not work that way. Prompt crafting and model evasion produce low-grade, repeated, individually sub-threshold outcomes. One crafted prompt that nudges a model slightly off policy is not a serious incident. Ten thousand of them, spread across a quarter, is a pattern that tells you the model is being probed and softened up. A per-event severity gate discards exactly the data that would let a regulator see the pattern.

This is not a loophole. It is a design mismatch. The duty was written for incidents that look like accidents: a system fails, harm occurs, you notify. The dominant real-world attack classes look more like erosion. Each event is small enough to log internally and forget. The aggregate is the actual risk, and the aggregate has no reporting trigger.

The consequence for a risk team is concrete. If your incident taxonomy is built around the Article 73 severity gate, you will have no internal record of the aggregate of sub-threshold prompt-crafting and evasion events. When a regulator later asks whether you saw the pattern coming, you will not be able to show that you did. The threat side of this problem is well documented. ThreatClaw's piece on how attackers trick AI incident response before anyone notices is worth reading alongside this: https://www.threatclaw.ai/blog/how-hackers-trick-your-ai-incident-response-before-anyone-notices

The best argument against this

The strongest objection is that regulators are not naive. They know about prompt injection and model evasion. They can ask for aggregate data, write guidance that treats repeated low-grade events as a pattern, and use their own market surveillance powers. On this view, the serious-incident duty is one instrument among many, and the gap I describe will be closed by supervisory practice rather than by the letter of Article 73.

That is a fair point, and it is partly right. But supervisory practice needs a paper trail, and the paper trail is created by the reporting regime. If the trigger is 'serious', the internal log that feeds the notification will be organised around seriousness. Firms will build the register the law asks for. Guidance issued after the duty starts will not retroactively create the sub-threshold records that were never kept. The objection also assumes regulators will have the capacity to chase aggregate patterns across thousands of firms. The evidence from other sectors is that they rarely do until after a visible failure. The design mismatch matters because it shapes what gets recorded, not just what gets reported.

What I think happens next

In the first two quarters after 2 December 2027, published Article 73 incident notifications will be dominated by a small number of high-severity single events. Prompt-crafting and model-evasion classes will be absent or near-absent from the notifications, even though they dominate documented ATLAS case counts. I commit to this by 30 June 2028.

What would prove it wrong: if the first published tranche of Article 73 serious-incident notifications contains a material share of prompt-crafting or model-evasion entries, at a rate comparable to their share of documented ATLAS cases, the claim is wrong. I would be glad to be wrong. It would mean the regime is capturing the pattern, not just the spike.

What to do about it

• Add a sub-threshold incident register that captures every prompt-crafting and model-evasion attempt regardless of outcome severity. Report its aggregate trend separately from the Article 73 log, so you can show a pattern even when no single event crossed the threshold.

• Pre-write the Article 73 notification template so that an aggregate-pattern escalation can be filed as a single serious incident. Do not wait for one event to cross the threshold before you have the paperwork ready.

• Map your incident taxonomy against MITRE ATLAS techniques, not just against the regulatory severity gate. If a technique has documented real-world cases and your register has no field for it, you have a blind spot.

• Track the regulatory calendar beyond Article 73. The EU AI Act's high-risk obligations for Annex III systems also apply from 2 December 2027, and the labelling rule for AI-generated content applies from August 2026. Colorado's original AI law was repealed and replaced by a narrower one starting January 2027. Canada has no comprehensive federal AI law right now, though OSFI's Guideline E-23 on model risk management takes effect in May 2027 for federally regulated financial institutions. Each of these creates its own record-keeping expectation.

• Review your incident response playbook against the threat side, not just the compliance side. A governance or risk programme can help you build the register and the escalation path, but the test is whether your team would notice a slow campaign of sub-threshold probes. ThreatClaw's comparison of four AI rulebooks and what actually keeps attackers out is a useful sanity check: https://www.threatclaw.ai/blog/four-ai-rulebooks-one-real-question-which-actually-keeps-hackers-out

The point is not that Article 73 is bad. It is that a severity gate selects for the wrong incidents. Build the record the gate does not ask for, or you will be explaining later why you did not see it coming.

More from our platforms

These sister platforms cover the parts of this problem that sit outside governance.

• Argus (argus.threatclaw.ai) records every trace an AI application produces and scans it for prompt injection, jailbreaks and data leaks, including the attacks hidden inside retrieved documents and tool results rather than in what the user typed. Governance decides what an AI agent is allowed to do. Argus shows what it actually did.
• ThreatClaw (www.threatclaw.ai) tracks the threat side of the same systems: 22 live intelligence feeds, exploitation predicted before it is officially confirmed, threat actor profiles, and detection rules you can deploy straight away. A control is only as good as the threat it is sized against.

Related reading:

• How Hackers Trick Your AI Incident Response Before Anyone Notices on ThreatClaw
• Four AI Rulebooks, One Real Question: Which Actually Keeps Hackers Out? on ThreatClaw