St. Louis County warns seniors about AI-generated tax scams

Scammers used AI to impersonate local officials and trick seniors into paying fake tax bills. They created videos that looked and sounded like real county officials. These videos were used to demand immediate tax payments to avoid arrest or legal trouble. The county issued a warning to the public about this specific scheme. The incident shows a gap in how organizations verify the authenticity of AI-generated content.

This incident highlights a failure of verification controls. The county did not have a clear process to distinguish between genuine government communications and AI-generated fakes. This mirrors the Knight Capital incident where a deployment error allowed a system to fire millions of orders in minutes. In both cases, a lack of proper controls allowed a system to act autonomously without adequate human or technical oversight. The damage in the St. Louis case was financial fraud, while Knight Capital lost money. The root cause is the same: a lack of a verification layer.

The EU AI Act addresses this type of risk through its transparency requirements. The law mandates that users be informed when they are interacting with an AI system. While this applies to specific high-risk systems, the principle of source verification is a fundamental control. Organizations must ensure that their public facing systems are clearly identifiable. This prevents confusion between human and machine actions. It also holds the organization accountable for the outputs of its systems.

This case serves as a warning for all organizations. You cannot simply trust that your brand or name is safe from misuse. A failure to control the dissemination of your brand creates a liability. The obligation falls on the organization to establish a chain of custody for its communications. This includes technical measures and procedural checks. Without these, the organization is responsible for the harm caused by unauthorized use of its identity.

More from our platforms

These sister platforms cover the parts of this problem that sit outside governance.

• Argus (argus.threatclaw.ai) records every trace an AI application produces and scans it for prompt injection, jailbreaks and data leaks, including the attacks hidden inside retrieved documents and tool results rather than in what the user typed. Governance decides what an AI agent is allowed to do. Argus shows what it actually did.
• ThreatClaw (www.threatclaw.ai) tracks the threat side of the same systems: 22 live intelligence feeds, exploitation predicted before it is officially confirmed, threat actor profiles, and detection rules you can deploy straight away. A control is only as good as the threat it is sized against.