Adversarial AI Testing: The Missing Layer in AI Governance

Your organisation almost certainly uses more AI than your security team knows about. The question isn’t whether your AI governance covers it. The question is whether any of it would hold up against someone trying to break it.

Most enterprises now run a strange split. On one side, AI governance is maturing: policies, acceptable-use rules, risk classifications, often mapped to the EU AI Act. On the other side, the actual AI footprint keeps growing faster than anyone can track.

According to the Cloud Security Alliance, over half of enterprise generative AI adoption is shadow AI, and 89% of enterprise AI usage is invisible to security teams (CSA, 2026). IBM’s 2025 Cost of a Data Breach Report found that one in five breached organisations suffered a breach involving shadow AI (IBM, 2025).

The regulation already expects more than paperwork

The EU AI Act doesn’t just ask whether you’ve documented your AI. For high-risk systems, it asks whether they’re actually robust and secure.

Article 15 of the AI Act requires high-risk AI systems to achieve “an appropriate level of accuracy, robustness, and cybersecurity” and to be resilient against attempts by unauthorised third parties to alter their behaviour by exploiting vulnerabilities (EUR-Lex, 2024). That’s a security requirement written into law, not a documentation exercise.

This is the same gap Damovo’s Adversarial team, Lares, has written about in other domains: compliance measures whether controls are documented, while attackers measure whether they work. AI is simply the newest place where that gap opens up, and it’s opening fast because adoption is outpacing every governance process built to contain it.

What “testing” AI actually means

AI systems fail in ways that don’t look like traditional software vulnerabilities, which is why traditional testing misses them.

When we test an AI deployment adversarially, the team isn’t running a vulnerability scan. We’re attacking it the way a real adversary would:

• Manipulating what the AI is told to do. A carefully crafted input can override an AI assistant’s instructions, turning a helpful internal tool into something that follows an attacker’s objectives instead.
• Abusing what the AI is allowed to touch. Modern AI agents can send emails, query databases, and call internal APIs. Tricking the agent into misusing those tools is one of the fastest paths to real damage.
• Extracting what the AI has seen. Models trained on or connected to sensitive data can be coaxed into revealing it, one carefully phrased question at a time.

None of these show up in a governance review. They show up when someone deliberately tries to break the system.

The OWASP Top 10 for Agentic Applications, released in late 2025, catalogues exactly these risks for autonomous AI agents (OWASP, 2025). The framework exists. What’s missing in most organisations is the evidence that their own deployments survive it.

Deep Dive: OWASP Agentic AI Top 10: Threats in the Wild

A practical way to close the gap: Adversarial AI Assessment

Damovo’s Offensive Cybersecurity Advisory Team, Lares, offers a structured way to find out what your AI actually does under attack, not just what your policies say it should do.

The Adversarial AI Assessment runs in three tiers, and organisations can enter at whichever matches their maturity:

1. AI Footprint Discovery. Before you can secure your AI, you have to find all of it, including the shadow AI your governance doesn’t know about. This is deliberate, expert-led reconnaissance, not scan output. The gap that matters is the one a scanner can’t see.
2. Adversarial AI Testing. Hands-on testing of your AI systems against real attack techniques: prompt manipulation, agent tool abuse, data extraction, and the other methods catalogued in the OWASP frameworks. The output isn’t a list of theoretical risks. It’s evidence of what an attacker could actually do, mapped to the defences that stopped them and the ones that didn’t.
3. AI Retesting Cadence. AI systems change constantly: new models, new integrations, new data connections. A one-time test goes stale fast. The retesting cadence keeps validation continuous, so the evidence stays current as the footprint evolves.

The throughline is simple and it’s the same principle behind everything Lares does: don’t tell us your controls work. Show us, under realistic attack conditions, with your team involved in closing what we find.

Learn more: Adversarial AI Assessment

Where to start

If your organisation has AI governance in place but has never tested what its AI systems actually do under attack, that’s the gap to close first. And if you don’t yet know the full extent of AI in use across the business, the discovery tier answers that question before anything else.

Sources

• Cloud Security Alliance, Shadow AI Apps: The Enterprise Attack Surface That Outpaces Monitoring (2026)
• IBM, Cost of a Data Breach Report 2025 (2025)
• EUR-Lex, Regulation (EU) 2024/1689 (Artificial Intelligence Act) (2024)
• OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026 (2025)