The Cheapest Way to Beat an AI Security System Is to Read Its Rulebook
Confidence in autonomous hacking tools has collapsed. A researcher says the real problem runs deeper: the governance rules we write to keep AI security systems safe can become a weapon in an attacker's hands.
Key points
- The share of organisations willing to rely on autonomous penetration testing, meaning AI systems that hunt for security weaknesses, dropped from 29% in 2025 to 9% in 2026, according to a Cobalt report cited by Dark Reading.
- Independent researcher Burak Oktenli argues the deeper problem is "cap weaponisation": attackers exploiting the written governance rules around an AI system rather than its code.
- An attacker who understands a system's recovery rules can drain it of authority over eight weeks without ever touching its software.
- The fix is cross-layer auditing: computing the system's authority in one place, enforcing it in another, and comparing the two records so any manipulation shows up as a mismatch.
- Security leaders should red-team the rulebook itself, not just the technical systems it governs.
Why are organisations losing faith in AI security tools?
Fewer than one in ten organisations now trust autonomous security tools to work without close human supervision. That is a steep fall from nearly one in three just a year ago.
The easy reading is that the technology overpromised. Oktenli thinks something more specific is happening: attackers have learned that the cheapest way to defeat an AI security system is not to crack it open but to work around the rules its owners wrote to keep it safe.
How does attacking the rulebook actually work?
Every AI security system ships with governance rules. These set an authority ceiling (the maximum action the system can take on its own), recovery protocols (instructions to step back when something looks wrong, then resume slowly), and an audit log (a record of every decision).
Those rules are published in product sheets and compliance standards. Anyone can read them.
Oktenli describes a scenario he calls cap weaponisation, meaning an attack aimed at the governance layer itself rather than the software underneath. Picture a defensive system sitting at the edge of a company's network. It works exactly as designed. Over eight weeks, nothing in its daily logs looks alarming.
Yet the system drifts into a state where humans must approve nearly every action, crippling its speed. No exploit was fired. The attacker simply read the recovery rule (step back fast, recover slowly) and ran short, cheap bursts of low-level probing to keep knocking the system down before it could climb back up.
The probes cost almost nothing. The damage is free, because the system is just following its own instructions.
What are the other openings an attacker could use?
The slow-drift attack is one of four routes Oktenli maps through the governance layer.
| Attack route | What it does |
|---|---|
| False inputs | Feeds the system fake readings so its authority calculation sees conditions that do not exist |
| Slow configuration drift | Slips under the authority ceiling gradually, without a clean exploit |
| Fake threat-clear signals | Tricks the system into recovering authority it has not earned |
| Weakest coalition node | In shared or coalition setups, targets the softest participant to drag down collective decisions |
None of these requires breaking encryption or finding a zero-day, meaning a software flaw the maker did not yet know about. They require patience and a copy of the specification.
What should security teams actually do?
Oktenli's answer is not to hide the governance rules. Hiding them kills the transparency that makes the system trustworthy. The fix is cross-layer consistency: compute the system's authority in one place, enforce the ceiling somewhere separate, record both, and have a review process compare them. An attacker who manipulates one layer without touching the others produces a mismatch the audit can catch.
Three practical steps follow. Root the authority-ceiling enforcement in hardware with its own independent record, so software cannot silently override it. Review system behaviour over long windows, not just daily logs; the eight-week drift Oktenli describes is invisible day to day but obvious across a quarter. Finally, pay red teamers, security professionals hired to attack your own systems, to assault the rulebook and not just the code.
Organisations whose staff understand how AI governance rules can be abused are better placed to spot slow-drift anomalies before they become a crisis. Train2Secure offers security-awareness programmes that cover exactly this kind of operational blind spot.
