Your Security Team Is Flying Blind on AI. Here Is Why.
The tools built to catch hackers and bad code were designed for a world where humans made every decision. AI agents don't ask permission, and your defences weren't built to watch them.
Key points
- AI agents, software programs that act without waiting for a human to approve each step, are moving from lab experiments into live business systems faster than most security teams can track.
- Existing code scanners and network filters cannot detect prompt injection, where a criminal slips hidden instructions into a live AI system to redirect its behaviour.
- Models pulled from public online libraries can arrive already tampered with, yet most companies apply none of the signing checks they already use for regular software.
- Every AI agent needs its own digital identity, scoped permissions and a full audit trail, requirements most corporate identity systems don't cover.
- A whitepaper co-authored by Broadcom and PlatformEngineering.org argues the engineering team that runs the underlying platform is now a frontline security function.
What gap are we actually talking about?
For the past decade, companies got smarter about catching security problems early. Scan the code before it ships. Block suspicious logins. That approach worked when a human being was behind every action.
AI agents break the model entirely.
An AI agent reads data, makes decisions and completes tasks without stopping to ask a person. It consumes APIs (the connectors that let software talk to other software) directly, touching customer records, billing systems and internal documents in a single automated chain, often with no human involved at all. Our 28 July story "Your AI Safety Certificate Is Worthless the Moment the Agent Goes Live" found the same blind spot in compliance-certified products once autonomous agents start calling internal systems.
Your existing tools weren't built for that.
How could criminals actually cause harm here?
Four attack types matter, and none of them show up on a standard security dashboard.
First, prompt injection. An attacker hides malicious instructions inside content the AI reads: a document, a support ticket, a webpage. The AI follows those instructions as if they were legitimate. No code scanner sees this because there's no static code to scan.
Second, model poisoning. Companies often pull pre-built AI models from public online libraries. If a model has been tampered with before download, and there's no equivalent of a verified signature on the package, the company deploys the corrupted version without knowing. We first covered model poisoning on 29 July 2026.
| Attack type | What it means in plain English | Current tools that catch it |
|---|---|---|
| Prompt injection | Hidden commands trick the AI mid-task | None |
| Model poisoning | Tampered AI model pulled from a library | None |
| Inference data leaks | AI response reveals private customer data | Rarely, if ever |
| Shadow AI sprawl | Staff use unapproved AI tools outside controls | None |
Third, inference data leaks. An AI's answer may contain private customer information or internal secrets. Standard data-loss-prevention tools watch email and file transfers; they don't watch AI conversation logs.
Fourth, shadow AI. Individual teams download and run their own AI tools outside any official approval process. Governance simply doesn't reach them.
What does the fix actually look like?
PlatformEngineering.org and Broadcom's paper is clear: no developer-side checklist closes these gaps. The fix has to live in the shared infrastructure the whole company runs on.
In practice that means four things: a verified library of approved AI models with a clear chain of custody; filters that strip injected commands before they reach the AI; data boundaries that mask private information inside AI pipelines automatically; and a continuous log of every decision an AI agent makes.
On identity: every AI agent needs its own account with limited permissions, just as an employee does. Most IAM (identity and access management) systems have never been asked to manage a non-human account that calls hundreds of APIs per minute. We've covered IAM gaps in six stories in the last 90 days, and agent identities haven't featured in any of them until now.
Should you worry?
Yes, and the timeline is the real problem. If your platform team can't answer basic questions about how agent permissions are scoped or where the audit logs live, that gap exists today, not in some future deployment. Get your platform engineering lead in the room before the next AI workload ships to production. The four control surfaces the paper describes aren't aspirational; they're the minimum.
