Twelve Controls That Actually Matter Once AI Ships to Production
Visibility into AI applications is a starting point, not a security posture. Here is what ongoing monitoring and defense of production AI systems looks like in practice.
Key points
- Production AI systems introduce risks that traditional application security doesn't cover, including prompt injection and training-data leakage.
- Monitoring means baselining query distributions and flagging statistical drift, not just logging API calls.
- Incident response for AI events needs its own runbook, including model rollback procedures.
- Governance records such as model cards and output audit logs deserve the same retention discipline as access logs.
- Organisations under the EU AI Act or FTC algorithmic accountability guidance face a compliance layer that makes documentation non-optional.
Deploying an AI model is not the finish line. It's the moment the attack surface opens.
Security teams have spent two years debating whether to allow AI tools inside the perimeter. Fewer have built repeatable processes for what happens after the model is live and sitting adjacent to sensitive data stores.
The gap matters. Production AI systems introduce risk categories traditional application security doesn't fully cover: prompt injection from user input, model output that downstream systems may trust blindly, training-data leakage surfaced through inference, and supply-chain exposure in third-party model weights or fine-tuning pipelines. Our June reporting on CVE-2026-4372 showed exactly how that supply-chain exposure lands in practice, no warnings, no trace.
A control framework for production AI needs to address four phases: continuous monitoring, anomaly investigation, incident response specific to model behavior, and governance that doesn't collapse under the pace of ML deployments.
Should you worry about monitoring?
Yes, but not just about logging API calls. Teams should baseline normal query distributions and flag statistical drift. A sudden shift in input topics or output length can signal prompt injection campaigns or model manipulation before any alert fires.
What does investigation actually require?
Tooling that can replay inference requests with full context. An AI interaction carries implicit state: the system prompt, the conversation history, the retrieval context if the system uses RAG (retrieval-augmented generation, where a model pulls in live documents before responding). Without capturing all of it, root-cause analysis is guesswork.
Common questions
How is AI incident response different from classical IR?
Containment options differ. You may need to roll back a fine-tuned model, invalidate a vector database index, or temporarily route traffic to a frozen baseline version while forensics runs. None of that appears in a standard incident response runbook.
Where do governance programs typically stall?
At documentation. Model cards and output audit logs get filed and forgotten. We've argued since May that embedding compliance inside the release pipeline is the only approach that survives a system that mutates between deploys.
None of this is exotic. Security teams already do analogous work for APIs and data pipelines. The discipline transfers; the tooling mostly doesn't, yet.
Security ownership of AI systems often stops at the model-deployment handoff. That's the wrong place to stop.
What security teams should do now:
- Map every production AI endpoint against a consistent risk taxonomy before expanding the fleet.
- Establish output logging with enough fidelity to reconstruct an inference session end-to-end.
- Write and drill an AI-specific incident response runbook, model rollback procedures included.
- Engage legal and compliance early on retention obligations under applicable AI regulation.
