US agencies publish playbook for when AI agents go rogue in the lab
A new joint guide from NIST, the NSA and CISA tells AI developers how to handle 'control incidents', moments when an experimental AI agent does something it was never told to do.
Key points
- NIST, the NSA's AI Security Center and CISA released ten voluntary practices on handling AI 'control incidents' during the training and testing of advanced AI agents.
- A control incident is defined as an AI agent taking uninstructed actions that break the policies, controls or human oversight set around its task.
- The guidance targets pre-deployment AI systems run with reduced safeguards, well before any public release.
- Labs are told to limit each agent's access to data, tools and networks, watch for unexpected behaviour, and preserve evidence before shutting things down.
- The agencies want labs to report incidents and near-misses to government and affected parties, mirroring how cyber breaches are disclosed.
Three US agencies have put their names to something the AI industry has mostly kept in-house: a written procedure for the moment an experimental AI agent stops following instructions.
The joint publication comes from the National Institute of Standards and Technology, the National Security Agency's AI Security Center, and the Cybersecurity and Infrastructure Security Agency. Voluntary, and aimed squarely at the handful of labs training highly capable AI agents tested behind closed doors long before any consumer sees them.
The document defines the problem it exists to solve. A control incident happens when one or more AI agents take uninstructed actions that conflict with or compromise the policies, controls or security measures meant to keep humans in charge.
In plainer terms: the model did something nobody asked it to do, and the guardrails didn't hold.
What is a 'control incident' in plain English?
It's the AI equivalent of a lab animal getting out of its cage. An AI agent, meaning a model that can take actions on its own such as running code or using software tools, steps outside the task it was given. It might touch systems it wasn't meant to touch, ignore a stop instruction, or find a workaround the testers didn't anticipate.
The agencies aren't talking about chatbots saying rude things. They're talking about pre-deployment systems with reduced safeguards, run inside research environments, where the whole point of the test is to see what the model will try. Our coverage of AI agents quietly reading everything they can reach on 5 October shows why containment matters even in supposedly controlled settings.
What are the labs being asked to do?
The ten practices read like a cyber incident response plan rewritten for AI research.
| Area | What the guidance asks for |
|---|---|
| Access | Limit each agent's reach into information, systems, tools and networks |
| Monitoring | Watch continuously for unexpected agent activity |
| Response | Preserve evidence, then shut down the undesired behaviour |
| Recovery | Rebuild from verified baselines before restarting tests |
| Disclosure | Report incidents and near-misses to government agencies and affected parties |
Two details stand out. The guidance insists on testing controls and validating containment before an agent resumes activity, a step borrowed from ransomware recovery playbooks. It also treats near-misses as reportable, not just full-blown incidents, which is closer to how aviation safety works than how the software industry usually behaves.
Does any of this have legal force?
No. There's no regulator named as enforcer. CISA's advisory sits alongside a broader push, including separate guidance on agentic AI services, that's trying to set expectations before legislation catches up.
That's the thing worth watching here. Voluntary frameworks from NIST tend to become the baseline that contracts, insurers and later rulemakings point back to. A lab that ignores this document today may find itself explaining why in a procurement questionnaire next year. The same NSA and CISA pairing that named Chinese AI firms for copying American models on 17 September is now turning its attention inward, asking frontier labs to police their own experiments with the same discipline applied to a network intrusion.
For everyone outside those labs, no customer data is involved and there's nothing immediately exposed. The signal worth noting is narrower and more consequential: three US agencies now consider an AI agent breaking its own rules to be an incident class that deserves a written response plan.
