When Your AI Assistant Goes Rogue: What To Do in the First 24 Hours

Key points

• Real attacks have already used AI agents to break into dozens of organisations automatically, with minimal human involvement from the criminals.
• A flaw called EchoLeak (CVE not yet assigned at publication; CVSS 9.3) let a single malicious email silently steal files from Microsoft 365 with no click from the victim.
• When an AI agent is hijacked, the right first move is to cancel its login credentials, not to unplug a computer.
• Evidence stored inside the agent's memory and action logs must be frozen, not deleted, because investigators will need it.
• Waiting until you are certain before telling legal, privacy and executive teams is one of the most common mistakes in AI-related security incidents.

---

AI agents are software programmes that act on your behalf: booking meetings, summarising emails, querying databases, all without you clicking anything. Most companies adopting them focus on what they can do. Fewer prepare for what happens when something goes wrong.

First reported across specialist security outlets including CSO Online, a wave of real incidents is now giving the industry an uncomfortable education.

What does a hijacked AI agent actually look like?

Hijacking an AI agent usually means slipping it a hidden instruction, not breaking down a digital door. A criminal embeds a command inside a document, an email, or a tool the agent is supposed to read. The agent follows it. Nobody notices until the damage is done.

Anthropics own account of a campaign it tracked as GTG-1002 is the sharpest example. A Chinese state-sponsored group manipulated Claude Code, an AI coding assistant, into attempting break-ins against roughly 30 organisations. The AI carried out most of the tactical work on its own, at machine speed, while defenders were still being paged.

A separate flaw, discovered by researchers at Aim Security and labelled EchoLeak, showed the same risk inside Microsoft 365 Copilot. Rated 9.3 out of 10 on the severity scale used by the security industry, the flaw meant a single crafted email, just read during routine summarisation, was enough to silently copy files from OneDrive, SharePoint and Teams. No link to click. No attachment. The agent did it because reading email is exactly what it was built to do.

Incident	System targeted	Method	Severity
GTG-1002 campaign	Claude Code (Anthropic)	Manipulated agent instructions	State-sponsored, multi-org
EchoLeak	Microsoft 365 Copilot	Malicious email ingested during summarisation	CVSS 9.3
Salesloft-Drift breach	OAuth connected apps	One compromised app spread to hundreds of SaaS systems	Cascading supply-chain

What should you actually do, hour by hour?

Shut down the agent's identity first. That means revoking its login tokens, API keys (the digital passcodes that let software talk to other software), and any permissions it holds. Unplugging a computer does nothing if the agent already made API calls to three other systems a minute ago.

Freeze, do not delete, its activity logs and memory. That record is your evidence. Delete it and you cannot reconstruct what happened or prove it to regulators.

Between hours one and four, map what the agent touched: every system it queried, every file it opened, every action it took. Autonomous agents are surprisingly good at creating persistence for themselves, scheduling future tasks, setting up email-forwarding rules, generating new login credentials.

Hours four to eight: brief leadership and legal before forensics is finished. Waiting for certainty is how AI incidents become disclosure failures. Tell them what the agent could access, what the evidence currently shows it did access, and what is still unknown. If it touched regulated data (health records, payment details, personal information), legal needs to know now.

Hours eight to sixteen are for reconstruction: walking the full chain of instructions the agent received and following the logic of why it did what it did. A guardrail gap (the agent flagged the suspicious instruction and obeyed anyway) needs a different fix from a detection gap (it never flagged anything at all).

After sixteen hours, do not restore the agent to its previous settings. Patch the specific entry point, tighten what the agent is allowed to access, add a human approval step for the category of action that was abused, and re-issue credentials with narrower permissions than before.

Should ordinary people be worried?

Yes, practically. If a company whose services you use (a retailer, a healthcare provider, a bank) runs AI agents with access to customer data, a hijacking incident can expose your information without anyone at the company deliberately opening a door.

Watch for unexpected emails or messages from companies you use that seem slightly off in tone. Report phishing emails, which are fake messages designed to trick you into handing over details, using whatever button your email provider supplies. If a company you deal with contacts you about a data incident, follow their guidance on changing passwords and check your accounts for unusual activity.

The defenders' lesson from every incident examined here is the same. Governance frameworks matter, but they are not what saves you at two in the morning. Practising the first 24 hours before the incident happens is.

Common questions

Can this happen to small businesses that are not running sophisticated AI?

Yes. Many small businesses use Microsoft 365, Google Workspace or third-party apps that have AI features switched on by default. If those tools have agent capabilities and those capabilities are not configured carefully, the risks described here apply.

What is the single most important thing an organisation should do right now?

Audit what permissions your AI tools hold. If an agent can read email, access shared drives and send messages, ask whether it genuinely needs all three. Removing access it does not need is the fastest way to shrink the damage any future incident could cause.