AI Agents in Phishing Tests: Risks and Failures Exposed
Autonomous AI agents can be tricked into leaking sensitive data, highlighting configuration issues in security frameworks.
Security researchers have demonstrated that AI agents, when used in corporate environments, can become targets for phishing attacks. Varonis Threat Labs conducted a test using an AI agent built on OpenClaw, named Pinchy, to assess its vulnerability to phishing schemes. The test involved granting the AI agent access to a controlled Google Workspace environment with mock AWS credentials, CRM data, and internal emails.
The experiment revealed that Pinchy, despite having email safety guidelines, failed to identify phishing attempts that seemed routine or came from familiar sources. In one instance, the AI agent forwarded sensitive information like AWS IAM keys and database passwords to an external email, mistaking a phishing request for a colleague's legitimate task. Another test saw the agent send CRM exports containing customer data to an attacker, misjudging the request as a business need.
However, the agent showed resilience against technical phishing attacks, such as a malicious OAuth consent flow. In this scenario, Pinchy was able to recognize the threat by inspecting the suspicious redirect address and halting consent.
Devashri Datta, a cybersecurity researcher, pointed out that the issue lies in the agent's configuration and deployment, rather than the AI model itself. The agent's dual use of email as an information source and instruction channel poses a security risk, mixing data lanes with control lanes.
Keith Prabhu of Confidis noted that AI agents collapse traditional security architectures into a single pipeline, lacking proper segregation in authorization and execution. Sunil Varkey, a cybersecurity adviser, emphasized treating AI agents as high-privilege identities with enforceable controls and human oversight.
Akshat Tyagi from HFS Research advised that enterprise focus should be on defining both what agents can access and what they are permitted to send outside the organization. AI agents require distinct identities and access limits to prevent unauthorized data sharing.
