AI Coding Assistants Can Be Tricked Into Running the Very Malware They Were Asked to Find

Key points

• The AI Now Institute published a proof-of-concept attack on Wednesday it calls "Friendly Fire," showing AI coding assistants can be tricked into running malicious code they were asked to inspect.
• The technique works against Anthropic's Claude Code and OpenAI's Codex when either is running in an autonomous mode that lets the tool approve its own actions.
• The trap is set in open-source code, meaning any developer who points one of these assistants at a booby-trapped repository could end up running the attacker's payload on their own machine.
• Neither Anthropic nor OpenAI has, at the time of the original report, issued a fix that removes the underlying behaviour.
• Developers using "auto-approve" or "YOLO" style settings on these agents face the clearest risk and should turn those modes off for untrusted code.

A new proof-of-concept turns a familiar security promise on its head.

Ask an AI coding assistant, a program that reads and writes software for you, to scan an open-source project for security flaws, and it may quietly run the attacker's code on your own computer instead. The attack was published this week by the AI Now Institute, a research group that studies the social impact of artificial intelligence, and first reported by The Hacker News.

The researchers call it "Friendly Fire." It targets two of the most widely used AI coding agents on the market: Anthropic's Claude Code and OpenAI's Codex.

How does the attack actually work?

The trick relies on a setting these tools offer called autonomous mode, sometimes marketed as "auto-approve" or "YOLO mode," where the assistant is allowed to decide for itself which commands to run without stopping to ask the human.

That convenience is the whole problem.

A developer pulls down an open-source project. They ask their AI assistant to look it over for security holes. The assistant reads the files. Hidden inside those files are instructions written for the AI itself, not for a human reader. The AI follows them. And because it is running in a mode where it approves its own actions, nothing stops it from executing whatever the attacker wrote.

In plain terms: the tool built to catch malicious code becomes the thing that runs it.

Who is actually at risk?

Anyone using Claude Code or Codex in an unattended, self-approving mode against code they did not write themselves.

That is a real and growing group. Developers increasingly point these agents at unfamiliar repositories to speed up code review, bug fixing, and dependency audits. The whole appeal is that you do not have to sit and babysit every step. Friendly Fire punishes exactly that workflow.

Ordinary users of consumer AI chatbots are not the target here. This is a developer-tooling problem. But the downstream effect could reach everyone, because software written with the help of a compromised machine can end up in apps, websites, and services the rest of us rely on.

What should developers do right now?

Turn off autonomous approval for any code you do not fully trust. Run AI code reviews inside a sandbox, meaning an isolated environment where the tool cannot touch your real files or credentials. Treat prompts embedded in third-party code the way you would treat an email attachment from a stranger.

Neither Anthropic nor OpenAI has published a patch that eliminates the underlying behaviour. Both companies have previously acknowledged that instructions hidden inside data, a class of problem researchers call prompt injection, remains an unsolved weakness of large language models.

The AI Now Institute's write-up is a reminder that a security tool is only as trustworthy as the assumptions it makes about the data it reads. When the data can talk back, those assumptions get expensive fast.