When AI Does What It's Told, For the Wrong Person

Key points

• Noma Labs researcher Sasi Levi published findings describing a technique called "workflow identity hijacking," in which an outsider can trigger privileged AI actions through a normal, unauthenticated entry point such as a support inbox or web form.
• The flaw does not require manipulating the AI model itself; the model behaves exactly as designed, but executes actions using the workflow creator's high-level permissions rather than the requester's.
• Because the resulting activity runs through valid credentials and trusted automation, it can blend into routine system logs without triggering standard security alerts.
• Cybersecurity researcher Vibhum Dubey described the pattern as a "confused deputy" problem, a known class of flaw where a trusted system is tricked into misusing its own authority on behalf of someone who should not have it.
• As first covered by CSO Online, Google's Threat Intelligence Group has separately noted that criminals are beginning to use similar multi-step AI workflows to carry out attacks autonomously.

---

What actually happened here?

Researchers at Noma Labs have identified a way that ordinary, innocent-looking messages sent to a company's AI-powered tools can quietly trigger actions that only senior employees should be able to authorise.

The technique works like this. Many companies now use AI-driven automation, meaning software pipelines that read incoming requests and carry out tasks automatically: pulling financial records, updating databases, sending files. These pipelines often accept input from public-facing channels: a customer support inbox, a shared document, a web form. Anyone can drop a message in.

The problem is what happens next.

When the AI pipeline picks up that message and acts on it, it does so using the identity and permissions of whoever built or owns the workflow, often a developer account or a high-level service account with broad access. The pipeline never checks whether the person who sent the original message was actually allowed to make that request.

Sasi Levi, lead researcher at Noma Labs, put it plainly: "The core failure is that the requester had no authority to make that request."

Why is this so hard to catch?

Standard security tools look for suspicious content. This attack sends none.

Levi's report includes a telling illustration. Imagine two people ask the same AI tool for the same financial report: a chief financial officer, and a random outsider. The message is word-for-word identical. A standard security filter sees two harmless prompts and flags neither. But the right answer is different for each sender. One person is allowed that information. The other is not.

Because the pipeline uses its own valid credentials to fetch the data, every step looks legitimate. There is no stolen password, no malicious code, no obvious red flag. Dubey described the detection challenge bluntly: "The individual events can look completely legitimate."

Spotting the problem, he said, requires correlating four things at once: who sent the original message, which identity the system used downstream, what resources were accessed, and what action was finally taken. Most organisations are not set up to join those dots automatically.

What does this mean for companies using AI tools?

The risk grows as companies expand AI automation. A pipeline that can quietly fetch a financial report for an unauthorised requester today could, depending on its configuration, send files, update records, or trigger further automated steps tomorrow.

Levi's report argues that fixing this requires moving security controls out of the AI model itself and into the surrounding infrastructure. Specifically, every step in a workflow should verify the original requester's permissions before acting, not just trust the identity of the system running the workflow.

Dubey offered a practical test: if a low-privileged user and an administrator send the same request and receive the same result, something is wrong.

Common questions

Does this affect me as a customer or user of a business?

If a company you deal with uses AI-powered tools to handle requests or manage data, a gap like this could mean an outsider accessed information that should have stayed private. There is no public list of affected companies; the research describes a class of design flaw, not a single breach.

What should organisations do right now?

Levi's report recommends that every AI workflow carry the original requester's identity all the way through to the point where an action is executed, and check permissions at that point, not just at the front door. Reviewing which service accounts your AI pipelines run under, and what access those accounts have, is a sensible immediate step.