Hidden Text in Emails Can Trick AI Assistants Into Showing You Fake Numbers
Researchers planted invisible instructions inside ordinary emails and watched an AI summariser rewrite invoice amounts and meeting dates without any warning to the reader.
Key points
- Researchers at Forcepoint X-Labs successfully manipulated an AI email summariser in all 10 of 10 test runs using hidden text.
- A fabricated invoice amount jumped from €8,750 to €46,200 in the AI-generated summary, with no alert to the reader.
- The attack method, called prompt injection, has ranked number one on OWASP's list of top risks for AI applications every year since 2023.
- OWASP is the Open Worldwide Application Security Project, a non-profit that publishes widely used security standards.
- Defenders can reduce risk by treating every AI-generated summary as unverified until checked against the original email.
An AI assistant that summarises your emails sounds like a time-saver. A new experiment shows it can also be a quiet liability.
Researchers at security firm Forcepoint X-Labs published a proof-of-concept study this week demonstrating that malicious instructions hidden inside an email's underlying code can hijack an AI summariser, causing it to show the reader entirely different facts from those the sender actually wrote. First reported by Dark Reading, the findings are a practical demonstration of a problem the security industry has discussed for years but rarely measured this concretely.
How did the attack actually work?
The criminals' technique here requires no hacking of your computer. It exploits the AI's inability to tell the difference between content it is supposed to read and commands it is supposed to follow.
The Forcepoint team built a test environment using a Microsoft Outlook add-in that passed email text to an AI language model, specifically Anthropic's Claude Haiku, which then produced a short summary. They crafted an otherwise ordinary-looking email and buried a hidden instruction inside it, written in tiny, white-on-white text so it was invisible in Outlook but fully readable by the AI processing the raw code underneath.
That hidden instruction told the AI to change specific details before displaying the summary. The original email stated an outstanding invoice of €8,750 and a particular date for a supplier review. The summary the AI produced showed €46,200 and a different date. The reader saw the AI's version. Nothing flagged the discrepancy.
The injection worked across all 10 test runs. Every single time.
What does this mean for ordinary people?
For now, the practical risk is targeted fraud. A criminal could send a supplier invoice with the real amount visible to a human reader but a larger figure planted in the hidden code, then rely on a busy employee reading only the AI summary before approving payment. No phishing link, no fake login page, just a manipulated summary treated as fact.
The danger grows if the AI assistant can act, not just summarise. An AI given permission to send replies, schedule meetings, or approve calendar invites on your behalf could be directed to do all of those things by the same hidden instruction, with no human ever reviewing the underlying command.
| Detail | Benign email | Injected email summary |
|---|---|---|
| Invoice amount | €8,750 | €46,200 |
| Supplier review date | Original date | Altered date |
| Warning shown to reader | N/A | None |
| Injection success rate | N/A | 10 out of 10 runs |
Forcepoint researcher Ben Gibney was direct about the limits of the study: one model, one message, ten trials. It is not evidence of a mass campaign. It is evidence that the underlying vulnerability is reliable.
What should organisations do?
Forcepoint's recommendations are practical. Keep the instructions you give an AI model strictly separate from the outside content it reads. Treat everything an AI produces as a draft to be spot-checked, not a finished fact. Strip an AI assistant's permissions down to the minimum it needs, so that even a successful injection cannot trigger payments or outbound messages automatically. And scan incoming email for HTML tricks that hide text from human eyes while leaving it readable by machines.
If your organisation uses AI tools that read or act on external content, email, documents, web pages, now is the time to inventory where those tools touch untrusted data. That list is your exposure, and in most organisations it has grown faster than anyone has mapped it.
