Hidden Text in Emails Can Trick AI Assistants Into Showing You Fake Numbers
Researchers planted invisible instructions inside ordinary emails and watched an AI summariser rewrite invoice amounts and meeting dates without any warning to the reader.
Key points
- Researchers at Forcepoint X-Labs successfully manipulated an AI email summariser in all 10 of 10 test runs using hidden text.
- A fabricated invoice amount jumped from €8,750 to €46,200 in the AI-generated summary, with no alert to the reader.
- The attack method, called prompt injection, has ranked number one on OWASP's list of top risks for AI applications every year since 2023.
- OWASP is the Open Worldwide Application Security Project, a non-profit that publishes widely used security standards.
- Defenders can reduce risk by treating every AI-generated summary as unverified until checked against the original email.
An AI assistant that summarises your emails sounds like a time-saver. A new experiment shows it can also be a quiet liability.
Researchers at security firm Forcepoint X-Labs published a proof-of-concept study this week demonstrating that malicious instructions hidden inside an email's underlying code can hijack an AI summariser, causing it to show the reader entirely different facts from those the sender actually wrote. First reported by Dark Reading, the findings are a practical demonstration of a problem the security industry has discussed for years but rarely measured this concretely. We covered a related technique on 27 August, when researchers hid nearly 500 characters of secret instructions inside an ordinary-looking email and the AI obeyed every time.
How did the attack actually work?
The technique requires no hacking of your computer. It exploits the AI's inability to tell the difference between content it's supposed to read and commands it's supposed to follow.
The Forcepoint team built a test environment using a Microsoft Outlook add-in that passed email text to an AI language model, specifically Anthropic's Claude Haiku, which then produced a short summary. They buried a hidden instruction inside an otherwise ordinary-looking email, written in tiny white-on-white text, invisible in Outlook but fully readable by the AI processing the raw HTML underneath.
That hidden instruction told the AI to change specific details before displaying the summary. The original email stated an outstanding invoice of €8,750 and a date for a supplier review. The AI's summary showed €46,200 and a different date. Nothing flagged the discrepancy. The injection worked across all 10 runs.
What does this mean for ordinary people?
For now, the practical risk is targeted fraud. A criminal could send a supplier invoice with the real amount visible to a human reader but a larger figure buried in the HTML, then rely on a busy employee reading only the AI summary before approving payment. No phishing link, no fake login page, just a manipulated summary treated as fact.
The danger compounds if the AI assistant can act rather than merely summarise. An AI given permission to send replies or schedule meetings on your behalf could be directed to do those things by the same hidden instruction, with no human ever reviewing the underlying command.
| Detail | Benign email | Injected email summary |
|---|---|---|
| Invoice amount | €8,750 | €46,200 |
| Supplier review date | Original date | Altered date |
| Warning shown to reader | N/A | None |
| Injection success rate | N/A | 10 out of 10 runs |
Forcepoint researcher Ben Gibney was direct about the limits of the study: one model, one message, ten trials. Not evidence of a mass campaign. Evidence that the underlying vulnerability is reliable.
What should organisations do?
Forcepoint's recommendations are practical. Keep the instructions you give an AI model strictly separate from the outside content it reads. Treat everything an AI produces as a draft to be spot-checked, not a finished fact. Strip an AI assistant's permissions to the minimum it needs, so a successful injection can't trigger payments or outbound messages automatically. Scan incoming email for HTML tricks that hide text from human eyes while leaving it readable by machines.
If your organisation uses AI tools that read or act on external content such as email, documents or web pages, now is the time to inventory where those tools touch untrusted data. That list is your exposure, and in most organisations it has grown faster than anyone has mapped it.
The thing to watch is not this specific test setup but what happens when these summarisers get broader permissions. The attack surface described here is a preview, not the full picture.
