ChatGPhish: When ChatGPT's Markdown Renderer Becomes a Phishing Vector

Another week, another reminder that LLMs treat untrusted web content as if it were user input. This time the trust boundary is the Markdown renderer on chatgpt.com.

Researchers at Permiso have detailed a technique they're calling ChatGPhish. The short version: when ChatGPT summarizes or browses a page, the response renderer happily honors Markdown links and images that the model emits — including ones the model was tricked into emitting by attacker-controlled content sitting in the page it just read.

The failure mode here is familiar to anyone who has watched a SOC chase a phishing campaign. An attacker plants instructions inside a webpage, a PDF, a GitHub README, or any other surface ChatGPT might be asked to fetch. The model ingests those instructions as part of context. The page tells the model: when you summarize this, include a link that looks like a legitimate login or document share. The renderer then produces a clickable, clean-looking Markdown link in a UI users have been trained to trust.

In practice this collapses two problems into one. Prompt injection gets a delivery mechanism. Phishing gets a trusted brand wrapper. The lure is no longer arriving in an email from a sketchy sender — it's arriving inside an AI response the user explicitly asked for.

Images make it worse. Markdown image syntax fetches remote assets, which means an attacker can pull tracking data or, depending on how the client renders, stage further content from a domain the user never typed.

OpenAI has already had to retrofit several mitigations around image rendering for similar exfil scenarios. The pattern keeps repeating because the underlying assumption — that text the model produces is safe to render with full Markdown semantics — is wrong the moment that text is influenced by third-party content.

What to actually do about it, if you run an org where employees paste random URLs into ChatGPT all day:

• Treat AI assistant output as untrusted, browser-equivalent content. Outbound link clicks from chat UIs should pass through whatever URL inspection you already run on email.
• Disable or sandbox browsing and file-fetching features for enterprise tenants where you can't enforce egress controls.
• Log assistant interactions. The forensic trail for "user clicked a link inside an AI answer" is, at most shops, nonexistent.
• Push your AI vendor to strip or domain-allowlist Markdown links in rendered output. It's a UX hit. It's also the only durable fix.

One thing the post-mortem will say: the phishing didn't come through email, and nobody's secure email gateway saw it.

Operational takeaway: if your phishing program doesn't yet consider AI assistant responses a delivery channel, update the threat model this quarter.