2.47 Million Simulated Attacks Suggest We're Measuring the Wrong Things in Phishing Training
New research points out a gap between what most companies track (who clicked a fake link) and what actually matters (whether stolen passwords are being spotted and staff are reporting suspicious emails).
Key points
- A study analysing 2.47 million simulated phishing attacks found that click rates alone are a poor measure of whether staff can actually resist real attacks.
- Researchers say organisations should also track credential theft, meaning how often fake login pages successfully capture staff passwords, and how many suspicious emails employees report.
- Most companies currently treat a drop in click rates as proof their training is working, but the research suggests that view is too narrow.
- Security awareness testing, the regular practice of sending fake scam emails to employees to see who falls for them, needs a broader scorecard.
Most security teams running phishing simulations have one number they care about: the click rate. How many people clicked the dodgy link goes into a dashboard, gets reported to the board, and shapes next quarter's training budget.
New research, covered by SecurityWeek, suggests that one number isn't enough.
An analysis of 2.47 million simulated attacks found two problems with a click-only view. First, it ignores credential harvesting, which is what happens when a fake login page quietly records the username and password a victim types in. Clicking a link is bad. Handing over your actual password is worse, yet plenty of organisations never measure whether their staff are doing exactly that.
Second, it ignores the other side of the equation entirely: reporting. A phishing email that gets spotted and flagged by an employee before anyone clicks is a win. Most simulations record that win as a neutral result, because nobody clicked. That's a flawed accounting.
The failure mode here is familiar to anyone who's written a postmortem. You measure what's easy to instrument, not what actually matters. Click rates produce a clean percentage that trends down over time, keeping everyone comfortable.
But a real phishing attack, the kind that empties a bank account or hands criminals the keys to a corporate email system, doesn't care about your click-rate trend. It cares whether one person on a Monday morning types their password into a convincing fake login page for a cloud-based office platform. We reported on 10 September that half of all real attacks now target login credentials, and that identity abuse is still doing most of the damage across enterprise environments.
What should companies do differently?
Three changes follow directly from the research. Organisations running phishing simulations should add fake login pages to some of their tests and record how many staff actually type in credentials. They should build a clear reporting channel so staff can flag suspicious emails with one click, then measure how often that happens. Both metrics, credential submission rates and reporting rates, belong alongside click rates in any security dashboard.
For employees, the practical implication is direct: if an email asks you to log in somewhere, type the address yourself rather than clicking anything in the message. If something looks off, tell your IT or security team, even without certainty.
One thing the postmortem will say, after the real breach, is that several people saw the email, thought it looked suspicious, and said nothing, because nobody told them that saying something was part of the job.
Measure what causes harm, not what fits neatly in a report.
