AI Security Scanners Are Drowning Teams in False Alarms, and the Fix Isn't More AI

Key points

• More than 60% of vulnerabilities flagged by application-scanning tools were false positives, unreachable code, or low-severity issues, according to Pixee CTO Arshan Dabirsiaghi.
• The Forum of Incident Response and Security Teams estimates the total number of formally tracked software flaws will rise 50% this year.
• One study found that 62% of open-source software libraries are never loaded when a program runs, meaning most flagged flaws sit in code nobody uses.
• Large language models flag different vulnerabilities on separate scans of the same code, making their output unreliable for consistent triage.
• Dabirsiaghi will present full test results at Black Hat USA in August 2026.

Security teams already spend most of their day chasing problems that turn out not to be problems. The tools meant to fix that are making it worse.

Arshan Dabirsiaghi, chief technology officer and co-founder of Pixee, an AI-focused application security company, tested more than a dozen software-scanning tools and found that six in ten alerts were junk: false positives (warnings about flaws that do not actually exist) or issues sitting in code the running application never touches. He will detail those findings at Black Hat USA in August.

Why does this matter to ordinary people?

Every app on your phone and every website you use runs on software. Security teams are supposed to find and fix the dangerous bugs before criminals do. When their tools cry wolf constantly, genuinely dangerous flaws get lost in the noise.

The failure mode is simple. Teams either automate everything and drown in low-quality patch requests, or they assign a human to check every alert. Dabirsiaghi puts the share of alerts that represent a real, reachable, exploitable flaw at around 8%. Nobody has enough staff to manually review 100 warnings to find the eight that matter.

AI was supposed to change that arithmetic. It hasn't.

Should you worry about LLMs making this worse?

Large language models (AI systems trained on enormous libraries of text and code) struggle because they work without context. They see a line of code that looks risky and flag it, without knowing how the wider application actually runs in production. Worse, they are inconsistent. Run the same scan twice and the model may call the same flaw a false positive one time and a genuine vulnerability the next. "LLMs are dumb," Dabirsiaghi said. "It'll just say there's a vulnerability on line X and go fix it even if it's not real."

Over-fixing is its own hazard. Patching code that was never broken can introduce new bugs, slow the application down, and erode the engineering team's trust in the security process entirely. As we reported on 6 July, machine-generated code is already flooding into production with fewer human eyes on it; adding automated over-patching on top compounds the exposure.

What actually cuts the noise?

The practical fix Dabirsiaghi argues for is reachability analysis: checking whether the vulnerable code is ever actually executed when the software runs. Research backs this up. One study found that 62% of open-source libraries are never loaded at runtime. Another found that of the open-source portions of Java applications (Java being one of the most widely used languages in business software), only 12% of that code is ever called. Cutting alerts to only reachable code would slash the workload dramatically.

Good triage also needs three layers of context: organisational, architectural, and code-level. A flagged use of MD5 (an older data-scrambling method) might be critical if it is protecting passwords, or completely irrelevant if it is used for a non-security internal task. Without that context, an AI model defaults to a generic severity rating that helps nobody.

The Forum of Incident Response and Security Teams estimates the number of formally catalogued software flaws, each assigned a CVE (Common Vulnerabilities and Exposures) identifier, will grow 50% in 2025. More volume, noisier tools, no extra headcount.

Reachability filtering is not glamorous. It does not ship as a slick vendor dashboard or get announced at a press event. But for any team running static analysis on AWS CodeGuru, Snyk, or similar scanners, it is the fastest lever available to cut the real alert queue without hiring. The vendors selling AI triage as a complete answer are getting ahead of what the technology can actually do right now, and Dabirsiaghi's August session will be worth watching for the numbers that prove it.