AI is getting better at breaking software than fixing it

Key points

• Veracode found that 44% of code written by AI contains at least one serious, well-known security flaw, based on tests of more than 100 model versions over the past year.
• No AI model tested by Veracode achieved a security pass rate above 68%, meaning every model produced unsafe code at least once in every three attempts.
• A separate study by 1Password's Off-By-1 Labs found that AI-generated fixes for known software vulnerabilities worked correctly only 26% of the time across more than 6,000 test runs.
• Researchers at Xint found 434 confirmed security flaws across 28 apps built or rebuilt by AI, including secrets accidentally baked into the code and runaway resource use.
• Experts say better training data, not just more training, is the most plausible path to improvement, but no major AI lab appears to be taking that approach yet.

AI coding assistants, the tools that help software developers write programs faster by suggesting or generating code automatically, are getting remarkably good at one thing: finding and exploiting weaknesses in software. They are not getting nearly as good at the other thing everyone hoped they would do, which is write safe software in the first place.

That gap is widening, and several research teams have now measured it carefully.

How bad is the problem, exactly?

Bad enough to be a consistent pattern across independent studies. Veracode, an application security company, tested more than 100 versions of AI coding models over the past year and found that 44% of the code those models produced contained at least one flaw from the OWASP Top 10, a widely recognised list of the most common and dangerous software vulnerabilities. Even the best-performing models failed the security check roughly one time in three.

What makes that figure striking is the contrast with syntax, the basic grammar of code. Every model tested got syntax right 99% of the time. The models are nearly perfect at writing code that runs. They are far less reliable at writing code that runs safely.

Dutch software quality firm Software Improvement Group found a similar pattern: AI-generated code produces roughly twice the security-rule violations compared with code written by humans. Its chief technology officer, Jasper Geurts, was careful to note that AI did not create the underlying problem. It amplified one that already existed.

Why can't AI just fix the holes it finds?

Fixing a flaw turns out to be harder than spotting one. To exploit a vulnerability, an AI only needs to find one working path in. To patch it properly, the AI must close every possible path, including ones that look slightly different in the code but lead to the same weakness.

Researchers at 1Password's Off-By-1 Labs saw this directly. They asked several AI models to generate patches for six real, previously fixed vulnerabilities in complex open-source software. Out of more than 6,000 attempts, the AI produced a correct, complete fix only 26% of the time. More than half the AI-generated patches either failed to close the hole, introduced a brand-new flaw, or did both at once.

The team noted in section 7.3 of their research paper that an AI agent sometimes fixed only the specific version of a bug shown in a test case, while missing character-for-character identical copies of the same bug sitting in nearby code. That points to a structural limitation in how current AI models process context, not simply a training gap that more data will cure.

What should developers, and their organisations, do right now?

No single fix exists yet. Experts across all three studies agree on a few practical steps, though. Teams should treat AI-generated code the way they treat code from a junior developer who works very fast: review it, run automated security scans against it, and never assume it is safe simply because it compiles and runs.

For ordinary people, the implication is indirect but real. Software built partly or entirely by AI tools, which now means a large and growing share of apps and websites, may carry security weaknesses that the developers themselves did not write and may not have noticed. If a service you use announces a data breach, it is worth checking whether your login credentials or personal details were exposed, changing your password for that service, and watching for unusual activity on any linked accounts.

Veracode's Chris Wysopal argues the root fix sits with the AI labs themselves. He believes they should stop training on the broad, mixed-quality code available publicly and instead license tightly reviewed code from industries where correctness is non-negotiable, such as banking software or aviation systems. As of now, no major lab appears to be doing that.

Common questions

Does this mean AI coding tools are too dangerous to use?

Not necessarily, but they need supervision. AI tools can speed up development significantly when teams also run security scans and code reviews on everything the AI produces. The risk comes from treating AI output as finished, verified work.

Will this get better as AI models improve?

Possibly, but not automatically. Researchers point out that the attention mechanisms built into current AI models may make complete vulnerability patching structurally difficult, regardless of how much additional training a model receives. Better-quality training data aimed specifically at secure coding would help more than simply training on more code.