AI Coding Tools Carry Real Security Risks, and the Danger Depends on What You're Building With

Key points

• Researchers analysed 1,760 complete software projects generated by 16 AI coding tools and found more than 27,000 security flaws in total.
• The average AI-generated project contained 15 confirmed vulnerabilities, including 4.3 rated critical or high severity.
• Two Anthropic Claude models scored best on security (80.4 and 76.4 out of 100), while OpenAI GPT-4o Mini scored lowest at 21.6.
• The highest-risk software frameworks carried 40 times more exposure than the safest ones, according to software governance firm Secure Code Warrior.
• Seventeen specific weakness categories appeared in every single model tested, with five accounting for the largest share of real-world risk.

Artificial intelligence coding assistants, tools that write software code on a developer's behalf, have spread quickly across companies of every size. They save time. They cut costs. And, according to a new study, they reliably introduce security weaknesses that companies are only beginning to measure.

Software governance firm Secure Code Warrior published its AI Trust Index this week, built on research conducted with RMIT University. The team tested 16 AI models from vendors including OpenAI, Anthropic, and Google, generating 1,760 complete software projects and scanning each one for vulnerabilities, meaning flaws that an attacker could use to break into a system, steal data, or cause damage.

The headline number is stark. Across all projects, researchers identified more than 27,000 vulnerabilities. The average project carried 15 confirmed flaws, with 4.3 of those rated critical or high severity.

Does it matter which AI tool your company uses?

Yes, but the answer is more complicated than a simple ranking suggests.

Models were scored from 0 to 100, where a higher number means fewer security problems in the code produced. Anthropic's Claude Sonnet 4.5 and Claude Opus 4.5 led the field with scores of 80.4 and 76.4 respectively. OpenAI's GPT-4o Mini came last at 21.6. That sounds like a clear result. It is not the full picture.

The study found that a model's score shifts dramatically depending on which software framework, meaning the underlying toolkit a developer uses to build an application, is being tested. Claude Opus 4.5 scored a perfect 100 when building applications in Django, a popular web framework. The same model scored 57.3 in one C programming environment and just 28.5 in another. Eleven frameworks were tested in total, and the highest-risk ones carried forty times more exposure than the safest.

JavaScript and Java EE carried most of the danger. C# and Java Spring barely registered.

Cost offered no protection. Researchers found little correlation between what a company pays for a model and how safely it writes code.

The failures were also predictable, not random. Each model produced the same categories of mistakes repeatedly. The most common problems were things the AI forgot to do: checking that a user is who they claim to be (authentication), and checking that data entered into a form is safe before processing it (input validation). Seventeen weakness categories appeared across all 16 models tested.

Secure Code Warrior CEO Pieter Danhieux highlighted five that carry the most volume: sensitive data leaking into log files, cross-site scripting (where attackers inject malicious code into web pages that other users then load), hard-coded credentials (passwords written directly into the software itself), predictable session tokens (the digital passes that keep a user logged in), and path traversal (a technique that tricks software into accessing files it should not).

None of these is new. Security teams have had defences for all five for years. The task now is making sure those defences apply to AI-generated code with the same discipline applied to anything else a developer ships.

For companies using AI coding tools, the practical takeaway is to match your security scrutiny to the framework your team is building in, not just to the AI model they chose.