Someone Finally Tested 100 AI Agents for Security. Here's the Framework They Used.

A research effort has put 100 AI agents through a structured security evaluation and published rankings — a rare attempt to apply consistent methodology to a space where 'secure by design' mostly means a blog post.

The framework doing the scoring is called the AI Risk Quadrant. It grades agents on three axes: susceptibility to compromise, potential impact if that compromise succeeds, and the strength of whatever defenses the agent actually has in place. Think of it as a rough analogue to CVSS — severity times exploitability, with a defensive-controls modifier bolted on.

That framing matters. A lot of AI security discourse collapses 'can you jailbreak this model' and 'what happens when you do' into a single scary number. Separating compromise likelihood from downstream impact is the right call. An agent with read-only access to a public knowledge base and an agent with write access to a production database can both be prompt-injected; the consequences are not remotely comparable.

The 'defensive strength' axis is where things get interesting — and contested. Defenses for AI agents are still largely informal: system-prompt hardening, output filtering, tool-call validation, rate limiting. None of it is standardized. Measuring it across 100 heterogeneous agents implies someone made judgment calls that deserve scrutiny.

What the rankings actually reveal about specific agents hasn't been widely detailed yet, but the existence of the methodology is the real story. The AI agent market has exploded faster than any shared security baseline could form. Vendors ship agents with access to calendars, codebases, customer data, and internal APIs — sometimes all four — without publishing anything resembling a threat model.

Prompt injection remains the canonical attack surface here. It is, at its core, an input-validation failure dressed in transformer weights. Classic stuff. The fact that it still works reliably against production agents in 2025 says something unflattering about how seriously the category takes security fundamentals.

A ranked list of 100 agents won't fix that. But a repeatable evaluation framework, applied consistently and publicly, creates accountability pressure that vendor white papers do not. If the methodology holds up to peer review, it's a useful tool. If it doesn't, picking it apart will still advance the conversation.

Either way, the industry needed someone to start counting.