Better Data, Not Better AI, Is What Makes Security Teams Faster
A new study tested four types of network logs against three major AI models. The data source mattered far more than which AI you picked.
Key points
- High-quality network logs improved AI-assisted security investigation accuracy to 95.2%, compared with 25.8% for basic connection records.
- Investigation time dropped to 14.7 minutes with rich logs, versus 27 minutes with basic flow data.
- The research used simulated versions of two real espionage campaigns: Volt Typhoon and Salt Typhoon, both tracked as Chinese state-linked intrusion clusters.
- Model upgrades alone cannot fix poor data: swapping AI providers made far less difference than improving what the AI was shown.
- Critical incident visibility was five times higher with enriched logs than with basic connection records.
Security teams across large organisations are rushing to deploy AI tools that can sort through thousands of alerts, pull out the suspicious ones, and write up incident reports automatically. The promise is real. But a research project published this year suggests most teams are asking the wrong question when they shop for AI.
The question most leaders ask: which AI model is best? The answer the research gives: it barely matters, so long as your underlying data is poor.
How did the researchers test this?
The study, called Provably Better Data, ran a controlled experiment using two well-known attack scenarios. One was built around Volt Typhoon, a hacking group tracked by Microsoft and other vendors as a Chinese state-sponsored cluster known for quietly embedding itself in critical infrastructure networks. The other used Salt Typhoon, a separate Chinese espionage group that has targeted telecommunications providers.
The researchers fed four different types of network logs, basically records of what computers did and said to each other, into three AI models: Anthropic Claude Opus 4.6, Google Gemini Pro 3.1 Preview, and older versions of both. Every model received identical instructions. The only thing that changed was the quality of the logs.
The four log types ranged from "enriched" logs that decode individual network protocols in fine detail, down to NetFlow records, which are basic connection summaries that note two computers talked but not what they said.
What did the numbers show?
The gap was stark.
| Data source | CTF accuracy | IR evidence coverage |
|---|---|---|
| Corelight enriched logs | 95.2% | 90.3% |
| Firewall logs | 58.3% | 61.3% |
| Snort 3 IDS alerts | 39.4% | 21.2% |
| NetFlow records | 25.8% | 30.9% |
For critical investigation questions, enriched logs let models answer 91.7% of mandatory questions. Basic NetFlow records supported just 18.3%.
Speed told the same story. Rich logs produced a completed investigation in 14.7 minutes. Basic logs took 27 minutes, because the AI kept hitting dead ends and retrying. Worse data nearly doubled the time spent.
One concrete example illustrates why. Investigators asked the AI to identify a computer by name from its network address. Enriched logs contained the answer directly: a computer called FINANCE01. Basic firewall logs could see that a certain security protocol had been used, but had never recorded the actual computer name. The AI could not guess what was never written down.
Does the AI ever make things up?
Rarely, when instructed correctly. The researchers told models to mark any question they could not answer as unanswerable rather than guess. Hallucinations, meaning confident but fabricated answers, stayed at zero for three of the four data sources. Snort 3 alerts produced an average of 1.9 fabricated answers per run, likely because partial information tempted the model to fill gaps.
The lesson is practical: structured instructions plus complete data keeps AI grounded.
What should organisations do?
This research, first covered in detail by CSO Online, carries a clear message for anyone deciding where to spend a security budget. Richer network telemetry, the detailed logs that capture what protocols did and what data moved, should come before another AI licence renewal.
For ordinary employees, the finding is mostly reassuring: AI-assisted security tools, when fed good data, can investigate incidents faster and with fewer mistakes than before. The catch is that "good data" requires infrastructure investment that is invisible to most staff. If your organisation's security team is asking for budget to upgrade its logging systems, this research gives them solid ground to stand on.
