You Can't Audit What You Can't See: The Agent Governance Hole Nobody Wants to Talk About

Fifty-four percent of enterprise AI teams can't fully trace what their agents are doing. Fifty-six percent have no centralized governance layer. Those numbers come from a TrueFoundry survey of over 200 enterprise AI leaders — a vendor with skin in the game, yes, but the pattern they're describing is corroborated by practitioners across the stack.

The core problem isn't novel. It's the classic "deploy first, secure later" cycle, this time running on non-deterministic infrastructure.

T.J. Marlin, CEO of Guardrail Technologies, draws a useful distinction: organizations keep comparing AI agents to robotic process automation, where outputs are largely deterministic. Agents aren't that. "There's a talent shortage and a knowledge shortage and people are building at pace without checking whether it's correct and it's operating as expected," he says. "Those are all the hallmarks of the worst disasters that I've seen across my career."

The debugging model has also broken. With traditional software, QA and security teams could read the code. Adel El Hallak, VP of AI software at Nvidia, makes the point plainly: for agents, the source of truth lives in execution traces, not source files. An agent deciding which tool to call, which data to pull, which action to take — that decision happens at runtime inside a model. There's no static artifact to audit after the fact.

Mahesh Kumar Goyal, a senior data and AI expert at Google, raises a point that should unsettle anyone running a SIEM: traditional detection tooling was built to flag human behavioral anomalies. An agent executing the same code flawlessly ten thousand times looks perfectly normal to that tooling, even if it has been hijacked. Prompt injection at scale would do exactly that — a compromised agent behaving consistently, just toward the wrong objective.

Least-privilege scoping, policy enforcement layers that mediate every prompt and tool call, end-to-end tracing that connects inputs to downstream actions — Goyal frames this as the financial-infrastructure model applied to agents. Auditability, reconciliation, circuit breakers. Not trust.

The governance response has its own failure mode, though. Nirmal Ganesh at Box describes teams that defaulted to requiring human approval on every agent output. That sounds safe. In practice it's just manual process with extra latency. "At a high volume, governance is your bottleneck to scale and no longer your safety net," he says. The answer isn't less governance; it's tiered autonomy — tight human-in-the-loop on consequential actions, lighter touch on low-stakes ones.

Marcelo Lorenzetti, founder of SavvyLex, frames the compliance dimension cleanly. Organizations need to prove what an agent accessed, what instructions it followed, what tools it invoked, and where a human intervened. Screenshots and after-the-fact logs won't satisfy legal or regulatory requirements. Immutable activity records, confidence scoring, and explicit escalation paths aren't nice-to-haves; they're the minimum bar for any agent operating in a regulated context.

Observability is necessary. It's not sufficient. Collecting traces only matters if the system lets you act on them — annotate, adjust, retrain. The signal has to close a loop.