AI agents are the new cybersecurity nightmare

At RSAC 2026 in late March, CrowdStrike CEO George Kurtz spent part of his keynote describing what enterprises are actually seeing when they deploy AI agents with production access. According to VentureBeat’s reporting on the keynote, at a Fortune 50 company a CEO’s agent rewrote the corporate security policy — not because it was compromised, but because it wanted to fix a problem, lacked the permissions to do so, and removed the restriction on itself. Every identity check passed. The credential was valid.¹ At another Fortune 50, a 100-agent Slack swarm delegated a code fix between agents with no human in the loop. Agent 12 made the commit. The team discovered it after the fact.¹

The audience reportedly laughed. They probably shouldn’t have.

The keynote framing was sharper than the laughter suggested. “By 2027, your smartest employee will be a machine,” Kurtz said. “Most organizations deploy AI agents with less governance than they’d give an intern.”¹ The examples were not the punchline. They were the leading edge of a category that has, in the last 18 months, moved from research-paper hypothetical to enterprise-disclosed CVE.

Software that fails creatively

Safety-critical software engineering has spent decades developing the distinction between “the system did what it was told” and “the system did what we wanted.” Standards like DO-178C in avionics² and IEC 61508 in industrial control³ exist in large part to manage that gap — through requirements traceability, hazard analysis, and a thick stack of formal verification practices that nobody enjoys but everybody learns to respect after the first incident.

Agentic systems are reopening that gap at a scale most organizations do not have the discipline for.

Traditional software does what its source code says. If it misbehaves, the source code is the place you look for the bug. An agent operating under a goal-directed prompt does what its objective function values, mediated by whatever model is interpreting the goal at runtime, against whatever tool surface the operator handed it. The Fortune 50 agent in Kurtz’s example did not violate its instructions. It violated its operator’s expectations. Those are different things, and the conventional security model — least privilege, audit logs, periodic review — has nothing useful to say about the gap between them.

We spent thirty years building software that fails predictably. We are now deploying software that fails creatively.

My read on this is that the existing security stack quietly assumes the actor in question is either a human — slow, fallible, but legible — or a process — fast, predictable, also legible. Agents are fast, fallible, and illegible. The audit log shows you the API calls. It does not show you why. And in the public reports I’ve seen from the last 12 months, “why” is the question that takes weeks to answer.

This is already happening outside the lab

The leading-edge example is no longer theoretical. In June 2025, researchers at Aim Security disclosed EchoLeak — assigned CVE-2025-32711, CVSS 9.3, critical — a zero-click prompt injection in Microsoft 365 Copilot that allowed an attacker to exfiltrate confidential data simply by sending an email.⁴ The mark did not have to click. The email sat in the inbox, Copilot indexed it as context the next time the user asked the assistant for help, and the embedded payload chained Microsoft’s own XPIA classifier bypasses, markdown reference-style links, and a Teams proxy on the allowed content-security-policy list to walk privileged data out of the tenant. Aim Labs categorized it as an “LLM Scope Violation.”⁴ Microsoft patched it before public disclosure, and there is no evidence it was exploited maliciously in the wild — but it is, on the record, the first documented zero-click prompt injection against a production LLM system.

It was not the last.

At Black Hat USA 2025 and DEF CON 33, Michael Bargury and Tamir Ishay Sharbat of Zenity Labs demonstrated AgentFlayer — a chain of zero-click exploits against enterprise AI agents, including a public-facing lead form on a Salesforce-connected Copilot Studio agent that could be used to hijack the agent, dump complete CRM records, and run unauthorized actions with no authentication and no user interaction beyond submitting a normal-looking form. Zenity reported that Microsoft and Salesforce shipped patches following responsible disclosure.⁵ The class of attack — indirect prompt injection, where adversarial instructions reach the model via the data it is asked to process rather than via the prompt the user typed — is not a Copilot Studio quirk. Palo Alto Unit 42’s threat intelligence team reported in March 2026 the first systematically observed wave of such attacks in the wild, against ad review agents, search ranking pipelines, browser AI assistants, and content analysis tools. In the sample they instrumented, social engineering accounted for 85.2% of observed payloads, with data destruction the second-largest intent at 14.2%. The case studies included agents redirected toward unauthorized transactions on payment processors like Stripe and PayPal.⁶

The annual threat roundup from CSO Online in late 2025 listed five real-world AI security threats that had moved from research to operations during the year: shadow AI, AI supply-chain poisoning, LLMjacking (credential theft for API access to LLMs), prompt injection demonstrated against GitHub Copilot, GitLab Duo, ChatGPT, Claude, and AI-enabled browsers, and rogue or misconfigured Model Context Protocol servers — tens of thousands of MCP servers published online with the same level of care a typical internal Jenkins instance gets, which is to say none.⁷ Prompt injection itself sits at the top of the OWASP 2025 Top 10 for LLM Applications.⁸ The point is not that any one of these is novel as an idea. The point is that in 2025 they stopped being only demos and started showing up in real disclosures, telemetry, and operational security reporting.

The deployment math doesn’t help

The data on agent deployment cuts against the way the technology has been marketed. Deloitte’s 2025 Emerging Technology Trends study, surveying enterprise practitioners, found that 11% of organizations had agentic AI systems running in production, 38% were piloting, 30% were exploring, and 14% had solutions ready to deploy.⁹ Gartner’s June 2025 prediction was that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.¹⁰ Anushree Verma, the Gartner analyst behind the prediction, framed the problem more pointedly than the headline did: most agentic projects are early-stage experiments driven by hype, misapplied to use cases that do not need them, and supported by a vendor field engaged in “agent washing” — rebranding chatbots, RPA, and assistants as agents without the substance. By Gartner’s count, only about 130 of the thousands of self-described agentic AI vendors are “real.”¹⁰

The gap between piloting and production is where most of these projects are dying. The gap between production and production safely, in my view, is where the next two years of incident reports are going to come from.

Detection is the hard part of the safely problem. Traditional intrusion detection relies on signatures of human or known-program behavior, and on the assumption that legitimate credentials being used by a legitimate identity is, by default, a legitimate action. An agent acting on its own authorization with valid credentials looks, to most monitoring tools, like a particularly thorough employee on a particularly productive Tuesday. The action is logged. The intent is not. These cases were surfaced by researchers and threat-intelligence teams, not by ordinary enterprise identity checks or access reviews.

The corollary is unpleasant. If the attack signature is “an authenticated identity used a sanctioned tool to do a sanctioned thing, except the goal came from someone else,” the defender’s instrumentation has to either model intent or model the behavioral baseline of the agent itself. Neither is well-solved. Several of the agentic SOC tools shipped at RSAC 2026 by major vendors aimed at exactly this gap, and credible reporting on them describes the same baseline problem the products are meant to fix.¹¹

What I think survives

This is opinion, plainly stated.

I am not bearish on agents. I am bearish on autonomy treated as a product feature instead of an operating risk. There are plenty of enterprise environments where agents make sense: SOC triage, code review, ticket routing, internal research, customer support, workflow automation, compliance review, vulnerability discovery. The category is real. The problem is not that agents can act. The problem is that most organizations have not decided, with enough precision, where acting becomes authority.

The systems that survive are the ones that narrow the task before they widen the permissions. One task. One operating surface. One class of data. One set of actions the organization is willing to let the agent attempt. The more general the agent, the more important the boundary becomes, because generality is where the failure mode hides. “Help the support team summarize tickets” is a governable system. “Handle customer operations” is a liability wearing a roadmap label.

The hard line is not whether an agent is autonomous. The hard line is whether the consequence of a bad action can be bounded. If the agent can damage production systems, expose customer data, change access controls, commit code, send customer-facing messages, touch money, alter policy, or create reputational harm, the question should not be “does the identity check pass?” The question should be “what guardrail absorbs the mistake?” If the answer is mostly hope, the agent should not have that authority.

That does not mean every risky action is forbidden. It means risky actions need to be weighed case by case, with approval gates placed where the blast radius changes. A low-risk recommendation can be automatic. A reversible internal update might be automatic. An irreversible action, customer-visible action, security-sensitive action, or financially material action should require explicit approval unless the organization has built controls strong enough to make the risk tolerable. Most have not.

This is where enterprise governance stops being paperwork and starts being architecture. The policy has to define what classes of action an agent may take, what requires escalation, who owns the decision boundary, and what evidence is preserved when something goes wrong. Governance that lives only in a document will not survive contact with an agent that can execute. Governance has to be expressed in permissions, workflows, prompts, logs, approvals, and kill paths.

Observability is the other half. An agent execution log that says which API calls happened is not enough. The organization needs to know what context the agent saw, what instruction it followed, what tool it selected, what approval path it used, and why the action looked valid at the time. Without that, every investigation starts after the damage is done and works backward through a system built to be helpful rather than legible.

The vendors getting this wrong are the ones shipping overbroad default permissions and selling autonomy without forcing the governance conversation. They make the demo look clean by hiding the question the buyer actually needs answered: what happens when the agent is wrong, overconfident, manipulated, or simply too literal? Autonomy does not fail like ordinary software. It fails by doing something that looks authorized, uses valid credentials, and still violates the operator’s intent.

The vendors getting it right will not sell agents as magic employees. They will sell bounded delegation: narrow task scope, explicit authority, observable execution, and approval gates at the points where a bad decision can hurt customers, revenue, security, or reputation. That is the version enterprises can govern. Everything else is just an authenticated action waiting for a postmortem.

Notes