Security and governance: the burning platform?
The widespread enterprise adoption of agents has exposed security gaps that static or traditional application security frameworks simply do not, or cannot, address. For example, a compromised agent can easily exfiltrate data, manipulate organisational processes, or escalate privileges before detection systems are aware there’s a problem.
And this is happening despite global organisations spending more than ever on security and governance. BCG’s latest survey of cybersecurity leaders shows that 83% are spending more on cyber, yet they still struggle to keep pace with emerging threats. Nearly nine in ten faced some form of cyber-attacks in the last year.
In practice, many security experts believe that agentic access should be strictly limited to the specific APIs, tools and data necessary for their designated function. Emerging governance frameworks routinely inject task-specific tool lists into system prompts, limiting agent awareness to approved capabilities before each session. Other standard controls might include:
· Separate runtime enforcement to prevent agents from utilising tools outside their scope
· Contextual privilege adjustment to allow agent permissions to vary based on current risk level, transaction value, or customer sensitivity
· Time-bounded access and multi-step approval requirements for high-impact actions.
Identity governance for agents remains at an immature stage but it’s essential for organisations to be able to distinguish between API keys representing agents from keys representing human service accounts, confirm agent identities specifically and track which agent initiated which action. Audit trails must record not just that an action occurred but provide context on which agent decided to act and what context was available to trigger the decision. Compliance frameworks such as ISO 42001, NIST AI Risk Management Framework, and MITRE ATLAS now explicitly require governance documentation and audit capabilities for AI systems.
Evaluation and observability in this context are critical. Organisations of all sizes and scales require baseline definitions of normal agent behaviour, so that any deviations become visible. Execution limits on loops, spend, and repeated tool calls prevent runaway workflows. Reversibility— the ability to stop an agent, its access, and contain its effects without manual interventions—transforms agent failures from incidents into recoverable events.
All these guardrails represent positive security developments but one only needs to look back to Gartner's 2025 research which found that 74% of IT application leaders viewed AI agents as a new attack vector, yet only 13% strongly agreed their organisations had adequate governance structures. This delta remains a significant obstacle to production-scale deployment.
Read Brad’s full piece here: Project Glasswing, Claude Mythos and what “Secure AI” really means for organisations | Version 1
Our CTO’s take: What does “Secure AI” actually mean?
This is a phrase that gets thrown around a lot in boardrooms and it means at least two very different things, both of which matter.
The first meaning is securing AI systems against attack.
If you’re deploying AI models in your business, whether that’s customer service agents, coding assistants, or decision support tools, those systems need to be hardened. Prompt injection, data poisoning, model theft, adversarial inputs etc. These are real attack vectors and they’re getting more sophisticated. CrowdStrike’s take on the Mythos announcement is worth noting: whoever builds the model is responsible for what it can do, but securing how it runs inside your environment? That’s on you. If an AI agent connects to your CRM, queries your database, or triggers a workflow, that’s not a model safety question. It’s a deployment governance question.
The second meaning is using AI to secure your systems better.
That’s what Glasswing is doing. Using AI’s ability to reason about code, spot patterns humans miss and work tirelessly to find vulnerabilities before attackers do. This isn’t theoretical anymore. Mythos found bugs that survived 27 years of expert human review. If you’re running internet-facing services, legacy applications, or open-source dependencies (and you are), then AI-powered vulnerability scanning is no longer a nice to have.