Architecting defenses: Essential capabilities for the AI era
Securing AI at scale requires a broader view than most organisations start with — one that accounts for multiple stakeholders, not just a single control point. When I sit down with security leaders to think through what “architecting AI defenses” actually means, my advice is consistently the same: focus on the capabilities you need, not on locking yourself into a specific product architecture that may not survive the next eighteen months of AI tooling change. The underlying capabilities are far more durable than any single implementation of them.
Before you can architect anything, you need a common language for what you’re defending against — and this is where established frameworks earn their keep. Not as regulatory checkboxes, but as blueprints that different teams can point to and mean the same thing.
For the developers and architects actually building with large language models and agents, the OWASP Top 10 for LLM Applications has become the closest thing to a shared reference for application-level risk — prompt injection, sensitive information disclosure, supply chain weaknesses, and excessive agency, among others. OWASP’s newer companion, the Top 10 for Agentic Applications, extends that same thinking to the harder problem: what happens when a system doesn’t just generate text, but plans, acts, and delegates across tools on its own. That distinction matters, because a defense built only around what a model says won’t catch a risk created by what an agent does.
For governance, the NIST AI Risk Management Framework organizes the work into four connected functions — Govern, Map, Measure, and Manage — giving organisations a voluntary but increasingly standard way to bring trustworthiness considerations into how AI systems are designed, developed, and evaluated. It’s a lifecycle model, not a one-time exercise.
And for understanding how attackers actually operate, MITRE ATLAS fills a different gap: a living knowledge base, modeled on the same logic as MITRE ATT&CK, that catalogs adversary tactics and techniques aimed specifically at AI and machine learning systems, along with real-world case studies and mitigations. Where OWASP tells a builder what to guard against and NIST tells a governance team how to structure oversight, ATLAS tells a defender what the attack actually looks like in the wild.
What I find most of these frameworks share is an assumption that AI risk isn’t confined to the model itself — it runs through data pipelines, infrastructure, and every component in between. That’s the same premise behind the Databricks AI Security Framework (DASF) 2.0, which maps risk across the full set of canonical components that make up an AI system — from data ingestion through model development to deployment and monitoring — and deliberately cross-walks its controls back to standards like MITRE ATLAS, NIST, and OWASP rather than inventing a competing taxonomy.
I find that kind of cross-referencing genuinely useful, because it reflects how security, data, and platform teams actually have to work together on AI — no single framework covers the whole surface, and organisations that pick just one tend to end up with blind spots exactly where the frameworks don’t overlap. Used together, they give a CISO something concrete to point to in an audit and a shared map for where each team’s responsibility starts and ends.
One capability I’d put near the top of any list: the ability to genuinely scrutinize your AI supply chain, not just take a vendor’s word for it. Can your AI and Model Context Protocol (MCP) vendors demonstrably show how they build and use AI safely themselves? Context-aware risk assessments that continuously evaluate third-party models, skills, plugins, and packages need to be standing practice, not a one-time vendor questionnaire.
This extends to development itself. Agent red-teaming belongs inside the development cycle, testing the infrastructure and training data an organisation is building on — not bolted on afterward as a compliance formality once something’s already in production.
Architecturally, most organisations I talk to eventually converge on the same underlying need: a “secure-by-design” posture built around some form of centralised control point for AI communications — however it ends up being implemented in their specific environment.
The capability that matters here is the ability to centrally manage how large language models, MCP connections, and agent-to-agent communications are governed, so that sanctioned usage can be enforced consistently rather than piecemeal across departments. Tied to that is agent identity security — treating every autonomous agent as an identity that needs least-privileged access, the same discipline organisations have spent the last decade applying to human and service accounts. This is precisely the excessive-agency risk OWASP’s frameworks call out directly: an agent that can read and act across a dozen systems with no meaningful access boundary is a supply chain risk in its own right, regardless of how well the underlying model behaves.
The last capability I’d flag is the one people underestimate most: treating AI systems as dynamic, living infrastructure rather than a static deployment you configure once and walk away from. These systems interact with users, cloud databases, and APIs continuously, in real time, and their behavior can shift in ways a point-in-time security review — or a framework checklist completed once at launch — will never catch.
That argues for a runtime layer of protection — something that can recognize and act on threats like prompt injection, model denial-of-service, or sensitive data exfiltration at the moment they actually happen, not after the fact in a log review. The goal of this capability isn’t just detection; it’s the ability to actively shape agent behavior, access, and risk in real time, which is what actually prevents a small anomaly from becoming a backdoor.
None of this depends on a specific vendor or product architecture. What it depends on is an organisation deciding, deliberately, which of these capabilities it actually has today — and being honest about which ones it doesn’t.
