I’m a father of four, and I’ve recently started teaching my oldest daughter how to drive, which is an experience all by itself. To be clear, she isn’t taking the car anywhere on her own yet. But that day is coming soon, which is both exciting for her and slightly terrifying for me. 😬
Teaching her to drive has also made me think a lot about autonomy. Right now, I’m sitting next to her. I can see what she sees, give her direction and intervene if I need to. Over time, though, that changes. Eventually I have to trust her to take the car without me sitting in the passenger seat. Giving her that independence doesn’t mean the rules disappear. In many ways, the rules become more important because I’m no longer there to supervise every decision.
And those rules aren’t as simple as “you can take the car” or “you can’t take the car.” Where are you going? Who is going with you? What time will you be home? Is the weather bad? Are you taking one of your siblings somewhere? What happens if the plans change? Sometimes the answer will be yes, sometimes it will be no, and sometimes it will be yes under certain conditions or only after checking with me first.
The more autonomy you give someone, the more important that context becomes. As I’ve been thinking about where we are headed with AI agents, I’ve realized we are dealing with a remarkably similar problem.
We’ve Started Calling Everything a Guardrail
“Guardrails” has become one of those terms in AI that can mean almost anything. Input and output filtering, tool restrictions, prompt instructions, content policies, schema validation, safety classifiers and access controls can all get grouped under the same heading. There is nothing inherently wrong with that, and guardrails are an important part of building safe AI systems. But I think we risk confusing guardrails with governance, particularly as AI systems move from generating information to taking action.
There is a fundamental difference between telling an AI system what it should and shouldn’t do and having an independent mechanism capable of enforcing those decisions while the system is operating. That distinction may not have seemed as important when our primary interaction with AI was asking a question and receiving an answer. It becomes much more important when the AI can independently decide what to do next.
The Problem Changes When AI Starts Doing Things
For the first wave of generative AI, much of the security conversation understandably focused on inputs and outputs. What information can the model receive? What information can it produce? Can we prevent harmful responses? Can we keep sensitive information from appearing in the wrong place? Those are still important questions, but agentic systems introduce another dimension because an agent doesn’t just answer a question. It can do work.
An agent might query a security platform, retrieve sensitive information, communicate with another agent, invoke a tool, create a ticket, change a configuration, disable an account or initiate a workflow involving several other systems. Once that happens, the important question is no longer simply whether the model produced an acceptable response. We need to know whether this particular agent is allowed to perform this particular action, against this particular resource, under these particular circumstances, at this particular moment.
That is a very different security problem, and it requires more than guardrails.
A Rule Is Not the Same Thing as Enforcement
Back to teaching my daughter to drive for a moment. Eventually, I might tell her she can use the car but needs to be home by a certain time. That’s a perfectly reasonable rule, but real life immediately makes it more complicated. What if there is an emergency? What if she’s picking up one of her siblings? What if the weather has become dangerous? What if the destination changes? What if someone else wants to drive? The rule still matters, but the context determines how it should be applied.
The response doesn’t necessarily have to be binary either. I might allow something, deny it, allow it under different conditions, change what is permitted, or require her to check with me before proceeding. This is much closer to how we think about policy enforcement in an agentic runtime. A policy establishes the organization’s intent, while the runtime evaluates that policy against the context of an actual action and then enforces the resulting decision.
This is the distinction I think matters: guardrails establish boundaries, while runtime policy enforcement governs execution.
The Runtime Has to Be in the Middle
There is another important part of this distinction. If the only place a policy exists is inside an agent’s prompt, then in many ways we are asking the agent to police itself. We can tell an agent not to access certain information, not to invoke a particular tool under certain circumstances, or that some actions require human approval. But ultimately the agent is still interpreting those instructions while deciding what it should do.
For enterprise systems, I don’t think that is enough. Policy enforcement needs to exist independently of the agent being governed. When an agent attempts an action, the runtime should be capable of intercepting that action, understanding the relevant context, evaluating the applicable policies and determining what happens next. The result might be to allow the action, deny it, modify it, or introduce a human approval requirement before execution continues.
The important architectural point is that the agent isn’t the final authority on whether the policy applies. I wouldn’t give one of my kids the house rules and then make them the sole authority responsible for determining whether they followed them. Organizations shouldn’t have to do that with autonomous agents either.
This Gets Much Harder With Multiple Agents
Now imagine that there isn’t one agent, but dozens, hundreds or eventually thousands of them. They have different jobs and capabilities, interact with different systems, have access to different information and communicate with one another. Some may belong to your organization, while others may come from vendors, partners or customers. At that scale, governance becomes much more than preventing an individual model from producing an undesirable response.
This is another place where having four kids provides a surprisingly useful analogy. The rules in my house aren’t necessarily identical for every child. Age matters, responsibility matters, where they’re going matters, who they’re with matters and what they’re doing matters. Just as importantly, permission doesn’t automatically transfer. If one of my children has permission to do something, another child doesn’t suddenly inherit that permission simply because they ask their sibling to do it for them.
Agentic systems face a similar problem. Agent A may have access to a particular dataset while Agent B does not. Agent A communicating with Agent B shouldn’t automatically transfer Agent A’s privileges. An agent authorized to investigate an incident may not be authorized to remediate it, and an agent operating for one organization shouldn’t suddenly gain access to another organization’s information simply because both agents participate in the same workflow.
Identity, role, security labels, organizational boundaries, provenance, the requested action and the context surrounding that action all become inputs into the decision. This is why I believe governance has to become part of the runtime architecture itself rather than something we simply surround the model with.
Governance Should Enable Autonomy
There is a temptation when discussing AI governance to think primarily about restriction. I think that’s backwards. The purpose of governance isn’t to prevent agents from acting; it’s to create an environment where we can safely allow them to act more.
The same is true with my kids. The goal isn’t to create enough rules that they can never make a decision without me. The goal is to establish boundaries I trust enough that they can make more decisions on their own. As they demonstrate that they can operate responsibly within those boundaries, I can give them more independence without having to participate in every decision.
Agentic systems need the same thing. If every meaningful action requires a human to review it, we haven’t really created autonomy. We’ve created a faster way to generate work for humans. But the opposite extreme doesn’t work either. Giving an agent unrestricted access to enterprise systems simply because we’ve instructed it to behave responsibly isn’t a security architecture.
The interesting space is between those two extremes. Agents should have enough freedom to pursue goals, make decisions, use tools, collaborate with other agents and recover when something doesn’t work, while remaining inside boundaries the organization defines and, critically, can enforce. Runtime policy enforcement is what makes those two ideas compatible.
Trust Before Autonomy
I’ve written before about the idea of trust before autonomy, and I keep coming back to it because I think it captures one of the fundamental challenges of agentic AI. Everyone wants more capable agents. We want agents that can reason longer, use more tools, work together, make decisions and accomplish increasingly complex objectives without requiring constant human intervention. But every increase in autonomy increases the importance of trust.
That trust can’t come solely from telling the agent what we expect it to do. It has to come from architecture. We need to know who the agent is, what it is trying to do, what information and systems are involved, and what policies apply in that particular context. Most importantly, we need an enforcement mechanism outside the agent itself that can determine whether the action should proceed and enforce that decision.
Guardrails remain an important part of that architecture, but guardrails aren’t governance. As agents become more autonomous, I think that distinction is going to become increasingly important.
Being a father of four has taught me that independence doesn’t come from eliminating boundaries. It comes from establishing boundaries that are clear and trustworthy enough that you don’t have to intervene in every decision. Teaching my daughter to drive is giving me a very real reminder of that right now. Before long, I won’t be sitting in the passenger seat, and I will have to trust that the combination of what she’s learned, the rules we’ve established and the boundaries around that independence are enough.
That’s ultimately what we need for agentic AI as well: not agents without boundaries, but agents with enough freedom to accomplish their goals while operating inside boundaries organizations can actually enforce.


