Dr. Ryan Ries here.
People want agents to do real work. Yet almost nobody has fully thought through what happens when an agent gets access it shouldn't have.
Before I dive in – we've got a few really cool upcoming events you need to check out.
- I’m leading a workshop about prioritizing AI initiatives. Register here.
- We’re hosting upcoming AI Roadmapping (AIM) workshops specifically for healthcare and financial services companies. This is NOT a technical session and actually requires 2-5 folks from your company to attend. It’s a small, intimate session where you leave with a roadmap to accomplish your AI initiatives. Check them out here:
Treat Agent Access Like You Treat Employee Access
A few weeks ago, Jonathan LaCour and I hosted one of our usual Agentic AI Unscripted sessions, where the audience can ask any questions around agentic AI, and Jonathan and I will give our honest, raw, uncensored opinions and reactions.
A pretty fun way to spend an hour while learning, if I do say so myself.
A viewer named Mia asked the question that applies to every organization implementing agentic AI.
Organizations get excited about agentic AI, then hesitate because of data security concerns. What actually builds trust and moves adoption forward?
My answer starts with the boring part. Agent IAM roles. Least privilege access. If you are running a multi-tenant system, one customer's agent should never be able to reach into another customer's data. That is not a “nice to have.” That is the entire model falling apart if you skip it.
Our own team just hit this problem head on. We recently shipped an agent for a life sciences analytics platform that lets users query permission-controlled data in plain language
The most challenging part of the engagement was making sure the agent could act as one specific user and nobody else. The team wrote up how they solved it, using what they're calling a Dual-Token Pattern, one token to get the agent in the door and a second, user-scoped token for every action it takes downstream. Worth a read if you're building anything similar: Amazon Bedrock AgentCore Identity: How to Handle OAuth in AI Agents.
But I want to call out something in particular.
An agent given broad, unscoped access will find the edges of that access, whether you meant for it to or not.
For example, I brought up the OpenClaw example. I read a story about how someone never expected their agent to reach into their bank account, and it did, because the agent had been handed the keys to everything on the machine, passwords included. Give a system permission to do anything, and eventually it will try something that you really did not want it to do.
Another example:
Chipotle's AI assistant on their website was running a model that also happened to write code. Customers figured out they could skip ordering a burrito and instead ask it to write a Python function. It complied. Nobody had told the assistant that writing code sat outside its job description, because nobody had scoped what it was allowed to do in the first place.
Multi-tenant isolation, scoped permissions, and an explicit allow list of what an agent can touch are crucial for running agents at any real scale.
The Dumbest Smart Guy You Know
Jonathan threw us back to his college days and had the perfect analogy for these agents.
He had a friend who was brilliant, aced every class, and still made some of the worst decisions any of them had ever seen. The group called him the dumbest smart guy they knew.
I’m sure most of us had a “dumbest smart guy” friend from college.
These agents are very sharp, occasionally reckless, and fully capable of getting you into a situation you did not see coming.
IT organizations have already built the muscle for managing risk across a workforce that occasionally makes mistakes. A 20,000 person company expects a certain percentage of employees to slip up somewhere. That math does not disappear because you swapped a person for an agent. If anything, the math changes shape, because one misconfigured agent can replicate a mistake at a speed and scale no single employee ever could. Scary!
What the Frontier Labs Are Saying
You might be wondering what the frontier labs have to say about all of this.
Anthropic's newer Fable 5 and Mythos models came up in the agentic AI unscripted session because of how differently they are being handled. Fable 5 retains user inputs, sometimes for 30 days, and for other material, up to two years. That is worth knowing before anyone on your team starts feeding it sensitive documents. Mythos goes further. Access is restricted, not sold openly, specifically because of the security risk the model poses if it lands in the wrong hands. A vendor telling the market a product is too capable to sell the normal way is not something we have seen much of before now.
Around the same time, Anthropic's leadership was on record talking about when governments should step in to evaluate models before they get released to the public. For years, the instinct across most labs was to resist that conversation. Hearing a lab openly discuss external oversight, while acknowledging that the financial incentive to move fast will always be in tension with getting it right, is a notable change in tone.
This connects directly to something else from the session. Self-improving systems, where a model gets good enough to meaningfully improve itself, sound powerful right up until you ask what guardrails and reward functions are steering that improvement. Anyone who remembers Microsoft's chatbot experiment from years back knows how fast an unsupervised learning loop can go sideways. It took the internet about an hour to turn that bot into something no one wanted representing their brand. The tools have improved a great deal since then. The underlying lesson has not changed. Self-improvement without a defined boundary is a risk you don't want to take.
This week I was at the AMD Advancing AI conference and the keynote by Dr. Lisa Su had a lot of great insight into how people are thinking about AI and training models. One Interesting tidbit that the Head of Compute at Anthropic said was how great the new AMD Helios platform was and how easy it was to use. They had one engineer working with Claude set up the cluster and then Claude started to figure out how to optimize itself on the new hardware. I am not sure about you all, but this is a bit scary to think about and reminds me of several horror movies we have all seen.
My Thoughts
None of this should talk anyone out of adopting agentic AI. It should talk you into doing it with your eyes open and with a partner by your side.
Before your team hands an agent any meaningful access, I would ask three questions.
What is the smallest set of permissions this agent needs to do its job, and have you actually scoped it down to that?
What happens the day it tries to do something outside that scope, and will you know?
Who inside your organization is responsible for reviewing what the agent produces before it reaches a decision maker, because the agent will not flag its own mistakes for you?
Responsible adoption is not a checkbox activity. It’s actually more like onboarding a new employee who happens to be capable of superhuman output and zero judgment about when to stop.
If you’re interested in building these systems for your organization with a partner who has built them for hundreds, we’re here for you! Reach out to our team here.
Until next time,
Ryan
Now, time for this week’s AI-generated image and the prompt I used to create it.
Generate an image of a muppet handing a giant ring of keys to a robot, who is already halfway through a door labeled 'DO NOT ENTER.' The muppet has a worried expression. Make the setting an office with a bank vault visible in the background.
