Ai

Your AI agents are a security nightmare

Your AI agents are a security nightmare
Listen to me read this!

I’ve noticed something about the tech industry over the past couple of months. Through the use of agentic code editors, developers are building small, custom solutions to problems they’re having. Recently, there’s been a surge in developers writing their own content platforms, abandoning bigger platforms like Hashnode and Medium. While I think this is genuinely amazing, we’re creating a gap that we have to address sooner rather than later.

That gap, of course, is maintenance. Even though AI coding is genuinely good now, new development is only a fraction of the software lifecycle. What happens when you get your app to prod and it suddenly breaks?

Do you know the codebase like you did back when everything was hand coded? Would you even know where to begin troubleshooting if your API started returning a bunch of 500s?

Let’s assume the answer is yes to both of those. Now let me ask the hard-hitting one: do you have time to troubleshoot and fix issues? This is a side project, not your day job. Many of us don’t have the luxury of dropping the “real work” tasks for a few hours while we figure out what’s going on.

I’m no exception. I’ve been building Good Roots Network, an app that connects local gardeners with people in their communities who need food. Growers list produce, gatherers like social workers and non-profits request it, and the system handles the coordination from availability through pickup. It’s API-driven from the ground up. State machines, role-based access, real-time inventory transitions. Real production infrastructure running in my personal AWS account.

There’s a lot of surface area to cover for a solo project. And when something goes wrong, I don’t have the bandwidth to dig in and figure out how to get it back on its feet - which means I’m potentially blocking people from getting access to fresh produce for themselves or others.

So I built an agent to be on call.

Before I continue, thank you to Teleport for sponsoring this post. All opinions are my own.

Building an agent that runs locally

The oncall-agent is a local, persistent process that automatically responds to production incidents. It subscribes to a Momento topic for real-time notifications and when something fires, it gets to work on its own. No need to wait for me to react.

The agent has an internal state machine: receive the incident signal, authenticate against AWS, investigate, then post a summary to Slack. During investigation, the agent loop runs on Amazon Bedrock, using the AWS CLI to query CloudWatch logs and metrics, inspect Lambda functions and DynamoDB tables. It also has access to my source code and deployments via a GitHub app to diagnose root cause and open a PR with the recommended code change.

I run it locally (rather than as a deployed service) for a couple of reasons. First, I specifically want it to make code changes if they’re needed, and pulling the source into a Lambda function in my account doesn’t seem like a great use of resources. Second, and more importantly, it touches production infrastructure directly. I want full control over the process, and I want the blast radius of any security issue limited to my machine, not a cloud-hosted process that’s running 24/7.

By the time I check Slack in the morning, the investigation and proposed solution is done and awaiting my approval.

Keeping the agent secure for a change

It’s tempting to just open up the doors on your machine and give the agent everything it needs to be successful (and then some). But as we’ve seen throughout 2026, doing so is a security nightmare. I don’t want to be another statistic who let an agent wreak havoc on my system because it was easy.

When it came time to give the agent AWS access, my first instinct was to put an access key in the .env file. Long-lived credentials. It’s the familiar pattern, it takes 30 seconds, and it works. But the more I sat with that approach, the more it felt like I was solving the wrong problem.

My issue with that approach was that the agent didn’t really exist as an identity in my system. It was just a privileged background process acting on my behalf with standing access.

So, I reconsidered the agent as a first-class principal. If it was going to investigate production issues autonomously, it needed to prove who it was at runtime, operate within a scoped session, and leave behind a traceable chain of actions.

Once I started thinking about the design this way, long-lived keys stopped making sense entirely. 👎

Identity is not the same thing as access

With this in mind, I needed something more than a vault. I needed a way for the agent to establish its identity at runtime.

That’s where Teleport fits into the design. Instead of access tokens being defined in a config file, the agent starts every investigation by authenticating into a new session. From there, it can request scoped AWS access and perform specific actions with clear attribution.

What I can now see in CloudTrail is a stable GUID that uniquely identifies the agent as its own principal. I can filter to that identity and see everything the agent has ever touched.

![Amazon CloudTrail logs showing a unique identifier for actions taken][https://assets.readysetcloud.io/teleport_1.jpg]

If something unexpected shows up, I know immediately it was the agent, and not a human or another process. That’s already a huge improvement over a shared access key where attribution is basically impossible.

Now, I have a legitimate session-bound identity operating on my system. Something that only has the permissions I gave it, but more importantly, is independently auditable.

When the agent runs tsh login, I set it up so I have to approve the MFA challenge myself. Fully autonomous credential rotation via Teleport Machine ID is available–and it’s the natural next step–but I’m not there yet. I want a few more practice reps with the agent before I remove myself from the auth flow entirely. Trust has to be earned, and I’d rather manually approve an MFA prompt than wake up to an agent that went sideways with no human checkpoint anywhere in the chain.

Agent security posture is something I’m really honing in on. I’m changing my mental approach from “does this agent have the right key?” to “was this agent authorized to do this specific thing, in this specific context, at this specific moment?” It might feel like a subtle difference, but it’s how we need to be thinking before we give agents the keys to our production systems. And a static credential sitting in an env file isn’t the way to answer it.

This isn’t a side project problem

The most common pattern I see these days is the same one I almost fell into: drop a long-lived token into the environment, grant it the permissions it needs, and ship it. Even if you do it as a “temporary stop gap” and swear you’ll come back to fix it, you won’t.

It always feels like it’s fine until something unexpected happens. The problem with unexpected behavior in autonomous systems is that you need more audit resolution, not less. Imagine an agent made unusual calls to your production services. Was it a bug? A prompt injection? A legitimate investigation path you just didn’t anticipate? With static credentials you can’t tell. The trail goes cold at “this IAM principal made these calls.”

Agentic AI is exposing some serious issues with the access model many of us are running. IAM was originally designed around humans doing episodic work. Even automated pipelines are episodic in that they run, finish, and stop. But that doesn’t map cleanly to processes that persist indefinitely, reason autonomously, and act faster than the blink of an eye.

When I think about what agents need to operate safely, I land on a few things. They need a cryptographic identity of their own, meaning something that can be uniquely identified (much like an individual developer). They need access that’s scoped and enforced at runtime–not granted once like an API key, and left open. They need some way to detect when they’re behaving outside expected parameters, or when the operating context has been tampered with. And if they’re coordinating with other agents or MCP servers, those connections need to be governed, too (you can’t have a secured agent talking to an unsecured tool).

These are all things Teleport is building with their Agentic Identity Framework. My agent only scratches the surface of all this with its short-lived credentials and isolated identity. But starting there forces you to think about the agent as a principal that needs its own security model, which is the right mental shift we need heading into mid-2026.

This probably sounds a little familiar

We’ve all seen this play out before in different forms. When containerization became mainstream, teams that had good practices with VM security often skipped the basics on containers because it felt like a “smaller” problem. When Lambda came out, teams that carefully managed server permissions dropped service roles for wildcard policies because it was a new execution model and they were moving fast (hopefully I’m not the only one who did this 😬).

Agentic AI is the same thing happening again. A new execution model, a lot of excitement, and a temptation to skip the identity and access fundamentals–because it feels like a side project, a POC, or just something you’re trying out.

The lesson is the same every time: the important components in production are still important even if the thing running is new, or small, or experimental. An agent with standing access and no attribution is a liability, whether it’s running inside a Fortune 500 or on your laptop handling incidents for a community gardening app.

The oncall-agent repo is open source. The Teleport integration is in the state machine if you want to see how it fits together in practice. And if you’re experimenting with agents at all, it’s worth triple-checking how they establish identity and access production systems. Move a small workflow to session-bound credentials to help you make better architectural decisions and make your automations easier to trust.

Tools like Teleport make that change practical without completely reworking your stack. It’s a small change in implementation, but a big change in how much autonomy you’re willing to give your systems.

Happy coding!

Related posts

Allen Helton

About Allen

Allen is an AWS Serverless Hero passionate about educating others about the cloud, serverless, and APIs. He is the host of the Ready, Set, Cloud podcast and creator of this website. More about Allen.

Share on:

Join the Ready, Set, Cloud Picks of the Week

Weekly writing and curated picks on cloud-native systems and practical AI. Browse past issues to see if it’s for you.
Browse past issues.

Join the Ready, Set, Cloud Picks of the Week

Thank you for subscribing! Check your inbox to confirm.
View past issues.  |  Read the latest posts.