AI Agents Need More Than Authorization

While prototyping a support agent last week (LangGraph, calling out to a refunds endpoint), we ran into something that wasn't really a bug. The agent was, technically, working exactly as designed. Valid API key, scoped correctly, hit the right route, passed the right schema. And it issued a refund that should never have gone through — not because of a prompt injection or anything exotic, just a normal conversation that ended somewhere it shouldn't have.

We went looking for where this should've been caught, and there's nowhere it would've been. Authentication had already answered who is this — a valid service, a valid key. Authorization had already answered is this allowed — yes, this key can call /refunds. Neither layer has an opinion on whether this specific refund should happen, given everything else going on in that conversation. Nothing in the stack does, really, except the system prompt, and a system prompt is not a control. It's a suggestion the model is statistically likely to follow.

why "just validate the input better" doesn't cover it

Normal API misuse is bounded — someone's hitting your endpoint with bad params, you write a validator, done. The set of bad inputs is finite-ish and you can enumerate a lot of it.

With an agent the sequence of calls is generated at runtime by a model. There isn't really a fixed "bad input" to check for, because the input that's wrong isn't malformed — it's a perfectly well-formed request that just shouldn't have been made, given everything else going on in that conversation. A $4,000 refund on a $40 order is a valid POST /refunds body. There's no schema that rejects it.

The same shape shows up anywhere an agent has a real action available to it:

a well-formed DELETE against a production table
a well-formed email blast to every customer
a well-formed payroll approval
a well-formed AWS key rotation

None of those requests are malformed. The problem was never in the shape of the data.

So it's not really an input validation problem. It's closer to "the system worked exactly as built and still did the wrong thing," which is a different category of bug than the ones most tooling is built to catch.

what we kept trying and abandoning

Our first instinct was a config file — refund_limits.json or whatever, checked by a middleware function before the call goes out. Works fine until someone asks "can support change this threshold without a deploy," and the answer is no, because it's a file in the repo.

Second attempt: move it to a DB row, check it at request time. Fine, except now every action has a synchronous DB round trip in front of it, and once you add a couple of these you've added real latency to something that used to be instant.

Third attempt, and this is where we landed: the limits stop being if amount > 50 and start needing actual logic — different threshold for this customer segment, different rule during this one incident window, different behavior if the data touches PII or not. At which point we were clearly building a small rules engine — badly, half-finished, inside a file called middleware.ts — instead of just admitting that's what was needed and doing it properly.

Haven't found a write-up of anyone doing this cleanly. The closest things we've found are OPA and Cedar, both of which are real, mature policy engines. They're powerful, but most application teams we've talked to don't want every product decision — a refund threshold, a support-tier exception — to require writing another policy language. That's less a knock on OPA or Cedar and more a mismatch: they're built for infra and platform teams making access-control decisions, not for a support lead adjusting a threshold on a Tuesday.

We kept coming back to the same question underneath all of this: before an agent takes an action that can't be undone, who decides whether it should happen — not whether it's allowed to. Whether it should. Authentication and authorization both stop one step short of that question. We think that's a missing primitive in most AI infrastructure right now.

so, we're asking around

If you've got an agent touching anything that matters in prod — genuinely curious how you're handling this part:

is there an actual check before the action fires, or is it living in the prompt and you're hoping
if there's a real check, is it a hardcoded if-statement, a config file, a DB lookup, something else
who's allowed to change the limits — does marketing/support/compliance have any way to do that without opening a PR

Not looking for "you should use X framework" answers, more interested in what's actually running in production right now, duct tape and all.