Let information in. Keep authority explicit.
Useful tool results should inform the task without granting new permissions.
A tool result
A connector returns content from an outside source.
Application checks
Validate the data and enforce the user’s permissions.
A scoped action
Only an independently authorized request can execute.
Even a trusted connector can return attacker-controlled text. The tool’s identity and the trustworthiness of the content it retrieved are separate questions.
Use schemas for structure and explicit checks for authority. Keep credentials out of model-visible data and validate access, arguments, and destinations around each tool execution.
An agent’s proposed action is a request, not an approval. For consequential actions, show the exact effect to an authorized person and enforce the decision in the application.
An agent reads a support ticket through an official company tool. The ticket says the customer needs help. It also tells the agent to send all account notes to a new email address.
The tool is legitimate. The ticket text still came from someone outside the application’s control.
This distinction is easy to miss: trusting a tool to fetch a record does not mean trusting that record to issue commands. If you are new to the entry point, first read indirect prompt injection.
Give each component one clear job
Consider a fictional support assistant with three responsibilities: read a ticket, draft a reply, and offer the reply for sending. The customer is entitled to help with their own account. A ticket cannot expand that entitlement.
The model can suggest a response. Application code decides whether the current user may access the account and whether a particular message can be sent. A mail service performs the authorized operation. Keeping these responsibilities explicit gives you places to inspect and test.
OWASP’s Excessive Agency guidance identifies excessive functionality, permissions, and autonomy as distinct ways an LLM application can gain more power than its task requires.
1. Preserve where information came from
When the ticket arrives, keep its record ID, source system, owning account, retrieval time, and the identity used to access it. Keep the ticket body separate from application instructions.
In our example, that means the application can say “this address was written in ticket 42” instead of silently treating the address as the customer’s verified contact.
A provenance label is useful evidence about origin. It is not proof that a statement is true or an action is allowed. A retrieved document cannot promote its own trust level by containing a field such as approved: true.
If a summary or extracted value is stored for later, preserve that origin as well. Otherwise, an instruction copied into memory may lose the warning signs of the original source. The risks of unsafe memory and context handling are covered in OWASP’s AI Agent Security cheat sheet.
2. Offer small capabilities
An assistant that drafts support replies does not automatically need a general shell, unrestricted database queries, or a tool that sends messages to arbitrary addresses.
For this product, prefer separate capabilities:
| Capability | Allowed job | Boundary |
|---|---|---|
| Read a ticket | Retrieve an authorized ticket | Check account access for each request |
| Save a draft | Store proposed reply text | No external delivery |
| Send an approved reply | Deliver the reviewed draft | Check actor, recipient, content, and approval |
These names are illustrative; naming alone enforces nothing. The implementation and credentials must impose the limits. A read_ticket wrapper using an unrestricted backend token still needs correct authorization for every requested ticket.
3. Validate meaning as well as shape
Structured output makes it easier to parse a proposed action. It does not make the values trustworthy.
Suppose an extraction step returns a valid email address. A schema can confirm that the value looks like an address. It cannot establish that the address belongs to the customer, is an approved recipient, or may receive the requested information.
Use authoritative account data for those decisions. Resolve the ticket’s owner from the backend, check the logged-in user’s permissions, and compare the requested operation against application policy. Reject missing or inconsistent context.
This is the same principle used in ordinary web authorization: default to denial when permission is absent and check access on every request. OWASP’s Authorization cheat sheet explains why a check at one earlier step does not cover later requests automatically.
4. Make approval describe the actual action
For a message that leaves the system, show the reviewer the exact recipient, subject, body, and attachments. Store approval against that specific proposed action. If the agent changes the recipient or content afterward, the prior approval must not silently cover the new version.
In our support example, a button labeled “Send this reply to the verified account contact” communicates far more than “Allow tool use.” The backend still checks permission when the send request executes.
For MCP integrations, apply the same checks at the server boundary. The protocol does not turn tool output into policy. OWASP’s MCP security guidance also emphasizes narrow permissions and binding requests to the correct user context.
5. Use a separate reader carefully
One design sends untrusted text to a tool-free extraction model and passes selected fields to an action-planning model. This can reduce how much raw hostile text reaches the component proposing actions.
It is a reduction in exposure, not a guarantee. The reader can produce an incorrect but schema-valid value. A free-text summary can carry the unwanted instruction onward. Even a short field can choose the wrong account or recipient.
For our example, the reader might extract the issue category and suggested reply. It should not become the authority on who owns the ticket or which address is permitted. Those answers come from independently checked records.
Try the boundary with mock tools
Create a fictional ticket and a mock mail service that only records attempted sends. Keep the verified contact in a separate test account record.
Run three scenarios: a normal support request, a ticket asking to change the recipient, and a quoted discussion of that same request. Inspect the draft, proposed calls, authorization decisions, and mock outbox.
The expected result is useful drafting without unauthorized delivery. A blocked send attempt is different from no attempt; record both. Then change the recipient after an approval and check that the old approval no longer permits sending.
Check your understanding: if the ticket provides a perfectly valid JSON object saying the new address is approved, what is missing? An independent authorization decision from the application.
Keep the remaining risk visible
These controls depend on correct backend code, credentials, identity handling, and review design. A narrow tool can still contain a bug. A reviewer can still miss a misleading attachment. Monitor outcomes and keep regression tests for the boundaries that matter.
The pattern to retain is read, propose, check, then act. Keep evidence from tools available for reasoning while keeping permission decisions outside that evidence. Use the evaluation harness guide to make the checks repeatable.