Agent Policy

An agent policy is the behavior an AI agent follows when mapping observations to actions. Given the current state, instructions, available tools, and prior history, the policy determines what the agent is likely to do next.

In reinforcement learning, a policy is often a learned function that selects actions from observations. In LLM-based agents, the policy is distributed across several layers:

• model weights learned during training and post-training;
• agent scaffolding;
• context engineering;
• tool availability and tool descriptions;
• memory retrieval;
• agent guardrails; and
• runtime choices made by the agent harness.

An agent policy is not the same as the agent itself. The agent is the full system that acts in an environment. The policy is the behavior that system exhibits under particular conditions.

This distinction matters when diagnosing failures. If an agent repeatedly calls the wrong tool, the cause may be the model, the tool description, the retrieved context, the orchestration logic, or the reward signal used during training. Treating all failures as "model quality" hides the real engineering surface.

Policies can be evaluated at different levels. A final-answer evaluation measures outcome quality. Agent evaluation also measures action choices, tool use, recovery behavior, cost, and safety. Training systems may use a reward function to update the learned part of the policy.

Agent policies should be bounded by externally enforced permissions. A policy can express a preference for safe behavior, but high-impact operations require authorization, sandboxing, and deterministic checks.

Hugging Face's agent glossary gives a practical framing of policy as the behavior followed by an agent rather than the full deployed agent system.