How to roll out actor-aware policies safely

What can go wrong when you roll out actor-aware policies?

An actor-aware policy changes how your application responds based on who or what is acting. Done well, it gives people a smooth experience, gives legitimate agents a structured path, and adds friction only where it matters. Done badly, it locks out real users, breaks integrations your customers depend on, and creates support tickets nobody can explain.

The difference is rarely the policy logic. It is the rollout. (For the policy model itself, see designing app-level policies for humans and AI agents.) This guide describes a staged process that teams can follow for any route: observe, preview, enforce for a slice, then expand. Each stage has a clear exit criterion and a fast way back.

The three modes

Think of every policy as running in one of three modes per route:

• Observe. Classifications are recorded. The application responds exactly as it would without them.
• Preview. The policy is evaluated and its decision is recorded, but not applied. You can see what would have happened.
• Enforce. The decision is applied to real requests.

The mode belongs to the route and policy pair, not to the whole application. Your documentation pages can be enforcing while your checkout is still observing.

Step 1: Observe until you know your baseline

Start with every route in observe mode. Collect, at minimum:

• the route and method,
• the actor class (Human, Human + AI, Agent or Unknown) and its basis (verified, declared, inferred or unknown), as defined in our taxonomy for application traffic,
• the response status and a simple outcome marker for the route, such as "account created" or "search returned results".

Run long enough to cover normal weekly cycles and any release you ship in that period. Then write down your baseline for each route you care about: the class mix, the Unknown rate, and the outcome rate per class. You will compare everything that follows against these numbers.

Exit criterion: you can describe, for each candidate route, who uses it and how outcomes differ by class.

Step 2: Write the policy as data

Policies are easier to review, test and roll back when they are data rather than scattered if statements.

type ActorClass = "human" | "human_ai" | "agent" | "unknown";
type Basis = "verified" | "declared" | "inferred" | "unknown";
type Action = "allow" | "adapt" | "challenge" | "review" | "limit";
type Mode = "observe" | "preview" | "enforce";

interface Rule {
  actor: ActorClass | "*";
  basis?: Basis[];
  action: Action;
}

interface RoutePolicy {
  route: string;
  mode: Mode;
  enforcePercent: number;
  rules: Rule[];
  fallback: Action;
}

export const policies: RoutePolicy[] = [
  {
    route: "GET /api/catalog",
    mode: "preview",
    enforcePercent: 0,
    rules: [
      { actor: "agent", basis: ["verified"], action: "allow" },
      { actor: "agent", basis: ["declared", "inferred"], action: "limit" },
    ],
    fallback: "allow",
  },
];

Every policy has an explicit fallback. That is the action used for Unknown results, for timeouts and for any case no rule covers. Choosing it deliberately is the single most important safety decision in the whole process.

Step 3: Evaluate in one place

A single evaluation function keeps behavior consistent across modes.

export function decide(policy: RoutePolicy, actor: ActorClass, basis: Basis): Action {
  for (const rule of policy.rules) {
    const actorMatch = rule.actor === "*" || rule.actor === actor;
    const basisMatch = !rule.basis || rule.basis.includes(basis);
    if (actorMatch && basisMatch) return rule.action;
  }
  return policy.fallback;
}

Then apply the mode:

export function applyPolicy(policy: RoutePolicy, actor: ActorClass, basis: Basis, requestKey: string) {
  const decision = decide(policy, actor, basis);
  const inSlice = bucket(requestKey) < policy.enforcePercent;
  const enforced = policy.mode === "enforce" && inSlice;
  record({ route: policy.route, mode: policy.mode, actor, basis, decision, enforced });
  return enforced ? decision : "allow";
}

function bucket(key: string): number {
  let h = 0;
  for (const c of key) h = (h * 31 + c.charCodeAt(0)) >>> 0;
  return h % 100;
}

The stable bucket means the same session or client stays in or out of the enforced slice, so people do not see behavior flip between requests.

Step 4: How do you measure false positives before enforcing?

Move one route to preview. Now every request records the decision it would have received. This is where you learn whether the policy is right.

A false positive is a request that the policy would restrict but that should have been allowed. You cannot measure it from classifications alone, because the classification is what you are testing. Measure it against independent signals of legitimate use:

• the request completed a normal successful flow, such as a signed-in user saving work,
• the client is a known partner integration or a verified operator you have agreed to serve,
• support tickets or user reports reference the affected route,
• a manual review of a sample of would-be restricted requests.

Review a sample by hand. Reading twenty real requests that the policy would have challenged is the fastest way to find a rule that is too broad.

Exit criterion: the would-be restricted set is dominated by traffic you genuinely intend to restrict, and you have an explanation for every category of false positive you found.

Step 5: Enforce for a slice

Switch the route to enforce with a small enforcePercent. Keep the slice small enough that a mistake affects few requests and large enough to produce a measurable signal within a day or two.

Watch the same metrics you set as your baseline:

• outcome rate per class on the route,
• error and challenge completion rates,
• support volume mentioning the route,
• latency, since evaluation sits on the request path.

If outcomes for Human traffic drop in the enforced slice compared with the rest, stop and investigate before expanding.

Step 6: Expand gradually

Increase the slice in steps, for example from a small share to a quarter, then half, then all traffic, holding at each step long enough to see a normal cycle. Expansion is boring when the earlier stages were done well. That is the goal.

Choosing fallbacks and failure behavior

Decide in advance what happens when the evidence is missing or the evaluation cannot run.

• Fail open means the request proceeds as it would without actor context. Use it for low-risk routes such as public pages, documentation and read-only catalog endpoints.
• Fail closed means your existing stronger check applies, such as re-authentication or a review queue. Use it for high-impact actions where you already have such a check.

Never let a timeout or an error produce a Human result. Missing evidence is Unknown, and Unknown takes the route's fallback. Our post on why Unknown is a valid result when classifying AI traffic explains why this rule matters.

async function actorWithTimeout(evaluate: () => Promise<{ actor: ActorClass; basis: Basis }>) {
  const timeout = new Promise<{ actor: ActorClass; basis: Basis }>((resolve) =>
    setTimeout(() => resolve({ actor: "unknown", basis: "unknown" }), 150),
  );
  return Promise.race([evaluate(), timeout]);
}

How do you roll back a policy quickly?

Rollback should be a configuration change, not a deploy. Keep policies in a store your team can edit quickly, and make these actions instant:

1. Set a route back to preview, which stops enforcement while keeping measurement.
2. Set enforcePercent to zero.
3. Disable a single rule without touching the others.

Record every policy change with who made it and when. When outcomes move, the first question is always what changed.

Avoid consequential decisions about people

Actor classifications describe evidence about an interaction. They are not a judgment about a person. Do not use an inferred class as the sole basis for decisions that affect someone's employment, credit, education, housing or access to essential services. For those flows, use actor context to choose a process, for example routing to human review, rather than to decide an outcome.

Communicate before you enforce

Policies that affect automated clients also affect the people and companies who run them. A little communication prevents a lot of support work:

1. Publish your rules for automated access. State which routes are open, what limits apply and how clients can identify themselves with signed requests using HTTP Message Signatures (RFC 9421) and the IETF Web Bot Auth protocol draft.
2. Tell partners and integrators in advance. If a known integration falls into a restricted slice during preview, contact its owner before you enforce.
3. Return useful responses. When a policy limits or challenges a request, respond with a clear status code, such as 429 Too Many Requests with Retry-After, and a link to your documentation. A machine-readable body in the Problem Details format gives automated clients a structured reason. Well-built agents adapt when they understand why.
4. Give your support team the evidence. When a ticket arrives about a blocked request, support should be able to look up the route, the class, the basis and the policy decision without asking engineering.

Teams that announce changes and explain them see far fewer surprises than teams that switch rules on quietly, and the clients that adapt become easier to recognize over time.

A checklist for each route

• Baseline recorded for class mix, Unknown rate and outcomes.
• Policy written as data with an explicit fallback.
• Preview run, sample reviewed, false positive categories explained.
• Enforcement started on a small slice with metrics watched.
• Failure behavior chosen: fail open or fail closed.
• Rollback tested before expansion.

Where Proof of Actor fits

Proof of Actor is designed around this process. Observation is the default, every result carries its basis, and Route policies are yours to define and enforce in your own code, route by route. Detect, Understand, Route: in that order.

To work through a rollout on your own application with our team, apply for early access. For the decision flow end to end, read How Proof of Actor works.

Sources

• RFC 6585: Additional HTTP Status Codes, Section 4: the 429 Too Many Requests status code
• RFC 9110: HTTP Semantics, Retry-After: telling clients when to retry
• RFC 9457: Problem Details for HTTP APIs: machine-readable error bodies
• RFC 9421: HTTP Message Signatures: how automated clients sign requests
• draft-ietf-webbotauth-httpsig-protocol: IETF Web Bot Auth draft for signed automated traffic (work in progress as of November 2026)