Not all automated traffic is the same
Automated clients reach your application every minute: search crawlers, monitoring tools, partner integrations, AI assistants fetching pages for a user, and autonomous agents completing tasks. For years, the only way most applications could tell them apart was the user-agent header, a string any client can set to anything.
That leaves application teams in a bind. Treat all automation with suspicion and you degrade the experience of legitimate agents your customers depend on. Trust the header and you invite anyone to claim to be a well-known crawler.
Verified automated requests change the equation. When an automated client signs its requests with a key it has published, your application can check the signature and know, cryptographically, which client signed the request. Proof of Actor makes that check part of a single, consistent actor result your code can route on. For a closer look at what that check does and does not establish, read what signed AI agent requests prove.
How request signing works
The emerging approach for automated traffic uses HTTP message signatures, standardized in RFC 9421, which defines how to create, encode and verify signatures over components of an HTTP message. The client signs selected parts of each request with a private key and includes the signature and its parameters in headers. It publishes the corresponding public key at a location tied to an identifier it controls, typically its domain.
When the request arrives, the receiver fetches or looks up the published key, reconstructs the signed content, and checks the signature. If it validates, the request was signed by the holder of that key. Work on signing automated traffic continues in the IETF Web Bot Auth working group, whose charter covers cryptographically authenticating automated clients such as crawlers and AI agents acting for end users; the Web Bot Auth glossary entry summarizes it. The blog post on what signed AI agent requests prove walks through a verified request end to end, and the APIs use case shows where verification pays off first.
The guide on verifying signed agent requests covers the mechanics in detail, including timestamps, replay windows and key caching.
What a verified signature proves, and what it does not
This distinction is the most important part of handling verified automated requests well.
A valid signature proves that the request was signed with a key associated with the published identifier. If the key belongs to an operator's domain, the request came from a client holding that operator's key.
A valid signature does not prove:
- who instructed the agent, or whether any person asked for this action;
- that a person delegated authority to the agent;
- the real-world legal identity behind the operator;
- that the requested action is safe or permitted in your application.
Those are separate questions. Your application answers them with its own credentials, authorization model and policy. Proof of Actor keeps them apart deliberately, and the page on Detection & Limitations spells out exactly what each kind of evidence can establish.
How Proof of Actor handles verified automated requests
Detect
Your integration passes the request context, including any signature headers, to Proof of Actor. Signatures are checked against published keys. Requests that validate are recorded with a verified basis. Clients that only name themselves are recorded as declared. Behavioural patterns are recorded as inferred. When there is not enough to go on, the result is Unknown. Nothing is promoted to verified unless a cryptographic check passed.
Understand
See which verified operators call your application, which routes they use, how often, and how that compares with declared and unknown automation. The evidence summary on each result tells you which key and identifier were involved, so your team can investigate any request quickly.
Route
Use the basis in your policies. Common patterns include:
- Give verified agents a structured response on read-only routes, such as an API or a Markdown version of a page.
- Raise rate limits for verified operators you recognise while keeping standard limits for declared clients.
- Keep your existing authorization on every write route, whatever the signature says.
- Define an explicit fallback for unsigned and Unknown callers on each route.
Your policies decide; Proof of Actor supplies the evidence.
Why the user-agent header is not enough
User-agent strings are declarations. Any client can send any value, and user-agent spoofing takes no skill. That is why crawler operators publish their own checks: Google documents reverse and forward DNS lookups and published IP ranges for verifying Googlebot, Bing documents a DNS check for verifying Bingbot, and OpenAI publishes IP range files for each of its crawlers and user agents. Proof of Actor treats a user-agent claim as declared evidence, never as verification. That means a client claiming to be a well-known crawler without a valid signature is reported with a declared or inferred basis, and your policy can treat it differently from a client that proved its identity.
This is what makes verified automated requests valuable: they give you a reliable way to extend trust to the clients that earn it, without extending it to everyone who copies their header.
Identity evidence, credentials and authorization
Three layers work together in any application that serves automated clients:
- Identity evidence: which client signed this request? A verified signature answers this.
- Credentials: which account is responsible? Your API key, token or session answers this.
- Authorization: may this caller do this? Your permission model answers this.
Proof of Actor strengthens the first layer and makes it visible. It does not issue identity or delegation credentials, and it does not replace your authorization. Keeping the layers separate is what lets you be generous with verified agents on low-risk routes and careful on high-impact ones.
Where it fits
Proof of Actor sits alongside your gateway, authentication and rate limiting. Integrations run as server-side middleware, at the edge, in your MCP server or through a generic HTTP integration, delivered hands-on with each founding team. If your traffic is mostly APIs and tools, the APIs and MCP Servers use cases show the patterns that matter most.
Start in observation mode
Every integration starts in observation mode. Results are recorded without changing how your application responds, so you can see how much of your automated traffic is verified, which operators are signing, and where the unknown rate is highest. Policies come later, route by route, when the evidence supports them. How It Works walks through the full flow.
Practical details that make verification reliable
Signature checking is only as good as the details around it. A few practices make verified automated requests dependable in production.
Check freshness
RFC 9421 defines created and expires signature parameters, and signed requests usually carry them. Reject signatures outside a short window, so a captured request cannot be replayed later.
Cache keys sensibly
Fetching an operator's published keys on every request adds latency. Cache them for a reasonable period and refresh when a signature fails with an unknown key identifier, so key rotation does not cause false failures.
Sign the right parts of the request
A signature that covers only a timestamp proves very little. Check that the signed components include the method, the path and the host, so the signature binds to the request your application actually received.
Record the evidence
Keep the key identifier, the published identifier and the result with each verified request. When something unusual happens, your team can see exactly which client signed what and when.
Proof of Actor handles these checks as part of the Detect step and reports the outcome as a basis your policies can read, so your application code stays simple.
Join the founding developer program
Proof of Actor is in private beta. Selected founding development teams can receive up to 12 months of free access, within agreed usage limits, with direct integration support from our engineers.
If automated clients reach your application and you want to treat the verified ones differently, apply for early access. Tell us about your application and the automated callers you see, and we will show you how Proof of Actor fits.
Sources
- RFC 9421: HTTP Message Signatures, IETF, February 2024.
- Web Bot Auth (webbotauth) working group charter, IETF.
- Verifying Googlebot and other Google crawlers, Google Search Central.
- How to verify that Bingbot is Bingbot, Bing Webmaster Blog.
- Overview of OpenAI crawlers, OpenAI.