How an automated client proves who it is

01

What a client name establishes

Applies to marketplace onlyown storefronthybrid

An automated client identifies itself with a header it selects. Operators publish the names they use: OpenAI’s documentation lists its crawlers, the purpose stated for each, and the IP ranges they originate from.⁵ That publication tells a site owner which strings are intended to be that operator’s, and it does not prevent any other party from sending the same string.

The check applied on top has historically been address-based. The operator publishes ranges, the site verifies the request came from one, and the list is fetched, cached, and expired. The cost of that approach scales with the number of operators, and it addresses clients running from a crawler fleet rather than from a buyer’s own infrastructure.

02

What a signature covers

Applies to marketplace onlyown storefronthybrid

RFC 9421 is a published Standards Track document specifying how a client produces a cryptographic signature over selected components of an HTTP message and how a server verifies it.¹ The word selected is load-bearing: the signer chooses which components are covered — method, path, particular headers — and verification applies to those and not to the rest.

The specification establishes how a request can be signed and verified rather than who is entitled to access anything.¹ A verified signature is therefore an input to a policy, and the policy is written separately.

03

Web Bot Auth and its current status

Applies to marketplace onlyown storefronthybrid

Web Bot Auth is the work applying message signatures to automated clients specifically. The current document covers signing and verifying automated client identity, key discovery, and deployment models, which is the material between a signature format existing and a site being able to act on one.²

Its status is stated in the document. It is an active Internet-Draft, which is work in progress and is cited only as such. It is an individual submission rather than working-group output, it replaced an earlier architecture draft, and the current revision carries an expiry date in December 2026.² A description of it as a standard is not accurate as of this lesson’s review date.

04

Verification as it is deployed today

Applies to marketplace onlyown storefronthybrid

Verification is running in production ahead of a finished standard. Cloudflare’s verified-bot criteria admit two routes, one through Web Bot Auth and one through IP validation.³ A storefront behind that infrastructure has this setting whether or not anyone in the business has looked at it.

Two limits apply to how far it carries. It describes one provider’s list and criteria, so verification there is verification by that provider. And the same documentation states that inclusion on the list creates no requirement for a site owner to serve the client.³ The identity result and the access decision remain separate records.

The decision side is a separate set of controls: per-crawler allow and block rules, visibility into crawler activity, and a pay-per-crawl path.⁴ The last of these is described as a private beta, which places it among things that exist rather than among things a seller can currently plan around.

05

Writing the access posture

Applies to marketplace onlyown storefronthybrid

A reseller storefront consumes these mechanisms rather than implementing them. The available work is writing the policy in advance, so that it exists before an incident rather than being decided during one.

1. Write the three questions separately: which client is this, may it fetch this path, and at what rate.
2. State what an unverified client receives, which for a public product page is ordinarily the page.
3. State what no client receives regardless of verification: checkout endpoints, account pages, bulk export paths.
4. Check whether the host or CDN already exposes verified-bot handling, and record its current setting.
5. Record the one condition that would change the policy, so the review has a stated trigger.

Three rows confirmed what the seller already believed and took a few minutes each. The fourth did not resolve, and it is the row that records something new: there is a path on this site whose handling of automated traffic nobody in the business can currently state.

A posture written this way does not depend on the identity question being settled. It holds if signatures become common, and it holds if the draft expires and is replaced, because each row states a decision about a path rather than a decision about a mechanism.

06

Checking a claim in this area

Applies to marketplace onlyown storefronthybrid

The vocabulary in this area is ahead of the deployment, and three questions can be checked against the primary documents.

• Which document is being implemented, and what is its current status: published, active draft, or replaced?
• Does the verification being offered establish identity only, or is a permission also being asserted?
• What happens by default when a client fails verification, and is that setting visible to the site owner?

The first question has a documented answer. RFC 9421 is published and Standards Track, and Web Bot Auth is an active individual-submission draft, so a claim describing the second as a standard can be checked against the draft’s own status.

07

The identity documents and their status

Applies to marketplace onlyown storefronthybrid

RFC 9421 specifies signing and verifying selected components of an HTTP message and does not address entitlement.¹ Web Bot Auth applies that to automated clients and is an active Internet-Draft, an individual submission with an expiry date in December 2026.² OpenAI publishes its client names, their purposes, and their address ranges.⁵ Cloudflare publishes verification criteria with two routes, and states that inclusion imposes no requirement on a site owner.³ Its separate controls cover per-crawler rules, crawler-activity visibility, and a pay-per-crawl path in private beta.⁴

Between them these establish what a signature covers, what the draft currently is, and what one provider verifies. None states which clients a site owner should serve, which is the entry the seller’s own posture fills.