TL;DR: Web Bot Auth is a standardized way for bots and AI agents to prove their identity using cryptographic signatures. It answers “who is this?” and deliberately leaves “should I let them in?” to the website. Most of the debate about Web Bot Auth comes from blending those two questions, and most of its future depends on how the second one gets answered.
Web Bot Auth is a relatively new term making a bit of a splash. And rightfully so! A new authentication standard for the internet? The death of the open web?
There are strong opinions surrounding this new verification proposition, but it takes more than a Hacker News docs link to understand the full context of what Web Bot Auth is, what it could enable, and where it falls short.
I think it helps to zoom out and understand what Web Bot Auth is before we argue about what it means.
So let’s start simple.
What even is Web Bot Auth?
Web Bot Auth is a standardized method for bots and AI agents to prove their identity using cryptographic signatures.
When you’re 15, you can get a driver’s permit, which lets you drive around but only with your parents in the car until you prove yourself. We’re at that permit stage with agent identity. Agents are borrowing the identity of the organization or individual they represent. Web Bot Auth allows agents to apply for their own license and use it to get past access providers like Cloudflare and reCAPTCHA, when merchants and website owners allow it.
How we landed on Web Bot Auth
All of the digital world’s work happens in a browser. While we build out agent-native infrastructure, browsers act as general-purpose infrastructure for agents to access applications, websites, and businesses. Unfortunately, automated traffic has largely been fraudulent in the past, and agents are getting caught in the crossfire. In June 2026, Cloudflare's CEO announced that bots had passed human traffic online for the first time in the internet's history. He expected that crossover by the end of 2027, and agentic traffic reached that point about 18 months early. At the time, Cloudflare Radar showed bots at 57.4% of requests vs. 42.6% from humans.
The traditional verification methods were failing spectacularly:
- User-Agent headers can be faked in one line of code, and poorly behaved bots often mask as Chrome specifically to avoid detection.
- IP address validation is a maintenance nightmare, because Cloud providers share IPs across customers. Also, allowlists that work for Google and Bing break down when 1,000+ different AI agents are trying to access the web.
- Per-site registration creates friction that only large, well-resourced bot operators can overcome, which centralizes power with incumbents.
So the IETF stepped in with a solution: HTTP Message Signatures (RFC 9421), which Web Bot Auth builds on. Bot operators sign their requests with a private key, and websites verify them with the matching public key.
- Unlike User-Agent headers, you can’t fake signatures without the private key.
- Unlike IP addresses, keys don’t change with your infrastructure.
- Unlike per-site tokens, a single signature works across the entire web.
How does Web Bot Auth work?
An operator generates an Ed25519 key pair and publishes the public key at /.well-known/http-message-signatures-directory on its domain. Every signed request then carries three headers:
Signature-Agenttells the site where to find the operator’s keys.Signature-Inputlists what was signed and the signature’s parameters, so the “signature base” can be reconstructed.Signatureholds the signature itself and is verified using the public key.
Signing @authority binds the signature to one host, so it can’t be replayed elsewhere. created, expires, and a nonce check that the key hasn’t expired and nonce hasn’t been reused.
If verification succeeds, the website knows with cryptographic certainty that this request came from whoever controls the private key at bot.example.com.
The IETF standard also requires asymmetric keys, so sites can verify without holding a shared secret, and key IDs must never correlate to a human. Keys identify an operator, but never a person.
However, this technical solution also created new problems around trust and centralization.
Identity is not trust
A valid signature can prove where the request comes from, but it says nothing about the operator’s reputation.
Key generation is free, so that means bad bots can also sign requests. Web Bot Auth isn’t focused on catching the bad bots; it’s to let productive and good work on the web stop looking like harmful traffic.
Like DKIM for email, signatures didn’t put a stop to spam, but it gave legitimate senders a way to prove who they are and build reputations that can’t be stolen.
IETF drew the line here and delegates end-user authentication, bot intent, and bot reputation to bot-protection programs, registries, and behavioural signals.
Decentralized protocol and centralized trust
The biggest criticism is that registries will centralize power and kill the open web, which is understandable.
Technically, registries are decentralized (anyone can run one), but most websites will use their access provider's default registry because it's easy. We’ve seen a similar issue before with email deliverability and certificate authorities. If you’re falsely blocked, the recovery options still vary by provider.
Web Bot Auth can’t settle this on its own, but it does offer a trust decision that’s visible and portable, which beats forgeable headers and opaque IP lists. The whole agent license path depends on multiple registries, transparent inclusion criteria, and site owners who maintain their own policies.
There does need to be some centralized trust if we all want driver’s licenses, don’t you think?
The real bottleneck is the site owner
The more pressing issue is that website owners and merchants have to be educated on why agents are doing good work and be given tools to manage what agents can do on their websites.
Should agents have access? → What should agents be able to do here?
We believe agents should be able to do the same work that humans do on the digital web. Native protocols for recognizing agents are how agents keep reliable access as usage scales.
It’s why we’re part of Cloudflare’s Signed Agents program, so participating websites can explicitly allow Browserbase sessions. Read our agent identity docs for access. If you run a website, check whether your bot-protection provider already verifies signatures.
Web Bot Auth is happening whether we like it or not. The question is what kind of ecosystem we build around it.
Contact Sales for Web Bot Auth access
Get started with identity for your browser agents
