Twenty-One AI Crawlers Came to Our Site. Five Rented Google Cloud IPs Were Wearing All of Them.
One address in Groningen made ten requests to our site in eight seconds. It arrived as Bravebot, then GPTBot, then GrokBot, then KimiBot, then GPTBot again, then MoonshotBot. In between it dropped the costume entirely and came as plain Chrome.
The paths it asked for were not pages. They were /.env.save, /.env.prod, /.aws/credentials, /actuator/env, /.env.production. A credential checklist, walked top to bottom, wearing a different AI company's name on almost every line.
This is not AI crawlers behaving badly
It would be an easier story if it were. It isn't.
Real GPTBot publishes its address ranges, and they are OpenAI's. Real ClaudeBot publishes Anthropic's. Real Googlebot comes from Google's crawler ranges, which are not the same thing as Google Cloud. Every address in this post sits in AS396982 — Google Cloud tenant space, rented by a customer. A GPTBot that originates from a GCP tenant IP is not GPTBot, by definition, no matter what the header says.
Across five such addresses — Groningen, Frankfurt, Council Bluffs and North Charleston twice — we counted twenty-one distinct AI crawler identities in use: GPTBot, ChatGPT-User, Claude-User, GrokBot, Bytespider, Perplexity-User, KimiBot, MoonshotBot, DeepSeekBot, Qwenbot, YiBot, Hunyuan, ChatGLM-Spider, cohere-ai, MistralAI-User, Bravebot, YouBot, Amazonbot, CCBot, DuckAssistBot and Googlebot.
Same ASN. Same target paths. Bursts of seconds. This is one operation with a costume rack.
The trick is old. The payoff is new.
People have faked Googlebot for twenty years, and Bingbot for nearly as long. Pretending to be a search crawler to get past a rate limit or into a members-only page is one of the oldest moves on the web, and every serious CDN has had verification for it since the 2000s.
What changed is not the technique. It is what the costume now opens.
In the last two years a very large number of sites added a rule that says, in effect, let the AI crawlers through. Sometimes it is a robots.txt courtesy. Often it is a real allowlist in a WAF or a bot-management product, written against the User-Agent string, because that is the field the vendor documentation shows you. Nobody allowlisted Bingbot into their credential files. Plenty of people have now allowlisted twenty AI user agents into everything, on the strength of a string any client can type.
That is the door. The costume is just a costume; somebody built a door that opens for costumes.
Google is the only party who can see both ends of this
We want to be precise about Google's position here, because it is neither "Google did this" nor "Google has nothing to do with it."
Google is the landlord. The operation runs on rented Google Cloud, the way a great deal of scanning runs on rented AWS and rented Azure. The tenant is the actor. We are not suggesting otherwise, and we would say the same of any host.
But one of the twenty-one costumes is Googlebot. So there is traffic leaving Google's own network, claiming to be Google's own crawler, from address space that Google knows perfectly well is not where its crawlers live. Every other party on the internet has to infer that mismatch from the outside, with partial information. Google can read it directly, on egress, as a single condition: this packet came from tenant space, it says Googlebot, our crawlers are not in tenant space.
There is no legitimate reason for a Google Cloud customer to emit a Googlebot User-Agent toward a third party. The same is true of Amazon and Amazonbot, and of any provider that both rents compute and runs a named crawler. That makes it one of the rare abuse signals with essentially no false positives, available to exactly one party, who also happens to hold the trademark being forged.
So the provider-side fix is: flag or block egress from tenant space that impersonates the provider's own crawler. It is cheap, it is unambiguous, and nobody downstream can do it. We reported the five addresses on October 25, with timestamps and the full User-Agent roster, to the abuse contact ARIN lists for that range.
We named four other networks in this post — a French ISP, a Polish datacenter, a Dutch host, a UK-registered network — without hesitation. It would have been cowardice to leave the fifth out because it is large.
We got this wrong ourselves, earlier today
Worth saying plainly, because it is the same mistake.
Eight hours ago, looking at a different question, I reported that our origin honeypot had recorded GrokBot and Meta-ExternalAgent fetching a Symfony debug endpoint, and I wrote it down as AI crawlers probing us. On this evidence that attribution is almost certainly wrong. It was the same impersonation, and I read a User-Agent as an identity in exactly the way this post is about.
It also reframes an earlier finding of ours. In October we reported that roughly 15% of AI-fetcher traffic to our sites was asking for secrets, and we built a handler that refuses those requests rather than serving fake credentials. We presented that as a statement about AI crawlers. A meaningful share of it was probably never AI crawlers at all.
We do not know what share. We are not going to guess at one.
The part that actually defends you
Here is the useful half, and it is not a blocklist.
Never grant access on a User-Agent. It is a free-text field that the client controls, which makes it a claim, not an identity. Verify instead, by either of the two methods the crawler operators themselves publish:
- Reverse DNS, then forward-confirm. Resolve the connecting IP to a hostname, then resolve that hostname back to an address and check it matches. Google, Bing and the larger AI operators all document this.
- Published address ranges. Most AI crawler operators now publish a JSON file of their egress ranges. Fetch it, cache it, check against it.
If a request claims to be GPTBot and fails both, it is not GPTBot, and whatever your allowlist was going to do for it should not happen.
This costs nothing, it is not a product, and you can do it this afternoon in whatever sits in front of your site. Our edge already separates a verified crawler from an unverified one that merely says it is one, and the twenty-one names above are why.
How we found it, which is its own lesson
We were not looking for this. We were chasing something else entirely.
Earlier today, RedEye Security's PHP2Shell write-up carried our telemetry, including a zero we had reported — no PHP2Shell-specific activity across 31,165 edge blocks. Checking whether we had earned that zero turned up a blind spot: we had no PHP decoys at all, so the zero could not have been anything else.
Building those decoys made us look at who had been probing PHP paths. Sixty-seven requests over twenty-six days, from thirteen addresses. And when we checked those addresses against our own corpus, ten of twelve were already there — nine of them first-party catches by our own honeypots at confidence 95, logged months earlier under completely different pretexts.
One of them first appeared on April 7 asking for .aws/credentials with a plain curl User-Agent. Six and a half months later, same network, same curl, different site of ours, now reading /vendor/composer/installed.json to inventory our PHP dependencies.
Another came back three separate times across three weeks, against three different properties of ours, with a different User-Agent each visit — an ancient Android, then a Chinese-locale Firefox 3.5, then Internet Explorer 6. Same single address, same city, every time.
The costume changes. The body does not.
Why that matters more than any list
The thing we keep relearning is that the durable part of an attacker is not the name on the thing they ship. Package names are single-use. User-Agents are free. Domains are cheap.
What recurs is infrastructure and behavior — the address, the network, the order they walk your paths in, the tooling fingerprint. Ten of twelve is a reuse rate high enough to be a control rather than a curiosity. We knew most of these operators before they came shopping, not because anyone reported them, but because they had already touched something of ours that was pretending to be worth stealing.
That is the whole argument for decoys over blocklists. A blocklist can only contain what somebody already wrote down. A decoy learns the body while the costume is still being chosen.
The honest scorecard
This post makes nine factual claims. Eight are measured directly from our own first-party records and dated: the ten requests and their timestamps, the five addresses, the twenty-one identities, the sixty-seven probes over twenty-six days, the ten-of-twelve overlap, the April-to-October recurrence, the three-visit User-Agent rotation, and the ASN.
The ninth — that these are impersonations rather than real crawlers — is an inference, though a strong one: it rests on the published fact that these operators crawl from their own ranges and not from Google Cloud tenant space. If any of those operators is in fact renting GCP for crawling and has not published it, we would be wrong about that name specifically, and we would correct it here.
We also counted one of the twelve addresses as a match on a historical record from February that our own system labels "not re-verified, not a current detection." We are not counting that as a first-party catch and neither should you.
And five percent of this is wrong somewhere. It always is.
Our threat feed and the free package and DNS checks that come out of this work are at analytics.dugganusa.com/stix/register. Registration is free and takes about thirty seconds.
How do AI models see YOUR brand?
AIPM has audited 250+ domains. 15 seconds. Free while still in beta.
Audit your site now → aipmsec.com
Was this useful? Thirty seconds, no cookies, no tracking, no third parties, your address hashed and never stored. If the box below does not load, the same question lives at https://analytics.dugganusa.com/nps.html?post=twenty-one-ai-crawlers-came-to-our-site-five-rented-google-cloud-ips-were-wearing-all-of-them