You found this in your logs
Reign is an affiliate marketplace. Our recruiter fetched a few pages of your site because it appeared in search results a merchant on our network competes in. This page says exactly what it did, what we kept, and how to stop it.
The line you are looking at
This is the exact User-Agent, in full. It is built from the address of whichever deployment did the fetching, so the link inside it always points at the page describing that deployment.
Mozilla/5.0 (compatible; ReignRecruiter/1.0; +https://goreign.co/recruiting)Most of our requests do not carry that string, and you should know why.
Every page we read is fetched through ScrapingBee, a commercial proxy — not as a retry on a refusal, as the only transport we use. Those requests arrive from ScrapingBee's IP addresses carrying ScrapingBee's own browser-like user agent, not ours. We do it because roughly an eighth of domains answer "unreachable" from a data centre, and recording that as "nobody is there" is the same mistake as inventing a figure. It does mean a request you cannot match to the string above may still have been us — and that a rule matching the string above does not stop us.
The one request that does come straight from us, carrying that string, is /robots.txt. We read it before every page, and a rules file we cannot read is treated as a refusal rather than as permission.
What it fetched
Your home page, then whichever of your own links look like a way to reach a person — contact, about, advertise. At most 10 pages, once. It follows your own navigation rather than guessing at URLs, specifically so we are not firing 404s at your server to learn how you spell "contact".
Once. We record that we looked, so a domain already read is not read again.
Everything behind a login, every form, every download. It reads HTML and nothing else, and it renders no JavaScript.
Two things: a contact address you have published, and evidence that your site carries affiliate links at all — an affiliate-network link, a disclosure, forum software, publisher-side ad tech.
What we keep, and why
If your site publishes a contact address, we store it along with the URL of the page it was printed on. That URL is the point: it is the record of where the address came from, and it is what makes “you published this so people in your trade could contact you” a claim you can check rather than one we assert.
We do not buy lists, we do not guess addresses from a person’s name, and we do not use a third-party enrichment database. If an address is not on your site, we do not have one.
We never pass your address to a merchant.
Merchants see the domain and the page you published your address on — never the address itself. It was collected so that we could write to you, and that is the only thing it is used for.
How to stop it
A Disallow in robots.txt works. Start there.
We fetch /robots.txt before every page and obey it. Name us directly and nothing else in the file has to change:
User-agent: ReignRecruiter
Disallow: /A group naming ReignRecruiter replaces your * group rather than adding to it, so it is the whole of what we read. A rules file we cannot read counts as a refusal, not as permission, and a path you disallow is never fetched — including through the proxy, because the check happens before the request.
Write to zac@darkrhino.com with the domain. It is suppressed across the whole network, permanently, and it is the only method that also stops the record being used.
Every message carries an opt-out link. One click suppresses your address across the whole network, permanently, and it works whether or not you ever reply.
Refuse requests whose User-Agent contains "ReignRecruiter" — a firewall rule, a WAF rule or a server config, anywhere that runs before your application. It stops anything that does arrive under our own name, which is the robots.txt read and little else — see the row below.
The page fetches do not carry our user agent at all: every one of them comes through ScrapingBee, under ScrapingBee's own browser-like agent and IP addresses. So a rule matching "ReignRecruiter" stops almost nothing — not because we route around your block, but because the proxy is the only transport we have. robots.txt is checked before the request is sent, which is why it works where an edge rule does not.
The whole of it is in the privacy notice. If you would rather see what this is actually for, check your own domain.