→ Our robots
DigifyScope robots
If you see one of these in your site logs, it is us:
DigifyScopeAudit/1.0Someone has requested an audit of this site (speed, security, cookies). It only comes when an audit has been requested.DigifyScopeBot/1.0A look at publicly available company websites: whether the site works, whether it has HTTPS, which system it runs on, the published contacts.
Our robot used to be called CoreDigifyBot and came from bot.coredigify.com. Same robot, just a new name. If your robots.txt already has lines for the old name, they still work.
How we behave
DigifyScopeBotobeysrobots.txt, includingCrawl-delay(up to 30 s). We follow the rules for bothDigifyScopeBotandCoreDigifyBot. If either of them disallows us, we do not come.- We also respect
<meta name="robots">,<meta name="digifyscopebot">,<meta name="coredigifybot">andX-Robots-Tagwithnoindex/nofollow. - For one site — no more often than once every few seconds; for one server — no more often than once a second, even if it hosts many sites.
- A
429or503response withRetry-After— we wait for the time given. If the site's protection stops us, we do not come back that day. - We read the home page and up to 6 pages it links to itself (contacts, “about us” and the privacy policy first, where the company details are). On a site that links to many other .lv sites, we crawl up to 15 pages once, with the same
robots.txtand pause rules. We do not guess addresses, log in, fill in forms or exploit vulnerabilities. - In the cookie check
DigifyScopeAuditonly clicks the cookie banner buttons “Accept” or “Reject”.
How to opt out
From DigifyScopeBot — add this to your robots.txt:
User-agent: DigifyScopeBot
Disallow: /
The old name User-agent: CoreDigifyBot works too. To keep the site out of the LV TOP list (our ranking of Latvian websites) but still allow a look, <meta name="digifyscopebot" content="noindex"> is enough.
From all checks, including DigifyScopeAudit — robots.txt does not stop a requested audit, because a person asked for it. For a .lv domain, fill in the form “Do not check my site” (in Latvian; confirmation with a DNS record takes effect at once) or write to info@digifyscope.com with the domain: we will exclude it and its subdomains from all checks, including the ones others request.
Where we come from
Our list of IP addresses is being finalised — before setting up an allowlist, write to info@digifyscope.com.
Machine-readable list (the same format as Google's robots): https://digifyscope.com/bot/ips.json.
How to make sure it really is us. Anyone can fake a User-Agent, so check the IP with reverse DNS: the address's PTR record must be bot.digifyscope.com (or end with .bot.digifyscope.com), and that name (forward DNS) must point back to the same IP.
host 203.0.113.10 # → bot.digifyscope.com
host bot.digifyscope.com # → 203.0.113.10
Our robot's address used to be bot.coredigify.com.
If your site blocks us
If your site's protection (Cloudflare, a firewall, CAPTCHA) stops our robot, the audit report shows a section “The site blocks automated checks — add us to your allowlist”. It is not an error on your site — only a sign that other robots (search engines, AI assistants) may also find it harder to get in.
- We do not get around a block: we do not change the User-Agent, solve CAPTCHAs or try from other addresses.
- After a block we send this site no requests for 24 hours; after a
429— for as long asRetry-Afterasks. - If you want a full audit, allow us. Best — User-Agent and IP together, not just the User-Agent (it can be faked).
Cloudflare — Security → WAF → Custom rules → “Create rule”, action Skip (tick “All remaining custom rules”, “Super Bot Fight Mode” and “Browser Integrity Check”), expression:
((http.user_agent contains "DigifyScope" or http.user_agent contains "CoreDigify") and ip.src in { OUR_IP })
Wordfence — Wordfence → Firewall → All Firewall Options → “Allowlisted IP addresses that bypass all rules”: add our IPs.
Sucuri — Firewall → Access Control → “Allowlist IP”: add our IPs.
Hosting firewall (ModSecurity, Imunify360 etc.) — forward this page to your hosting company or developer and ask them to allow our IP addresses.
Once you have allowed us, run the audit again. If you have confirmed that the site is yours, or if you write to us, we can lift the back-off at once.
Googlebot comparison mode
Some hacked sites show search engines different content than people (for example casino or “18+” ads the owner never sees in their own browser). To spot that, we can open the site once with the Googlebot User-Agent and compare it with the normal view.
- Only for sites whose owner has confirmed ownership (or our own sites) — never in bulk checks, in audits requested by others or in free audits of unverified sites.
- The request comes from the same IP addresses of ours, so a reverse DNS check will not show Google — that is the sign it is us.
- Never to get around a block; every run is recorded in our log.