How to use the blacklist
The blacklist lets you exclude specific URLs or entire domains from being scanned. Here is how it works and when to use it.
What is the blacklist?
The blacklist is a list of URL patterns that dislike404.com will skip during a scan. Any URL that matches a blacklist entry will not be crawled and will appear in your scan results with the status User Blacklist.
This is useful when you want to exclude URLs that are intentionally restricted, irrelevant to your monitoring, or belong to external services that block automated access.
Where to find the blacklist
There are three ways to add entries to your blacklist:
During setup — there is an optional blacklist field when adding a new website
From the dashboard — via the blacklist button in your website control panel, where you can add, edit, and delete entries at any time
From your scan results — each crawled URL has an option to add it directly to your blacklist, which is the quickest way to exclude a specific URL you just spotted in your results
Changes take effect on the next scan.
How blacklist entries work
Each entry is a URL pattern. You can use exact URLs or the wildcard character *, which matches zero or more characters. dislike404.com checks every crawled URL against your blacklist — if it matches any entry, the URL is skipped.
Wildcard examples
Exact URL: https://example.com/dont_scan_me.html
Both HTTP and HTTPS: http*://example.com/dont_scan_me.html
Entire path and everything under it: http*://example.com/dont_scan_me/*
Subdomain: http*://dontscan.example.com*
Entire domain: *example.com*
When to use the blacklist
The goal after every scan should be a clean result with no errors. Every URL in your results either represents a real issue that needs to be fixed, or a false positive that should be added to your blacklist.
You are seeing false positives in your results — for example, JavaScript-generated URLs that the crawler attempts to resolve but cannot reach. This is one of the most common sources of false positives and adding these patterns to your blacklist is the recommended way to handle them. If you are unsure whether an error is a real issue or a false positive, check out our guide on how to interpret your scan results
An external website consistently blocks the dislike404.com crawler and returns 403 or 429 errors
You have pages behind authentication that the crawler cannot access
You link to a third-party service that does not allow automated access
You want to exclude a section of your website that is under construction
You are not interested in monitoring certain areas of your site
The global blacklist
In addition to your personal blacklist, dislike404.com maintains a global blacklist of domains that are known to block automated access. URLs matching the global blacklist are automatically skipped for all users and appear in your results with the status Global Blacklist. You cannot edit the global blacklist.
If you believe a domain has been added to the global blacklist by mistake or should be removed, please contact us and we will review it.