SEOmise reads your pages as SEOmiseBot and says so in every request. It never pretends to be a browser, never ticks a box or solves a puzzle, and never sends your pages through an unblocking service. When your firewall or bot protection answers with a challenge, a block (a 403 or 429 from a bot manager) or a "tick the box" check instead of the page, SEOmise reads the page another way, or shows you how to let it in.
1. Read through your SEOmise connector (automatic)
If the website has the SEOmise connector (WordPress plugin 0.52.0, Laravel package 0.42.0 or PHP connector 0.36.0, or later), the connector may read your site, and you have verified the website (Domain verification), SEOmise asks the connector for each blocked page. The connector reads the page from inside your own server and sends it back signed.
- This works in scans, the website audit, fix checks and install checks.
- The page is checked like every other page and marked read through your SEOmise connector. Its checks count as measured, so the grade is not "Partial" because of them.
- Up to 200 pages a scan (a setting), at most 60 a minute (the connector's own limit), and only pages of your own website. The connector's answer is checked again: HTML only, up to 2 MB, and a page that is itself a firewall's challenge is not taken.
- A connector read is part of the scan, so it waits for the scan's turn in the queue.
- If the connector cannot help (it is offline, too old to read pages, or your firewall stops it too), the page says Your connector couldn't read this page: and the reason, and the box below offers to let SEOmise in.
Pages that need a real browser (the background rendering of paid projects and the live checks) cannot be read through the connector, which never renders: they keep the plain read and say why.
2. "Let SEOmise in"
When pages were blocked, the website audit, the scan panel and Site health show a box: " blocks automated visitors, so N pages couldn't be read."
If your site's Cloudflare zone is connected under MiseGuard → CDN with a write token that has Zone → Zone WAF → Edit:
- The box shows Let SEOmise in, with exactly the rule it would add. Only the workspace's owner or an admin can press it.
- SEOmise adds one allow rule to that zone only, for SEOmiseBot only: its user agent together with SEOmise's published addresses or your secret header. Never "allow all", and never the user agent alone.
- The change goes through CDN → Changes: recorded, shown with who asked for it, and Undo removes it with one click.
- About a minute later SEOmise re-checks the blocked pages (in the queue's first lane) and tells you how many open now.
If Cloudflare is not connected, choose:
- Connect Cloudflare: the usual connection under MiseGuard → CDN, then come back for Let SEOmise in.
- Do it yourself: the box shows only the steps for your site's platform and the firewall SEOmise found, using SEOmise's real user agent, published addresses and Web Bot Auth details:
- Cloudflare: a custom rule with the action Skip, placed first.
- Sucuri: the allowlist under Access Control.
- Wordfence (WordPress): Firewall Options → allowlisted addresses.
- Akamai, Imperva, DataDome, HUMAN, DDoS-Guard: a bot-management exception for SEOmiseBot only.
- Shopify, Wix, Squarespace, Webflow and other hosted platforms: there is no firewall setting of yours; a CDN in front of the site, or the platform's support, can let SEOmiseBot in.
- A host's or server's own firewall: allow the user agent "SEOmiseBot", or add it to the "block unknown crawlers" exceptions.
3. Re-check
After any fix, press Re-check the blocked pages (a few times an hour). SEOmise reads the blocked pages of your latest scan again and says:
- All pages open: scan again to check them.
- Some open: the others are still blocked.
- Still blocked: the rule may take a few minutes, or another firewall is in the way.
Check again still reads your start page at once.
Free audits and prospects
The free audit and prospect checks have no connector and no workspace website, so nothing is unblocked for them: a blocked page gives an honest Partial result and the "blocked reader" card.
Settings (staff)
Under Settings → Policies → Blocked sites: reading blocked pages through the connector (on) and the pages per scan (200), the box (on), re-checks per person per hour (6), pages per re-check (20), the wait before the re-check after Let SEOmise in (60 seconds), and the Cloudflare allow rule's own settings. Platform Admin → System → Crawl control shows this week's connector reads and Let SEOmise in rules.
Not what you needed? Ask Mise or contact support.
