- Help centre
- Website health and changes
- What does Crawlers and bots show?
What does Crawlers and bots show?
Short answerWeb SEO → Crawlers and bots shows which bots read your site, how often, and where search engines run into trouble: pages that answer search engines differently, and blocked or noindex pages they read often, with fix steps for your platform. Visits come from the site connector or from access logs you upload.
Crawlers and bots is under Web SEO in each project.
Where the visits come from
- The site connector (WordPress, Laravel or the universal PHP connector, recent versions): SEOmise asks it once a day for the bot visits per page. No visitor addresses are kept.
- Access logs you upload: your server's access logs, including Cloudflare Logpush files. Hosted platforms such as Shopify, Wix or Squarespace usually give no logs; the page says so.
What you see
- Visits per day, per bot.
- Each bot with its classification: verified (checked against the search engine's published list), claimed (it only says it is) or unknown. You can correct a classification for your project.
- Alerts: pages that give search engines different answers, and blocked or noindex pages that are read often, each with fix steps for your site's platform. Alerts can also be e-mailed.
- The pages search engines read, joined with Indexing status.
Not built yet: reading Cloudflare Logpush straight from your own storage, and views of what AI answer engines read.
Not what you needed? Ask Mise or contact support.
Still stuck?
Ask Mise, or a person
Mise answers from these help articles in seconds. If the answer is not here, raise a ticket and our support team replies by e-mail.
