SEOmise
Book a demo Join the beta

Region and currency

Join the beta Book a demo
  1. Home
  2. All modules
  3. Grow
  4. What AI crawlers see

Grow

What AI crawlers see

How much of your page AI crawlers can read, and what your robots.txt lets them read

Many AI crawlers don't run JavaScript. See how much of your page they can read, what your robots.txt tells each AI crawler, and how to fix it.

Join the beta Book a demo

Illustration of the screen, without data.

Why it matters

From the problem to what SEOmise does

The problem

A page AI assistants cannot read

Text and links drawn by scripts are invisible to crawlers that don't run JavaScript, and a robots.txt rule can shut out an AI search crawler without anyone noticing.

What What AI crawlers see does

How much of your page AI crawlers can read, and what your robots.txt lets them read

  • One plain verdict. "Many AI crawlers don't run JavaScript: they can read about 15% of this page's text." Factual, with its source named.
  • What scripts add. Words, links to your other pages, headings and structured data types without JavaScript and in a browser.
  • Every named AI crawler. AI search crawlers and AI training crawlers from our list, allowed or blocked by your robots.txt.

How it works

Three steps, start to finish

  1. CompareYour audit already reads the page as sent and in our browser: words, links, headings and structured data, side by side.
  2. Read robots.txtAllowed or blocked for each named AI crawler, for this page, with its own rules where your robots.txt has them.
  3. Fix and checkThe steps for your platform, a guided walk through them, or our team. The next audit or scan shows the change.

Features

What you get with What AI crawlers see

One plain verdict

"Many AI crawlers don't run JavaScript: they can read about 15% of this page's text." Factual, with its source named.

What scripts add

Words, links to your other pages, headings and structured data types without JavaScript and in a browser.

Every named AI crawler

AI search crawlers and AI training crawlers from our list, allowed or blocked by your robots.txt.

Every platform

The fix steps name your platform: WordPress, Laravel, plain PHP, Shopify, Wix, Webflow, Squarespace, JavaScript apps, static sites and any other site.

No extra visit

Built from what the audit and the browser check already read.

Before and after

What changes with What AI crawlers see

Before: guessing. After: with What AI crawlers see. Drag the handle, or use the arrow keys, to compare.

Guessing

  • No idea what AI assistants can read
  • A robots.txt rule nobody remembers
  • Generic advice

With What AI crawlers see

  • The share of your text AI crawlers can read
  • Each AI crawler allowed or blocked
  • Fix steps for your platform
BeforeAfter

Illustration of the difference, not a measured result.

What it needs

No surprises after you sign up

Turn a card over for the detail.

JavaScript rendering

JavaScript rendering

The browser column needs JavaScript rendering; without it the panel shows robots.txt and the page as sent.

Every platform

Whatever your site runs on

What What AI crawlers see does on each kind of website, from SEOmise's site connectors to step-by-step guidance.

WordPress

WordPress and WooCommerce
Fully supported

Works on any public website, whatever it is built on.

Laravel

Sites built on Laravel
Fully supported

Works on any public website, whatever it is built on.

Any PHP site

Custom PHP, CodeIgniter, Symfony, Drupal, Joomla
Fully supported

Works on any public website, whatever it is built on.

Every other platform

Shopify, Wix, Squarespace, Webflow, static sites and the rest
Fully supported

Shopify, Wix, Webflow, Squarespace, JavaScript apps, static sites and every other platform: nothing to install.

About the site connectors

Where What AI crawlers see is included

What AI crawlers see is not part of a public plan at the moment.

In the SEO Toolbox Beta: €19 a month · €190 a year · €22.80 for 30 days. No contract. Cancel any time; you keep it until the end of what you paid for.

The tool's price gives you access. Each question uses 1 credit; each question uses 1 credit; each question uses 1 credit; credits are bought separately.

Ask for early access

See pricing

Not sure it fits?

Ask Mise, our assistant, in the help centre, or book a call and we will show you What AI crawlers see on a site like yours.

Book a demo Ask Mise

Questions

What AI crawlers see: common questions

Do AI crawlers run JavaScript?

Many don't: they read the HTML your server sends. We say "many AI crawlers", never a claim about one company's crawler.

Should I block AI training crawlers?

That is your choice. Blocking AI search crawlers keeps your site out of their answers; blocking training crawlers does not.

Which plans include What AI crawlers see?

In the SEO Toolbox it is sold on its own: €19 a month, €190 a year and €22.80 for 30 days. No contract. Cancel any time; you keep it until the end of what you paid for. The tool's price gives you access. Each question uses 1 credit; each question uses 1 credit; each question uses 1 credit; credits are bought separately. (Beta: ask for early access.) Prices and limits are on the pricing page.

Guides

What AI crawlers see: how-to guides

Every guide in the help centre

Try What AI crawlers see on your own site

Add your site; your first scan becomes your Day 0 baseline. Nothing is bought or published without your approval.

Join the beta Book a demo Ask Mise

No ranking guarantees: just clear work, with the evidence.