Is my site blocking AI assistants? (robots.txt and llms.txt)

Two kinds of AI bot

• AI search and answer bots read pages so an assistant can answer a question and link to you: OAI-SearchBot and ChatGPT-User (ChatGPT), Claude-SearchBot and Claude-User (Claude), PerplexityBot and Perplexity-User (Perplexity), Bingbot (Bing and Copilot) and Googlebot (Google Search, AI Overviews and AI Mode). If you block these, those assistants cannot cite or link to your site.
• Training crawlers collect text to train AI models: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended and CCBot. Blocking them does not remove your site from AI search. Allowing them is a business choice, so SEOmise reports it but does not score it.

Check who can read your site

1. Open Grow with SEOmise → AI SEO and run the check.
2. Open the AI crawlers and files tab. Who can read your site lists every bot as Blocked or Allowed, from your robots.txt.
3. If AI search bots are blocked, Let the AI search bots back in shows the lines to add to robots.txt. The training crawlers stay as they are.

If your site has no robots.txt, every bot may read it.

The check also looks for nosnippet or max-snippet:0 on your pages. These settings stop Google quoting a page, including in AI Overviews and AI Mode.

llms.txt

llms.txt is a short, plain-language summary of your business with links to your key pages, kept at /llms.txt on your site. It is a new proposal that some AI tools read; no major search engine has confirmed using it, so it counts for little in the score.

• Press Draft an llms.txt, check the summary line, then save it as /llms.txt on your site.
• Or put the Publish an llms.txt guide fix in the fix jar. The WordPress connector (0.15.0 or newer) and the Laravel connector (0.8.0 or newer) can publish it for you. On other sites you get the file to upload.