Can AI assistants read your site?
Paste your robots.txt. We show, for 17 AI and search crawlers, whether they can reach the page you pick. Search bots decide if you get cited. Training bots do not.
| Crawler | Job | Result |
|---|
Search bots and training bots are different
Each AI company runs separate crawlers. Search and user bots fetch pages so the assistant can answer and cite you. Training bots collect pages to train future models. You can block training and still be cited. For example, blocking GPTBot does not stop ChatGPT search from citing you, but blocking OAI-SearchBot does.
robots.txt is not the whole story
A firewall or CDN bot rule can block these crawlers even when robots.txt allows them. OpenAI also asks advertisers not to block OAI-AdsBot or OAI-SearchBot on ad landing pages. If a crawler shows as allowed here but you still never appear, check your CDN's bot settings.
How we read your file
We follow the standard (RFC 9309): the most specific user-agent group wins, the longest matching path rule wins, and Allow beats Disallow on a tie. Wildcards * and $ are supported. Your file never leaves your browser.