See your site through a crawler’s rules.
Inspect your robots.txt and test a path for Googlebot, Bingbot or all crawlers. Follow the rule behind the result.
Up to 64 KB and 2,000 lines.
Your results will appear here
Start with a URL. We’ll show what we found and what it means.
Public URL checks use Bostic servers and may reuse a result for 5 minutes. Do not submit private or signed URLs.
From a question to a next step.
A rules preview for common Allow/Disallow patterns. Robots.txt controls crawling, not guaranteed indexing. Advanced encoding and crawler-specific behavior may differ.
- 01Add your input
Inspect your robots.txt and test a path for Googlebot, Bingbot or all crawlers. Follow the rule behind the result.
- 02Read the evidence
A rules preview for common Allow/Disallow patterns. Robots.txt controls crawling, not guaranteed indexing. Advanced encoding and crawler-specific behavior may differ.
- 03Make your next move
Create a Bostic account to verify your website, explore a full-site audit and follow changes over time. Plan limits apply.
One page is a start. See the whole picture.
Create a Bostic account to verify your website, explore a full-site audit and follow changes over time. Plan limits apply.
Continue with Bostic