AI crawler access planner

Enter your official website. Review the information we find, then edit the draft.

Review & edit

First retrieve the current rules, then choose a policy. Existing rules remain available for comparison.

Current robots.txt

Enter your website above.

1. Choose a policy

robots.txt is a request to compliant crawlers, not a security control. User-agent names can be spoofed. The draft does not change your live website.

YOUR EDITABLE DRAFT

Your draft will appear here.

Review the draft with your website team before making it live.