About this Tool
A robots.txt file tells search engine crawlers which pages or sections of your site should not be processed or scanned.
A robots.txt file tells search engine crawlers which pages or sections of your site should not be processed or scanned.