extension ExtPose

Robots.txt Disallowed

CRX id

hgkaceioogcdcihdfobicbnfcmnbpnec-

Description from extension meta

This extension will report if a page is disallowed in robots.txt

Image from store Robots.txt Disallowed
Description from store See if a URL is allowed or disallowed in robots.txt for a specific user-agent token. If the URL is disallowed, it will not be crawled by the crawler with the matching user-agent token.

Statistics

Installs
Category
Rating
0.0 (0 votes)
Last update / version
2020-06-05 / 1.1
Listing languages
en

Links