look into a robots.txt file on websites with a single click.
Inspect the source of every robots.txt file from the current website in your browser and display the content directly. With robots.txt caching, privacy settings and public robots.txt file archive.
:: Full changelog
https://www.robots-viewer.com/changelog
:: Public stats
https://www.robots-viewer.com/statistics
Latest reviews
- (2019-04-16) Turan Saat: perfect tool !!
- (2019-01-13) Ottmar Laux: All good. Works perfect!!!
Latest issues
- (2022-01-31, v:1.5.8) Don Hejlik: Ecosia hijacked Google Chrome
A search engine entitled Ecosia.org attached itself to Chrome in my desktop and two laptops, all with Windows 10, and set itself up as my search engine and I can't disable it. When I click on Settings, Search Engine, Manage Search Engine, it shows Bing, Yahoo, DuckDuck, etc., but when I click on the three dots, all that comes up is "Make Default," and "Edit," neither of which do anything. Robots.txt says to uncheck the statement,"disallowing search engine to fetch the page." How do I do that to get rid of Ecosia. Thank you in advance for any help you provide. - (2018-03-07, v:1.3.8) chayse Jarvis: urgent ill buy if this works
does this provide my website with protection from crawlers, and auto info gatherers.