CRX id
mnmjnaoidpknegiheffggblfkfdiidca
mnmjnaoidpknegiheffggblfkfdiidca
Description from extension meta
View websites according to robots.txt rules
Image from store
Description from store
RoboView: Visualize & Debug robots.txt Files
RoboView lets you experience websites exactly as search engine crawlers and AI bots do, by enforcing robots.txt rules in real-time. Perfect for SEO professionals, web developers, and content creators who need to verify crawler accessibility.
Key Features
- Multiple Bot Simulation - Switch between popular crawler user-agents:
* Search engines (Google, Bing, Baidu, Yandex)
* AI crawlers (GPTBot, Claude, Perplexity)
* Social media bots (Twitter/X, Facebook, LinkedIn)
- Visual Indicators - Instantly see which content is allowed or blocked:
* Blocked images appear grayed out with clear visual overlays
* Blocked scripts and resources are highlighted
* Full-page overlay for completely disallowed pages
- Two Enforcement Modes:
* Visual Mode: Show what's blocked without affecting functionality
* Strict Mode: Actually block resources like a real crawler would
- Cross-Domain Analysis - Examines robots.txt files from all resource domains, including CDNs and external hosts
- Detailed Statistics - Track blocked resources and robots.txt status across sites
- Developer Tools:
* Robust robots.txt parser with accurate pattern matching
* Comprehensive debug logs for troubleshooting
* Copy button for sharing logs with your team
Perfect For
- SEO specialists validating crawler access
- Web developers debugging robots.txt files
- Content creators ensuring AI training data access
- Security professionals verifying protection of sensitive content
RoboView makes robots.txt testing quick and visual - no more guesswork about what crawlers can access on your site!