Updated: 2026-07-11
π Paste your robots.txt content
π Robots.txt Specification
β’ User-agent: Specifies which crawler the rules apply to (* = all crawlers)
β’ Disallow: Paths the crawler should not access
β’ Allow: Paths the crawler can access (can override Disallow)
β’ Sitemap: Declares the sitemap URL for crawlers
β’ Crawl-delay: Sets the request interval in seconds
β’ File location: Should be at the root of the website (/robots.txt)
About Robots.txt Validator
Validate your robots.txt file for syntax errors, SEO issues, and best practices. Get a quality score and actionable fix suggestions.
Key Features
- Syntax validation with line-by-line analysis
- Quality score (0-100) with color-coded rating
- Error, warning, and info detection
- Actionable fix suggestions for each issue