User-agent: * Allow: / Disallow: /quality-reports/ # AI crawler permissions User-agent: GPTBot Allow: / # ChatGPT Search crawler (separate from GPTBot) User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: anthropic-ai Allow: / User-agent: CCBot Allow: / # Well-known AI resources # llms.txt: https://free-toolbase.com/llms.txt # llms-full.txt: https://free-toolbase.com/llms-full.txt # Sitemap Sitemap: https://free-toolbase.com/sitemap.xml IndexNow: 5117df90-ca22-4ffb-9482-a09420c05601