User-agent: * Allow: /social_instagram/ Sitemap: https://www.nextdoo.cloud/sitemap.xml ############## # custom # ############## User-agent: * Allow: / Allow: /web/image Allow: /web/assets/ Allow: /web/content/ Disallow: /web/ Disallow: /my/ Disallow: /website/info Disallow: /groups Disallow: /shop Disallow: /shop/cart Disallow: /calendar # AI Search Crawlers - Allowed User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / # AI Training Only - Blocked User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: /