User-agent: Amazonbot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: GPTBot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: * Allow: / Content-Signal: ai-train=no, search=yes, ai-input=no # Server-rendered file endpoints. These 403 every crawler (pdf-bot-guard), # so keep bots away instead of letting them collect access-denied errors. Disallow: /*/pdf$ Disallow: /*/stepsheets-pdf$ Disallow: /*/original-stepsheet$ Disallow: /*/social-image$ Disallow: /print/ # Authenticated / per-user surfaces: nothing here is indexable. Disallow: /admin Disallow: /settings Disallow: /notifications Disallow: /my-dances Disallow: /my-playlists Disallow: /my-collections Disallow: /my-custom-stepsheets Disallow: /saved-playlists Disallow: /favorites Disallow: /following Disallow: /timeline Disallow: /upload Disallow: /login Disallow: /register Disallow: /forgot-password Disallow: /reset-password Disallow: /two-factor # Editing and creation flows (all require auth, all soft-404 for bots). Disallow: /*/edit$ Disallow: /*/create$ Disallow: /*/analytics$ Disallow: /*/embed$ # Internal endpoints. Disallow: /api/ Disallow: /healthz # Legacy video paths (kept blocked); canonical channel pages live at /video-channels. Disallow: /videos # Stable on-host entry point; 301s to the generated index on the CDN. # Declaring it here is also what authorises the CDN-hosted shards to list # bootstepper.com URLs (sitemap cross-submission). Sitemap: https://bootstepper.com/sitemap.xml