AI crawlers / which sites block them / Entertainment and media
Which entertainment sites block AI crawlers?
Of 29 entertainment sites we checked on 2026-11-16, 17 (59%) block at least one AI crawler in robots.txt, and 11 (38%) block GPTBot.
| Site | AI crawlers blocked | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Checked |
|---|---|---|---|---|---|---|---|
| ign.com | 25 of 29 | Blocked | Blocked | Allowed | Blocked | Blocked | 2026-11-16 |
| netflix.com | 20 of 29 | Allowed | Allowed | Allowed | Allowed | Blocked | 2026-11-16 |
| pitchfork.com | 19 of 29 | Allowed | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| primevideo.com | 18 of 29 | Allowed | Allowed | Allowed | Blocked | Blocked | 2026-11-16 |
| billboard.com | 17 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| hollywoodreporter.com | 17 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| rollingstone.com | 17 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| variety.com | 17 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| gamespot.com | 14 of 29 | Blocked | Blocked | Blocked | Allowed | Blocked | 2026-11-16 |
| polygon.com | 14 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| hulu.com | 13 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| nba.com | 9 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| espn.com | 8 of 29 | Blocked | Allowed | Blocked | Allowed | Blocked | 2026-11-16 |
| kotaku.com | 8 of 29 | Allowed | Allowed | Allowed | Allowed | Blocked | 2026-11-16 |
| metacritic.com | 5 of 29 | Blocked | Allowed | Allowed | Allowed | Blocked | 2026-11-16 |
| disney.com | 2 of 29 | Blocked | Allowed | Blocked | Allowed | Allowed | 2026-11-16 |
| playstation.com | 1 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| apple.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| crunchyroll.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| fifa.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| max.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| mlb.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| nfl.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| paramountplus.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| peacocktv.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| rottentomatoes.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| spotify.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| steampowered.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| xbox.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
Open a site for the full list of 29 crawlers. 1 site in this group could not be read.
How we know: we read each site's public robots.txt and test 29 AI crawler names against it. A site can also block crawlers at its firewall, which nobody outside can see, so "allows" here means "robots.txt doesn't say no". Each site is rechecked about once a week.
Other kinds of site: News · Reference · Shopping · Social and community · Developer tools · AI companies · Finance · Travel · Education · Government and public bodies · SaaS and business software