AI crawlers / which sites block them / Social and community

Which social and community sites block AI crawlers?

Of 30 social and community sites we checked on 2026-11-16, 19 (63%) block at least one AI crawler in robots.txt, and 17 (57%) block GPTBot.

SiteAI crawlers blockedGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotChecked
facebook.com29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
instagram.com29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
nextdoor.com29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
reddit.com29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
threads.net29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
x.com29 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
lobste.rs28 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
linkedin.com26 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
pinterest.com26 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
flickr.com22 of 29BlockedBlockedBlockedAllowedBlocked2026-11-16
snapchat.com19 of 29BlockedBlockedBlockedAllowedBlocked2026-11-16
tiktok.com18 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
letterboxd.com13 of 29BlockedBlockedBlockedAllowedBlocked2026-11-16
mastodon.social13 of 29BlockedBlockedBlockedAllowedBlocked2026-11-16
tumblr.com11 of 29AllowedBlockedBlockedAllowedBlocked2026-11-16
medium.com8 of 29BlockedBlockedAllowedAllowedAllowed2026-11-16
vimeo.com6 of 29AllowedAllowedAllowedAllowedBlocked2026-11-16
strava.com4 of 29BlockedBlockedBlockedAllowedAllowed2026-11-16
meetup.com1 of 29BlockedAllowedAllowedAllowedAllowed2026-11-16
bsky.app0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
dev.to0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
deviantart.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
discord.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
hashnode.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
news.ycombinator.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
producthunt.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
soundcloud.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
substack.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
twitch.tv0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
youtube.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16

Open a site for the full list of 29 crawlers.

How we know: we read each site's public robots.txt and test 29 AI crawler names against it. A site can also block crawlers at its firewall, which nobody outside can see, so "allows" here means "robots.txt doesn't say no". Each site is rechecked about once a week.

Other kinds of site: News · Reference · Shopping · Developer tools · AI companies · Finance · Travel · Education · Government and public bodies · Entertainment and media · SaaS and business software