AI crawlers / which sites block them / Travel

Which travel sites block AI crawlers?

Of 20 travel sites we checked on 2026-11-16, 3 (15%) block at least one AI crawler in robots.txt, and 2 (10%) block GPTBot.

SiteAI crawlers blockedGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotChecked
airbnb.com5 of 29BlockedBlockedAllowedAllowedAllowed2026-11-16
lonelyplanet.com1 of 29BlockedAllowedAllowedAllowedAllowed2026-11-16
skyscanner.net1 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
aa.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
agoda.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
amtrak.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
booking.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
expedia.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
hilton.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
kayak.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
lufthansa.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
lyft.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
marriott.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
priceline.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
rome2rio.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
southwest.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
trainline.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
trivago.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
uber.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
vrbo.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16

Open a site for the full list of 29 crawlers. 10 sites in this group could not be read.

How we know: we read each site's public robots.txt and test 29 AI crawler names against it. A site can also block crawlers at its firewall, which nobody outside can see, so "allows" here means "robots.txt doesn't say no". Each site is rechecked about once a week.

Other kinds of site: News · Reference · Shopping · Social and community · Developer tools · AI companies · Finance · Education · Government and public bodies · Entertainment and media · SaaS and business software