AI crawlers / which sites block them / Finance

Which finance sites block AI crawlers?

Of 26 finance sites we checked on 2026-11-16, 3 (12%) block at least one AI crawler in robots.txt, and 2 (8%) block GPTBot.

SiteAI crawlers blockedGPTBotClaudeBotGoogle-ExtendedPerplexityBotCCBotChecked
marketwatch.com23 of 29AllowedBlockedBlockedBlockedBlocked2026-11-16
cnbc.com19 of 29BlockedBlockedBlockedBlockedBlocked2026-11-16
seekingalpha.com14 of 29BlockedBlockedAllowedAllowedBlocked2026-11-16
bankofamerica.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
bankrate.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
binance.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
capitalone.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
chase.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
citi.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
coinbase.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
creditkarma.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
fidelity.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
intuit.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
kraken.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
morningstar.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
nerdwallet.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
nyse.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
paypal.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
revolut.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
robinhood.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
sofi.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
tradingview.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
vanguard.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
visa.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
wellsfargo.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16
wise.com0 of 29AllowedAllowedAllowedAllowedAllowed2026-11-16

Open a site for the full list of 29 crawlers. 4 sites in this group could not be read.

How we know: we read each site's public robots.txt and test 29 AI crawler names against it. A site can also block crawlers at its firewall, which nobody outside can see, so "allows" here means "robots.txt doesn't say no". Each site is rechecked about once a week.

Other kinds of site: News · Reference · Shopping · Social and community · Developer tools · AI companies · Travel · Education · Government and public bodies · Entertainment and media · SaaS and business software