AI crawlers / which sites block them / Finance
Which finance sites block AI crawlers?
Of 26 finance sites we checked on 2026-11-16, 3 (12%) block at least one AI crawler in robots.txt, and 2 (8%) block GPTBot.
| Site | AI crawlers blocked | GPTBot | ClaudeBot | Google-Extended | PerplexityBot | CCBot | Checked |
|---|---|---|---|---|---|---|---|
| marketwatch.com | 23 of 29 | Allowed | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| cnbc.com | 19 of 29 | Blocked | Blocked | Blocked | Blocked | Blocked | 2026-11-16 |
| seekingalpha.com | 14 of 29 | Blocked | Blocked | Allowed | Allowed | Blocked | 2026-11-16 |
| bankofamerica.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| bankrate.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| binance.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| capitalone.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| chase.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| citi.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| coinbase.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| creditkarma.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| fidelity.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| intuit.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| kraken.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| morningstar.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| nerdwallet.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| nyse.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| paypal.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| revolut.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| robinhood.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| sofi.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| tradingview.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| vanguard.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| visa.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| wellsfargo.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
| wise.com | 0 of 29 | Allowed | Allowed | Allowed | Allowed | Allowed | 2026-11-16 |
Open a site for the full list of 29 crawlers. 4 sites in this group could not be read.
How we know: we read each site's public robots.txt and test 29 AI crawler names against it. A site can also block crawlers at its firewall, which nobody outside can see, so "allows" here means "robots.txt doesn't say no". Each site is rechecked about once a week.
Other kinds of site: News · Reference · Shopping · Social and community · Developer tools · AI companies · Travel · Education · Government and public bodies · Entertainment and media · SaaS and business software