AI crawlers / Applebot-Extended
Applebot-Extended: what it is and how to block or charge it
Applebot-Extended is run by Apple. A robots.txt token, not a separate crawler: it controls whether content Applebot crawls may be used to train Apple's AI models. Blocking it does not remove a site from Siri or Spotlight.
| Operator | Apple |
|---|---|
| Purpose | Collects content for AI training |
| robots.txt token | Applebot-Extended |
Block Applebot-Extended in robots.txt
User-agent: Applebot-Extended
Disallow: /Because Applebot-Extended is only a robots.txt token, robots.txt is the only way to control it; it never shows up in your server logs.
Or charge Applebot-Extended per page
Applebot-Extended can't be charged directly, because it's a permission token rather than a visitor. Pay-per-crawl works on crawlers that fetch pages themselves, like pay-per-crawl GPTBot and ClaudeBot.
Is Applebot-Extended reading your site?
See the full list of AI crawlers. Facts here follow each operator's own documentation; tell us if one changes.