Pay-per-crawl / setup / WordPress

Block or charge AI crawlers on WordPress

WordPress runs a large share of the web. The plugin below charges AI crawlers per page; no server access needed. Works for GPTBot, ClaudeBot, PerplexityBot, CCBot, Bytespider and 22 more AI crawlers; Googlebot and Bingbot are not affected.

  1. 1
    Create your free routeOne field: your site. No wallet, no card.
  2. 2
    Add it to WordPressInstall the plugin. No code.
  3. 3
    Watch your reportSee which AI crawlers visit. Charging stays off until you turn it on.

Option 1: block them

Code8 lines
# Add to your site's robots.txt (with an SEO plugin's robots.txt editor, or a robots.txt file in the site root):
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: PerplexityBot
User-agent: Bytespider
User-agent: Google-Extended
Disallow: /

Blocking also keeps you out of AI answers and citations. Pick crawlers one by one with the robots.txt generator.

Option 2: charge them per page

First create your free route (site, price, wallet); this page then fills in your route id.

  1. Download the plugin (.zip)
  2. In WordPress: Plugins > Add New > Upload Plugin, choose the file, Install, Activate.
  3. Settings > AIPayGen pay-per-crawl: paste your route id and gateway secret, and save.

The plugin only acts on AI crawlers and paid requests; logged-in users, the admin, cron and the REST API are untouched. If the gateway can't be reached, the page is served as usual.

Before anything is sold through AIPayGen, your site is checked once. Plugin 2.6 or later does this by itself: it serves your AI Terms at /.well-known/ai-terms, which shows the site is yours, so it verifies within about an hour of installing (we check for that file). Nothing to add. If that check can't reach it, your report shows other ways to prove it. On an older plugin, update it.

Lost your gateway secret? Get a new one by email.

Behind Cloudflare's proxy (an orange-cloud DNS record) or another CDN, the plugin (2.6) sees the CDN's addresses instead of your visitors'. AI crawlers are still counted, but those whose companies publish their addresses show as "(fake)", and crawlers hiding their name aren't caught by address. Plugin 2.7 reads Cloudflare's visitor address; this note goes away then.

Using a page cache or CDN?

Page-cache plugins can serve a saved copy before WordPress (and this plugin) runs. Add all 27 AI crawler names the plugin checks (listed below, one per line) to your cache plugin's list of user agents never to cache: WP Super Cache calls it Rejected User Agents, WP Rocket Never Cache User Agent(s), LiteSpeed Cache Do Not Cache User Agents, W3 Total Cache Rejected user agents. If a CDN caches your HTML too, make it bypass the cache for the same names, or not cache HTML.

GPTBot
OAI-SearchBot
ChatGPT-User
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
PerplexityBot
Perplexity-User
GoogleOther
CCBot
Bytespider
Amazonbot
meta-externalagent
FacebookBot
cohere-ai
Diffbot
YouBot
MistralAI-User
DuckAssistBot
PetalBot
Timpibot
ImagesiftBot
omgili
AI2Bot
ElevenlabsCrawler
ElevenlabsBot

Check it works

Report-only (the free start, no wallet):

curl -I -A GPTBot https://your-site.com/   # 200: report-only serves the page to AI crawlers too
curl -I https://your-site.com/             # 200 for everyone else

Both answer 200 in report-only mode; then open your report: that visit shows up usually within the hour (the plugin sends its counts about once an hour, when WordPress runs its scheduled tasks, which happens on site traffic). It may be listed as GPTBot (fake): it came from your address, not one OpenAI publishes, which is the check working. Your report's Test my setup button also checks it for you.

Charging (with a wallet):

curl -i -A GPTBot https://your-site.com/   # 402 with your price
curl -I https://your-site.com/             # 200 for everyone else

The 402 is counted too: open your report: that visit shows up usually within the hour (the plugin sends its counts about once an hour, when WordPress runs its scheduled tasks, which happens on site traffic). It may be listed as GPTBot (fake): it came from your address, not one OpenAI publishes, which is the check working.

Or run the free AI crawler check on your site. Other platforms: Cloudflare · Nginx · Caddy · Apache · Next.js · Vercel (not Next.js) · Netlify · Express / Node.js · Django / Python.

Show it on your site

This badge shows the date your AI Terms file last passed our check, and links to a page that lists your terms in plain words. Until your file passes, it says "not checked": check your file. Replace your-site.com with your site's address.

HTML

Code1 lines
<a href="https://aipaygen.com/ai-terms/site/your-site.com"><img src="https://aipaygen.com/badge/ai-terms.svg?site=your-site.com" alt="AI Terms checked by AIPayGen" height="20"></a>

Markdown

[![AI Terms checked by AIPayGen](https://aipaygen.com/badge/ai-terms.svg?site=your-site.com)](https://aipaygen.com/ai-terms/site/your-site.com)

It says only that the file passed our check on that date. It is not an approval, and it can't tell you whether any AI company follows the terms.

Uninstall

Plugins > Installed Plugins: deactivate AIPayGen pay-per-crawl, then Delete. Deleting it removes its settings and stored counts from WordPress. Your site then serves every visitor as before.

To delete your route and its data at AIPayGen too, use delete my sites and data (by the email you confirmed). Your sites and their reports: my sites.