Draft 0.1 · open for comment

AI Terms: tell AI what it may do with your pages, and what it costs.

AI Terms is a file a website publishes at /.well-known/ai-terms that says, for each use of its content by AI (search, answer, training, agent), whether that use is allowed, allowed for a price, or not allowed. What that means, in plain words.

It adds what robots.txt can't say: the purpose, the price and how to pay, who the crawler really is, and a receipt for every paid read.

What robots.txt can't say

robots.txt answers one question: may this crawler name fetch this path? It has no idea of a use (AI search versus training), no price, no way to check a crawler is who it says, and no record of what was taken. AI Terms keeps robots.txt and adds those four.

Four uses

searchIndexing pages to show links or snippets in AI search results.
answerFetching a page live to answer a person's question right now.
trainingCollecting pages to train or fine-tune AI models.
agentAn autonomous agent reading pages or calling endpoints to complete a task.

Each use is allow, paid (with a price per page or request) or deny, with optional attribution. Parts of a site can override with paths.

How a read works

  1. AskThe site publishes its terms per use at /.well-known/ai-terms: what is free, what costs what, what needs attribution.
  2. ProveThe crawler's identity is checked against its operator's published addresses (and signed requests as they arrive).
  3. AgreeThe crawler declares its purpose in an AI-Purpose header, and so accepts that use's terms.
  4. PayPaid uses answer 402 with an x402 quote; the crawler pays the site's wallet directly and gets the page.
  5. ReceiptThe paid response carries the settled transaction and the declared purpose: a record of licensed use for both sides.

An example file

A site that lets AI search and answers in free with attribution, and charges $0.002 a page for training and agents:

Code48 lines
{
  "ai_terms": "0.1",
  "site": "https://example.com",
  "updated": "2026-11-16",
  "uses": {
    "search": {
      "access": "allow",
      "attribution": "required"
    },
    "answer": {
      "access": "allow",
      "attribution": "required"
    },
    "training": {
      "access": "paid",
      "price": {
        "amount": "0.002",
        "currency": "USD",
        "per": "page"
      }
    },
    "agent": {
      "access": "paid",
      "price": {
        "amount": "0.002",
        "currency": "USD",
        "per": "page"
      }
    }
  },
  "default": "paid",
  "identity": {
    "verify": [
      "published-ip-ranges"
    ],
    "unverified": "treated-as-unidentified"
  },
  "purpose_header": "AI-Purpose",
  "payment": {
    "protocol": "x402",
    "version": 2,
    "endpoint": "https://aipaygen.com/g/abc123def456",
    "network": "eip155:8453",
    "asset": "USDC"
  },
  "receipts": "x402 PAYMENT-RESPONSE header: the settled transaction on Base, verifiable on-chain",
  "spec": "https://aipaygen.com/ai-terms"
}

Built on what exists

robots.txt still decides what may be fetched. llms.txt guides what to read. RSL licenses can be linked from the terms. The four uses are meant to map onto the IETF AI preferences vocabulary as it settles. Identity uses published IP ranges today and signed requests (Web Bot Auth) as they arrive. Payment is x402. AI Terms is the file that ties them into one loop a machine can follow.

Make your file

No signup. Pick what each use may do, copy the file, and serve it at /.well-known/ai-terms on your site. Charging needs somewhere to send the payment, so paid uses come from a free AIPayGen route, which generates and serves the file for you.

Check a file

Enter a site to check the file it serves, or paste a file. Checked against draft 0.1: the JSON Schema plus the rules a schema can't express. Building a reader? Run it against the test vectors.

For agent builders

Before your agent reads or pays for a page, read the site's terms. This is the whole check, with the Python reader (one file, standard library only):

Code10 lines
import ai_terms

terms = ai_terms.fetch("https://example.com")          # None if the site publishes none
if terms:
    rule = ai_terms.resolve(terms, "agent", "/blog/post")
    if not ai_terms.allowed_within(rule, max_price_usd=0.01):
        raise SystemExit("not allowed for agents at my price")

# on every request, say what you are doing:
headers = {"AI-Purpose": "agent"}

Three rules cover a well-behaved client: don't use a page for a use the site denies, pay before a use that is paid, and cap what you'll pay, because a file can name any price. Writing a reader in another language? Check it against the test vectors and open a pull request; it will be listed.

Use it today

Every site on AIPayGen pay-per-crawl gets its terms generated from its settings at /v1/gateway/terms/<your-route>, linked from every 402 answer, and enforced at the edge. The format is open: implement it anywhere, and send comments or pull requests on GitHub.