AI Terms / read it in code
Read AI Terms in your code
AI Terms is a file a site publishes at /.well-known/ai-terms that says, for each use of its pages by AI (search, answer, training, agent), whether it is allowed, allowed for a price, or not allowed. Two small open-source readers let a crawler or agent check it before it reads or pays.
JavaScript and TypeScript: ai-terms on npm
npm install ai-termsOr pin this release: npm install ai-terms@^0.1.0. Version 0.1.0, MIT, no dependencies, Node 18 or later. It is an ES module (use import) with TypeScript types included. npm · source
Code10 lines
import { fetchTerms, resolve, allowedWithin, contentUsage } from "ai-terms";
const terms = await fetchTerms("https://example.com"); // null when the site publishes none
if (terms) {
const rule = resolve(terms, "agent", "/blog/post"); // { access: "allow" | "paid" | "deny", ... }
if (!allowedWithin(rule, 0.01)) throw new Error("not allowed at my price");
console.log(contentUsage(terms, "/blog/post")); // e.g. "train-ai=n, ai-use=y"
}
// Say what you're doing on every request:
const headers = { "AI-Purpose": "agent" };| Function | What it does |
|---|---|
fetchTerms(site, { timeoutMs?, userAgent?, purpose?, fetch? }) | Reads https://<host>/.well-known/ai-terms. https only, redirects not followed, at most 100,000 bytes read. Returns the terms, or null when there are none or they can't be read. |
resolve(terms, use, path?) | The rule for one use (search, answer, training or agent) on one path: access, and price, attribution or free when set. The path is compared without its query. |
allowedWithin(rule, maxPriceUsd) | True when the use is allowed, or paid at or under your cap. Always cap what you pay: a file can name any price. |
freeAllowance(rule) | The free allowance on a paid or denied use ({ requests, per }), or null. Sites give it to verified crawlers only. |
contentUsage(terms, path?) | The IETF AI preferences value the terms imply for a path, e.g. "train-ai=n, ai-use=y" ("" when nothing can be stated). prefsFor gives the same as an object. |
validate(doc, { origin? }) | Checks a document against AI Terms 0.1: { valid, errors, warnings }. |
parse(text) | The terms in a fetched text, or null when it isn't an AI Terms file. |
termsLink(linkHeader) | The URL in a Link header's rel="ai-terms", or null. |
It also exports prefsFor, globMatch, normalizePath, isEndpointUrl, the USES list and the format's types. fetchTerms in a browser needs the site to allow cross-origin reads; everything else runs anywhere.
Python
One file, standard library only: readers/python/ai_terms.py (MIT). Copy it into your project.
Code7 lines
import ai_terms # copy ai_terms.py into your project: one file, standard library only
terms = ai_terms.fetch("https://example.com")
if terms:
rule = ai_terms.resolve(terms, "agent", "/blog/post")
if not ai_terms.allowed_within(rule, max_price_usd=0.01):
raise SystemExit("not allowed at my price")Both readers pass the same tests
The published test vectors (validation, path resolution and AI preferences) run against both: 58 of 58 pass. The JSON Schema describes the file. Both readers share an author with the reference implementation, so they are not independent implementations; if you write one, open a pull request on GitHub.
Then pay, if a use is priced
A reader tells you what a site allows and what it costs; it doesn't pay. To pay for a page, see for AI companies.