Marketplace · API · x402

pennyregwatch.com/v1/robots

Check robots.txt before crawling or scraping a page. Decide whether GPTBot, ClaudeBot, Googlebot, or another crawler may fetch the exact URL and return ALLOW/DENY, winning rule, crawl-delay, sitemap URLs, and live provenance.

Endpoint	`https://pennyregwatch.com/v1/robots`
Trust	✓ Verified callable · 100% uptime over 1 checks · ~564 ms
Pricing	Pay per call with x402, no account. Call it to get an exact quote before paying.
network	eip155:8453
scheme	exact
pay to	0x5deaebaf98034cc0d9abe3757a440b121c42ef0a
method	GET
last updated	2026-10-23T22:35:48.66Z

Use it

Call it (an x402 client pays automatically)

curl -i https://pennyregwatch.com/v1/robots
# 402 reply -> sign PAYMENT-REQUIRED with an x402 client (e.g. @x402/fetch) -> retry with PAYMENT-SIGNATURE

This listing as JSON

curl https://aipaygen.com/v1/catalog/api/x402-5f3cfd3b9ab1385c

Search the whole marketplace from your agent (MCP)

{
  "mcpServers": {
    "aipaygen": {
      "url": "https://aipaygen.com/mcp"
    }
  }
}