robots.txt generator for AI crawlers
Pick which AI crawlers to block and copy the result into your site's /robots.txt. Search engines like Googlebot and Bingbot aren't affected.
# Pick crawlers above"Block AI training only" blocks crawlers that collect content for training while still letting AI search and user-requested fetches through, so you can still be cited. Want to be paid instead of choosing? Pay-per-crawl charges AI crawlers per page with HTTP 402. Not sure what a crawler does? See the AI crawler list. To state paid or free terms in a standard format, use the RSL license generator.