# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # AI training-set scrapers. They replay years-old image URLs in bulk. User-agent: ImageBot Disallow: / User-agent: MathPicDatasetCrawler Disallow: /