# CRELoaded Generated Robots.txt # Robot Exclusion File -- robots.txt # Author: CRELoaded Team # Last Updated : May 11th 2005 User-Agent: * Crawl-delay: 30 Disallow: /admin/ Disallow: /download/ Disallow: /temp/ Disallow: /pub/ Disallow: /urchin/ Disallow: /test/ Disallow: /ajax__*.php