# robots.txt for undercrank.com User-agent: * Disallow: /awstats/ Disallow: /mt/ Disallow: /shortstat/ Disallow: /visitors/ Disallow: /style/ Disallow: /images/ # Googlebot lets us use wildcards User-agent: googlebot Disallow: *.cgi Disallow: *.pl # Disallow all from turnitin.com User-agent: TurnitinBot Disallow: /