# More information about robots can be found at # http://info.webcrawler.com/mak/projects/robots/norobots.html # Which robots/spiders to keep out User-agent: * # Directories that should not be indexed Disallow: /cgi-bin/ Disallow: /ht/ Disallow: /htis/ Disallow: /convert/ Disallow: /old/ Disallow: /tech_only/ Disallow: /tools/ Disallow: /php/ Disallow: /daniel/ Disallow: /training/ Disallow: /webrequest/forms/ Disallow: /statistics/daily/ Disallow: /statistics/monthly/reports/ Disallow: /statistics/weekly/reports/