# robots.txt for http://www.kloth.org - 2007-03-02 # # No bots, except crawlers for public search indexes, like Google, Msn, Yahoo/Slurp, Ask/Teoma # If you want your bot allowed, contact hostmaster at kloth.org # User-agent: * Disallow: / User-agent: Exabot Disallow: /int99/ User-agent: Gigabot Disallow: /int99/ User-agent: Googlebot Disallow: /int99/ User-agent: Mediapartners-Google Disallow: /int99/ User-agent: msnbot Disallow: /int99/ User-agent: Slurp Disallow: /int99/ User-agent: Teoma Disallow: /int99/ User-agent: Voilabot Disallow: /int99/ User-agent: yacy Disallow: /int99/ User-agent: ZyBorg Disallow: /int99/ # #User-agent: ia_archiver #Disallow: / #