User-agent: twiceler Disallow: / User-agent: Twiceler-0.9 Disallow: / User-agent: HTTrack/3.0x Disallow: / User-agent: HTTrack 3.0x; Windows 98 Disallow: / User-agent: HTTrack Website Copier/3.0x (offline browser; web mirror utility) Disallow: / User-agent: MJ12bot/v1.2.0 Disallow: / User-agent: MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+) Disallow: / User-agent: * Disallow: User-agent: SurveyBot Disallow: / User-agent: WebsiteWiki Disallow: / User-agent: ShopWiki Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-Agent: AboutUsBot Disallow: / User-agent: Semager Disallow: / User-agent: dotbot Disallow: / User-agent: * Disallow: /site/index.cfm$