User-agent: id_a_f_3 !Susie (http://www.sync2it.com/susie) Sync2It bookmark management & clustering engine C R http://www.sync2it.com id_a_f_6 <a href='http://www.unchaos.com/'> UnChaos </a> From Chaos To Order Hybrid Web Search Engine.(vadim_gonchar@unchaos.com) UnCHAOS search robot R Site is dead http://www.unchaos.com/ id_a_f_7 <a href='http://www.unchaos.com/'> UnChaos Bot Hybrid Web Search Engine. </a> (vadim_gonchar@unchaos.com) UnCHAOS search robot R Site is dead http://www.unchaos.com/ id_a_f_8 <b> UnChaosBot From Chaos To Order UnChaos Hybrid Web Search Engine at www.unchaos.com </b> (info@unchaos.com) UnCHAOS search robot R Site is dead http://www.unchaos.com/ id_a_f_9 <http://www.sygol.com/> http://www.sygol.com Sygol Search (Italy) robot R s.also SygolBot http://www.sygol.com/ id_a_f_4 ( Robots.txt Validator http://www.searchengineworld.com/cgi-bin/robotcheck.cgi ) SearchEngineWorld's robots.txt validator C Services is no more available http://www.searchengineworld.com/cgi-bin/robotcheck.cgi id_a_f_171105_1 (DreamPassport/3.0; isao/MyDiGiRabi) DreamCast DreamPassport browser B http://www.dricas.com/dp/ id_a_f_290606_1 (Privoxy/1.0) Privoxy web proxy P s.also Privoxy/3.0 (Anonymous) http://www.privoxy.org/ id_a_f_230507_1 */Nutch-0.9-dev Unknown Yahoo robot R 123.113.184.2xx http://www.yahoo.com id_a_f_5 +SitiDi.net/SitiDiBot/1.0 (+Have Good Day) SitiDi.net search (Germany) robot R http://www.sitidi.net/ id_a_f_1 -DIE-KRAEHE- META-SEARCH-ENGINE/1.1 http://www.die-kraehe.de Die Kraehe Meta-Search-Engine (Germany) link checking R http://www.die-kraehe.de id_a_f_060206_1 123spider-Bot (Version: 1.02, powered by www.123spider.de 123spider.de (Germany) web directory link checking C http://www.123spider.de/ id_a_f_180806_1 192.comAgent 192.com - UK web directory R 217.160.75.2xx http://www.192.com/ id_a_f_060606_1 1st ZipCommander (Net) - http://www.zipcommander.com/ 1st ZipCommander Net - IE based browser B http://www.zipcommander.com/ id_a_f_10 2Bone_LinkChecker/1.0 libwww-perl/5.64 2Bone online link checker C http://www.2bone.com/links/linkchecker.shtml id_a_f_11 4anything.com LinkChecker v2.0 4Anything robot R http://www.4anything.com id_a_f_110207_1 8484 Boston Project v 1.0 Unknown guestbook spamming or harvesting tool from diff. IPs S s. various honey pot sites http://www.projecthoneypot.org/bsh_X19tb2RlPWdsb2JhbCZfX2J5PWMmY3RyeT11cyZ1YWc9ODQ4NCtCb3N0b24rUHJvamVjdCt2KzEuMA.. id_a_f_2 :robot/1.0 (linux) ( admin e-mail: undefined http://www.neofonie.de/loesungen/search/robot.html ) neofonie search robot Germany R http://www.neofonie.de/loesungen/search/ id_a_f_12 A-Online Search A-Online.at robot - now Jet2Web Search R http://www.jet2web.net/portal id_a_f_090707_1 A1 Keyword Research/1.0.2 (+http://www.micro-sys.dk/products/keyword-research/) miggibot/2007.03.27 A1 Keyword Research - search engine and keyword optimization software C http://www.micro-sys.dk/products/keyword-research/ id_a_f_100906_1 A1 Sitemap Generator/1.0 (+http://www.micro-sys.dk/products/sitemap-generator/) miggibot/2006.01.24 MiggiBot website crawler engine - A1 Sitemap Generator R http://www.micro-sys.dk/products/sitemap-generator/ http://www.micro-sys.dk/developer/miggibot/ id_a_f_250408_2 aardvark-crawler Aardvark web crawler for Sun's Blog recommendations R 192.9.71.7x http://blogs.sun.com/plamere/ id_a_f_13 AbachoBOT Abacho / Crawler.de robot R http://www.abacho.com id_a_f_14 AbachoBOT (Mozilla compatible) Abacho / Crawler.de robot R http://www.abacho.com id_a_f_15 ABCdatos BotLink/5.xx.xxx#BBL ABCdatos - Castilian program & tutorial directory R http://www.abcdatos.com id_a_f_16 Aberja Checkomat Aberja Hybridsuchmaschine (Germany) link checking R http://www.aberja.de id_a_f_17 abot/0.1 (abot; http://www.abot.com; abot@abot.com) Nameprotect copyright search robot (24.177.134.x) R s. also - np/0.1_(np;_http://www.nameprotect.com... - aipbot/1.0 (aipbot; http://www.aipbot.com... http://www.nameprotect.com/ id_a_f_18 About/0.1libwww-perl/5.47 About robot R http://www.about.com id_a_f_051206_3 Accelatech RSSCrawler/0.4 Accela Technology RSS feed crawler R 125.100.242.2xx http://www.accelatech.com/ id_a_f_19 accoona Accoona Search robot R 65.17.255.xx http://www.accoona.com id_a_f_20 Accoona-AI-Agent/1.1.1 (crawler at accoona dot com) Accoona Search robot R 65.17.255.xx http://www.accoona.com id_a_f_140906_3 Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com) Accoona Search robot R 65.17.255.xx http://www.accoona.com id_a_f_21 Ace Explorer Ace Explorer - IE based browser B http://www.aceexplorer.com id_a_f_280806_1 Ack (http://www.ackerm.com/) Ackerm search robot R 64.74.153.xx http://www.ackerm.com/ id_a_f_22 AcoiRobot Acoi picture finder robot R http://monetdb.cwi.nl/acoi/projects.html id_a_f_23 Acoon Robot v1.50.001 Acoon.de search (Germany) robot R 80.237.153.10x http://www.acoon.de id_a_f_24 Acoon Robot v1.52 (http://www.acoon.de) Acoon.de search (Germany) robot R 80.237.153.10x http://www.acoon.de id_a_f_100508_1 Acoon-Robot 4.0.x.[xx] (http://www.acoon.de) Acoon.de search (Germany) robot R 80.237.209.xx http://www.acoon.de id_a_f_010108_1 Acoon-Robot v3.xx (http://www.acoon.de and http://www.acoon.com) Acoon.de search (Germany) robot R 80.237.153.10x http://www.acoon.de id_a_f_110507_1 Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org) Acorn Search Project R 124.157.145.1xx http://acorn.no-ip.org/ id_a_f_25 ActiveBookmark 1.x LibMaster.com Active Bookmark HTML page creator C B http://www.libmaster.com/software.php id_a_f_060806_2 Activeworlds Activeworlds 3D homepage browser B http://www.activeworlds.com id_a_f_26 ActiveWorlds/3.xx (xxx) Activeworlds 3D homepage browser B http://www.activeworlds.com id_a_f_27 Ad Muncher v4.xx.x Ad Muncher - banner killer P http://www.admuncher.com/ id_a_f_28 Ad Muncher v4x Build xxxxx Ad Muncher - banner killer P http://www.admuncher.com/ id_a_f_061006_1 Adaxas Spider (http://www.adaxas.net/) website directory adaxas link checking C 85.10.199.xx http://www.adaxas.net/ id_a_f_29 Advanced Browser (http://www.avantbrowser.com) Avant Browser - IE based browser B http://www.avantbrowser.com id_a_f_30 AESOP_com_SpiderMan Aesop robot R http://www.aesop.com id_a_f_31 agadine/1.x.x (+http://www.agada.de) Agada search (Germany) robot R s. also Mozilla/4.0 (agadine3.0) http://www.agada.de id_a_f_32 Agent-SharewarePlazaFileCheckBot/2.0+(+http://www.SharewarePlaza.com) SharewarePlaza link checking R http://www.sharewareplaza.com id_a_f_33 AgentName/0.1 libwww-perl/5.48 Linkomatic submission verifier R http://www.linkomatic.com/ id_a_f_34 AIBOT/2.1 By +(www.21seek.com A Real artificial intelligence search engine China) 21seek.com (China) robot (218.17.90.xxx) R http://www.21seek.com id_a_f_130807_1 AideRSS/1.0 (aiderss.com) AideRss - Postrank RSS and Blog filtering C 72.44.35.2xx http://www.aiderss.com/ id_a_f_35 aipbot/1.0 (aipbot; http://www.aipbot.com; aipbot@aipbot.com) Nameprotect copyright search robot (24.177.134.x) R s. also - np/0.1_(np;_http://www.nameprotect.com... - abot/0.1 (abot; http://www.abot.com... http://www.nameprotect.com/ id_a_f_36 aipbot/2-beta (aipbot dev; http://aipbot.com; aipbot@aipbot.com) Nameprotect copyright search robot (24.177.134.x) R s. also - np/0.1_(np;_http://www.nameprotect.com... - abot/0.1 (abot; http://www.abot.com... http://www.nameprotect.com/ id_a_f_37 Aladin/3.324 Aladin robot R http://www.aladin.de id_a_f_180408_1 Alcatel-BG3/1.0 UP.Browser/5.0.3.1.2 Phone.com UP.Browser for mobiles on Alcatel cellphone B id_a_f_38 Aleksika Spider/1.0 (+http://www.aleksika.com/) Aleksika Danmark - Search engine optimization spider R http://www.aleksika.com id_a_f_39 AlkalineBOT/1.3 Vestris robot R http://alkaline.vestris.com/ id_a_f_40 AlkalineBOT/1.4 (1.4.0326.0 RTM) Vestris robot R http://alkaline.vestris.com/ id_a_f_41 Allesklar/0.1 libwww-perl/5.46 Allesklar.de robot R http://www.allesklar.de id_a_f_42 Alligator 1.31 (www.nearsoftware.com) Alligator download manager D http://www.nearsoftware.com id_a_f_43 AltaVista Intranet V2.0 AVS EVAL search@freeit.com Altavista robot R http://www.altavista.com id_a_f_44 AltaVista Intranet V2.0 Compaq Altavista Eval sveand@altavista.net Altavista robot R http://www.altavista.com id_a_f_45 AltaVista Intranet V2.0 evreka.com crawler@evreka.com Altavista robot R http://www.altavista.com id_a_f_46 AltaVista V2.0B crawler@evreka.com Altavista robot R http://www.altavista.com id_a_f_47 AmfibiBOT Amfibi Search robot R 64.111.217.9x http://www.amfibi.com/ id_a_f_48 Amfibibot/0.06 (Amfibi Web Search; http://www.amfibi.com; agent@amfibi.com) Amfibi Search robot R 64.111.217.9x http://www.amfibi.com/ id_a_f_141105_2 Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com) Amfibi Search robot R 64.111.217.9x http://www.amfibi.com/ id_a_f_49 amibot amibot - Amidalla search engine robot (62.241.33.xx) R s. also libwww-perl/5.65 http://www.amidalla.com/ id_a_f_50 Amiga-AWeb/3.4.167SE AWeb Amiga browser B http://www.amitrix.com/aweb.html id_a_f_51 AmigaVoyager/3.4.4 (MorphOS/PPC native) Voyager - Amiga browser B http://www.vapor.com/voyager/ id_a_f_091205_1 AmiTCP Miami (AmigaOS 2.04) Amiga Miami TCP Stack http://home.ptd.net/~strdustr/amirc/Netware.html id_a_f_210608_1 Amoi 8512/R21.0 NF-Browser/3.3 NF embedded browser on Amois Skypephone B http://3skypephone.com/ id_a_f_190206_3 amzn_assoc Amazon.com robot for checking their affiliate sites C s. also aranhabot id_a_f_52 AnnoMille spider 0.1 alpha - http://www.annomille.it Annomille Italian historical oriented robot R http://www.annomille.it id_a_f_53 annotate_google; http://ponderer.org/download/annotate_google.user.js annotate Google - Firefox extension for annotating Google search results B http://ponderer.org/annotate_google id_a_f_54 Anonymized by ProxyOS: http://www.megaproxy.com Megaproxy user P http://www.megaproxy.com id_a_f_55 Anonymizer/1.1 faked user agent id_a_f_56 AnswerBus (http://www.answerbus.com/) AnswerBus natural language search using COLLATE technology R http://www.answerbus.com http://collate.dfki.de/kurzdarstellung.html id_a_f_57 AnswerChase PROve x.0 AnswerChase search tool http://www.answerchase.com id_a_f_58 AnswerChase x.0 AnswerChase search tool http://www.answerchase.com id_a_f_59 ANTFresco/x.xx ANT Fresco Browser B s. also Mozilla/x.xx (compatible; ANTFresco....) http://www.antlimited.com/products/fresco.htm id_a_f_60 antibot-V1.1.5/i586-linux-2.2 Antibot (discontinued) robot R http://www.antidot.net id_a_f_61 AnzwersCrawl/2.0 (anzwerscrawl@anzwers.com.au;Engine) Anzwers (Yahoo) Australia robot R http://au.anzwers.yahoo.com/ id_a_f_030206_1 Apexoo Spider 1.x Apexoo Search spider R 216.240.143.xx http://www.apexoo.com/ id_a_f_62 Aplix HTTP/1.0.1 JavaOS app. for SEGA Saturn Internet and Sanyo Internet-TV B id_a_f_63 Aplix_SANYO_browser/1.x (Japanese) JavaOS app. for Sanyo Internet-TV B id_a_f_64 Aplix_SEGASATURN_browser/1.x (Japanese) JavaOS app. for SEGA Saturn Internet B id_a_f_65 Aport Aport robot R http://www.aport.ru id_a_f_66 appie 1.1 (www.walhello.com) Walhello Internet Search robot R http://www.walhello.com id_a_f_140608_1 Apple iPhone v1.1.4 CoreMedia v1.0.0.4A102 CoreMedia player on Apple iPhone B http://www.apple.com/iphone/ id_a_f_120707_1 ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby.com;) Araby search - Arabia R 209.85.31.2xx http://www.araby.com/ id_a_f_260608_1 ArachBot Covac Arachnid Web Crawler R s.also Covac TexAs Arachbot http://www.covac-software.com/ id_a_f_67 Arachnoidea (arachnoidea@euroseek.com) Euroseek spider R http://www.euroseek.com id_a_f_190206_2 aranhabot Amazon.com robot for checking their affiliate sites C s. also amzn_assoc id_a_f_68 ArchitextSpider Excite spider R http://www.excite.com id_a_f_69 archive.org_bot Heritrix - The Internet Archive's open-source crawler (207.241.225.2xx) R s.also - InternetArchive/0.8-dev - Mozilla/5.0 (compatible;archive.org_bot/... http://www.archive.org/ id_a_f_70 Argus/1.1 (Nutch; http://www.simpy.com/bot.html; feedback at simpy dot com) Simpy Bookmarklet crawler (69.55.233.xx) C s. also Simpy http://www.simpy.com/ id_a_f_031205_1 Arikus_Spider Arikus inContext search engine software R http://www.arikus.com/inContext-enterprise.html id_a_f_210208_1 Arquivo-web-crawler (compatible; heritrix/1.12.1 +http://arquivo-web.fccn.pt) Tomba project: the Portuguese web archive R 193.136.192.xx http://arquivo-web.fccn.pt/ http://arquivo-web.fccn.pt/crawler?set_language=en id_a_f_251007_1 ASAHA Search Engine Turkey V.001 (http://www.asaha.com/) Asaha search robot (Turkey) R 62.68.194.2xx http://www.asaha.com/ id_a_f_71 Asahina-Antenna/1.x ASAHINA Antenna information detecting agent R http://masshy.fastwave.gr.jp/hina/release/ id_a_f_72 Asahina-Antenna/1.x (libhina.pl/x.x ; libtime.pl/x.x) ASAHINA Antenna information detecting agent R http://masshy.fastwave.gr.jp/hina/release/ id_a_f_73 ask.24x.info Ask 24x Info robot R http://ask.24x.info/ id_a_f_74 AskAboutOil/0.06-rcp (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@askaboutoil.com) Ask About Oil - Petroleum related search (24.227.212.xxx) using Nutch R http://askaboutoil.com/search.jsp http://www.nutch.org id_a_f_120806_1 asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com) askEd! / Inferret search (Japan) robot using Nutch R 131.112.125.1xx http://asked.jp/ id_a_f_75 ASPSeek/1.2.5 ASPSeek search engine software -Yahoo-Inc. / Telecom Canada robot R http://www.aspseek.org id_a_f_76 ASPseek/1.2.9d Swsoft.net robot using Aspseek R http://www.aspseek.com id_a_f_77 ASPSeek/1.2.x ASPSeek search engine software R http://www.aspseek.org id_a_f_78 ASPSeek/1.2.xa ASPSeek search engine software R http://www.aspseek.org id_a_f_79 ASPseek/1.2.xx ASPSeek search engine software R http://www.aspseek.org id_a_f_80 ASPSeek/1.2.xxpre ASPSeek search engine software R http://www.aspseek.org id_a_f_81 ASSORT/0.10 Associative Sort robot R site is down http://pcmath126.unice.fr/assort-robot.html id_a_f_82 asterias/2.0 Singingfish media spider (64.12.186.2xx) via AOL search R s. also Mozilla/4.0 (compatible; MSIE 6.0 compatible; Asterias Crawler ... http://search.singingfish.com/sfw/home.jsp id_a_f_83 AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html) Atlocal local business search robot R http://www.atlocal.com/ id_a_f_041207_2 Atomic_Email_Hunter/4.0 Atomic Email Hunter email extracing and harvesting S id_a_f_84 Atomz/1.0 Atomz robot R http://www.atomz.com id_a_f_85 atSpider/1.0 atSpider (ceased) email harvester / spambot S id_a_f_060707_1 Attentio/Nutch-0.9-dev (Attentio's beta blog crawler; www.attentio.com; info@attentio.com) Attentio social media monitoring and analysing R 85.88.35.xx http://www.attentio.com/ id_a_f_231105_2 AU-MIC/2.0 MMP/2.0 Samsung SPH-A660 phone with Sprint software B http://www1.sprintpcs.com/explore/showcase/Showcase.jsp?scTopic=pcsVision id_a_f_240208_1 AUDIOVOX-SMT5600 Audiovox SMT5600 (AT&T) Smartphone mobile phone browser B http://www.audiovox.com/ http://www.audiovox.com/manuals/owners/SMT%205600%20QSG%209-7%20FINAL.pdf id_a_f_86 augurfind Augurnet Swiss (was www.augurnet.ch) search robot R id_a_f_87 augurnfind V-1.x Augurnet Swiss (was www.augurnet.ch) search robot R id_a_f_88 autoemailspider Auto Email Pro Email harvester S was http://autoemailspider.com - site is dead id_a_f_89 autohttp Linkscan tool from Elsop C s. Linkscan/x ? http://www.elsop.com/linkscan/ id_a_f_050208_1 autowebdir 1.1 (www.autowebdir.com) Autowebdir - The Automatically Generated Web Directory R 84.104.43.x http://www.autowebdir.com/ id_a_f_90 AV Fetch 1.0 Altavista robot ?? R http://www.altavista.com/ id_a_f_91 Avant Browser (http://www.avantbrowser.com) Avant Browser - IE based browser B s. also Orca Browser http://www.avantbrowser.com id_a_f_92 AVSearch-1.0(peter.turney@nrc.ca) National Research Council Canada robot R http://www.nrc-cnrc.gc.ca/main_e.html id_a_f_93 AVSearch-2.0-fusionIdx-14-CompetitorWebSites Unknown robot from 205.203.108.xx (telerate.com) id_a_f_94 AVSearch-3.0(AltaVista/AVC) Altavista robot R http://www.altavista.com/ id_a_f_95 AWeb AWeb Amiga browser B http://aweb.sunsite.dk/ id_a_f_96 axadine/ (Axadine Crawler; http://www.axada.de/; ) Axada search Germany robot R http://www.axada.de id_a_f_97 AxmoRobot - Crawling your site for better indexing on www.axmo.com search engine. Axmo search robot R http://www.axmo.com id_a_f_250306_1 Azureus 2.x.x.x Azureus Java BitTorrent Client D http://azureus.sourceforge.net/ id_a_f_98 BaboomBot/1.x.x (+http://www.baboom.us) BaBoom Web Portal (ODP) robot (66.98.254.xx) R http://www.baboom.us id_a_f_050406_1 BackStreet Browser 3.x BackStreet Browser - Offline browser / website downloader D http://www.spadixbd.com/backstreet/ id_a_f_140508_1 BaiduImagespider+(+http://www.baidu.jp/search/s308.html) Baidu search (Japan) image crawler R 119.63.193.94.[x]xx http://www.baidu.jp/ id_a_f_99 BaiDuSpider Baidu spidering engine - used by diff. IPs R http://www.baidu.com id_a_f_080608_1 Baiduspider+(+http://help.baidu.jp/system/05.html) Baidu spidering engine - used by diff. IPs R http://www.baidu.com id_a_f_100 Baiduspider+(+http://www.baidu.com/search/spider.htm) Baidu spidering engine - used by diff. IPs R http://www.baidu.com id_a_f_080407_1 Baiduspider+(+http://www.baidu.com/search/spider_jp.html) Baidu search (Japan) crawler R 119.63.193.[x]xx http://www.baidu.jp id_a_f_031107_1 Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) Balihoo - Vertical search engine crawler (beta) R http://www.balihoo.com/ http://www.balihoo.com/pdfs/BalihooFactSheet.pdf id_a_f_101 BanBots/1.2 (spider@banbots.com) Project BanBots Perl script robot C s. also Mozilla/5.0 (compatible; BanBots/2.0b.. http://www.banbots.com/ id_a_f_140506_3 Barca/2.0.xxxx Barca Pro email & PIM software B http://www.pocosystems.com/home/index.php?option=com_content&task=view&id=105&Itemid=54 id_a_f_120206_2 BarcaPro/1.4.xxxx Barca Pro email & PIM software B http://www.pocosystems.com/home/index.php?option=com_content&task=view&id=105&Itemid=54 id_a_f_102 BarraHomeCrawler (albertof@barrahome.org) Barrahome crawler R 64.246.56.xx http://www.barrahome.org id_a_f_011006_1 bCentral Billing Post-Process Unknown user agent from Microsoft 204.71.191.1xx id_a_f_103 bdcindexer_2.6.2 (research@bdc) Business.com robot R 208.144.233.xxx http://www.business.com id_a_f_104 BDFetch Brandimensions Brand Protection robot R http://www.brandimensions.com/ id_a_f_105 BDNcentral Crawler v2.3 [en] (http://www.bdncentral.com/robot.html) (X11; I; Linux 2.0.44 i686) Bdncentral Sitesearch robot R http://www.bdncentral.com id_a_f_111205_3 BeamMachine/0.5 (dead link remover of www.beammachine.net) beammachine web directory (Germany) link checking R C http://www.beammachine.net/de/ id_a_f_106 beautybot/1.0 (+http://www.uchoose.de/crawler/beautybot/) Beauty robot for Cosmoty - German beauty and wellness search R http://www.cosmoty.de http://www.uchoose.de/Projekte/Lifestyle/beauty/ id_a_f_230406_1 BebopBot/2.5.1 ( crawler http://www.apassion4jazz.net/bebopbot.html ) A Passion for Jazz music related search robot R 68.6.204.2xx http://www.apassion4jazz.net/ id_a_f_107 BeebwareDirectory/v0.01 LinkcheckerBeepware (site is down) web directory link checking C http://directory.beebware.co.uk id_a_f_108 Big Brother (http://pauillac.inria.fr/~fpottier/) Big Brother link checking tool C http://pauillac.inria.fr id_a_f_130606_1 Big Fish v1.0 GoonGee.com link popularity checking C 216.89.111.x http://www.goongee.com/big-fish/ id_a_f_109 BigBrother/1.6e BB4 network monitoring C http://www.bb4.com id_a_f_110 BigCliqueBOT/1.03-dev (bigclicbot; http://www.bigclique.com; bot@bigclique.com) BigClique Search robot R http://www.bigclique.com id_a_f_080206_4 BIGLOTRON (Beta 2;GNU/Linux) Biglotron search (France) robot R http://www.biglotron.com/ id_a_f_171106_2 Bigsearch.ca/Nutch-x.x-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com) Bigsearch.ca search robot R 72.0.207.1xx http://www.bigsearch.ca/ id_a_f_111 Bilbo/2.3b-UNIX Bilbo - web frontend for the Nessus Security Scanner C http://home.broadpark.no/%7Etnilsen-1/Linux/Bilbo_-_Nessus_WEB/bilbo_-_nessus_web.html id_a_f_170806_1 BilgiBetaBot/0.8-dev (bilgi.com (Beta) ; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) Bilgi.com (Beta) search robot - Turkey R 212.156.230.2xx http://www.bilgi.com/ id_a_f_080607_1 BilgiBot/1.0(beta) (http://www.bilgi.com/; bilgi at bilgi dot com) Bilgi.com (Beta) search robot - Turkey R 212.156.230.2xx http://www.bilgi.com/ id_a_f_112 billbot wjj@cs.cmu.edu Carnegie Mellon School robot/link checking ? C http://www.cs.cmu.edu id_a_f_050206_3 Bitacle bot/1.1 Bitacle Blog Search Archive robot R http://bitacle.org/ id_a_f_050206_2 Bitacle Robot (V:1.0;) (http://www.bitacle.com) Bitacle Blog Search Archive robot R http://bitacle.org/ id_a_f_113 Biyubi/x.x (Sistema Fenix; G11; Familia Toledo; es-mx) Biyubi Navigator - Mexican browser for Fenix OS B http://www.biyubi.com id_a_f_114 BlackBerry7520/4.0.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/5.0.3.3 UP.Link/5.1.2.12 (Google WAP Proxy/1.0) Blackberry Wireless Internet browser via Google WAP Proxy B P http://www.blackberry.com/products/service/web.shtml id_a_f_010306_1 BlackWidow BlackWidow web site scanner / downloading tool D http://www.softbytelabs.com/BlackWidow id_a_f_115 BlackWidow FS Consulting (was www.fsconsult.net) Black Widow web crawler R id_a_f_116 Blaiz-Bee/1.0 (+http://www.blaiz.net) Blaiz Enterprises RawGrunt search R 203.87.123.1xx http://www.rawgrunt.com/ http://www.blaiz.net id_a_f_160307_1 Blaiz-Bee/2.00.8222 (BE Internet Search Engine http://www.rawgrunt.com) Blaiz Enterprises RawGrunt search R 203.87.123.1xx http://www.rawgrunt.com/ http://www.blaiz.net id_a_f_240706_1 Blaiz-Bee/2.00.xxxx (+http://www.blaiz.net) Blaiz Enterprises RawGrunt search R 203.87.123.1xx http://www.rawgrunt.com/ http://www.blaiz.net id_a_f_117 BlitzBOT@tricus.net Blitzsuche Germany robot R http://blitzsuche.rp-online.de/ id_a_f_118 BlitzBOT@tricus.net (Mozilla compatible) Blitzsuche Germany robot R http://blitzsuche.rp-online.de/ id_a_f_090307_1 BlockNote.Net BlockNote web page editor B http://blocknote.net/ id_a_f_119 BlogBot/1.x blogdex robot from MIT.edu R http://blogdex.media.mit.edu/ id_a_f_071206_1 BlogBridge 2.13 (http://www.blogbridge.com/) BlogBridge RSS reader B http://www.blogbridge.com/ id_a_f_120 Bloglines Title Fetch/1.0 (http://www.bloglines.com) Bloglines article search R http://www.bloglines.com id_a_f_250907_1 Bloglines-Images/0.1 (http://www.bloglines.com) Bloglines graphics crawler R http://www.bloglines.com id_a_f_071206_2 BlogMap (http://www.feedmap.net) FeedMap / BlogMap geo coding service C http://www.feedmap.net/BlogMap/ id_a_f_121 Blogpulse (info@blogpulse.com) Intelliseek's BlogPulse blog search R 64.158.138.xx http://www.blogpulse.com id_a_f_050208_2 BlogPulseLive (support@blogpulse.com) Intelliseek's BlogPulse blog search R 64.158.138.xx http://www.blogpulse.com id_a_f_161206_2 BlogSearch/1.x +http://www.icerocket.com/ IceRocket Web search robot R s. also BlogzIce ... http://www.icerocket.com id_a_f_020707_1 blogsearchbot-pumpkin-3 Art of Computing blog search project R 88.198.44.2xx http://artofcomputing.net/blog/?p=4 id_a_f_241205_1 BlogsNowBot, V 2.01 (+http://www.blogsnow.com/) BlogsNow realtime link tracker robot R http://www.blogsnow.com/ id_a_f_261107_1 BlogVibeBot-v1.1 (spider@blogvibe.nl) BlogVipe news and Blog crawler (Netherlands) R 212.61.21.xx http://www.blogvibe.nl id_a_f_122 blogWatcher_Spider/0.1 (http://www.lr.pi.titech.ac.jp/blogWatcher/) blogWatcher robot from Okumura Group Tokyo (131.112.182.xxx) R http://www.lr.pi.titech.ac.jp id_a_f_123 BlogzIce/1.0 (+http://icerocket.com; rhodes@icerocket.com) IceRocket Web search robot R s. also BlogSearch ... http://www.icerocket.com id_a_f_124 BlogzIce/1.0 +http://www.icerocket.com/ IceRocket Web search robot R s. also BlogSearch ... http://www.icerocket.com id_a_f_310108_1 BloobyBot Blooby search (beta) robot R 206.166.206.18x http://www.blooby.com/ id_a_f_270507_1 Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) Balihoo - Search Engine for Advertising Media R 204.228.230.xx http://balihoo.com/ id_a_f_125 bluefish 0.6 HTML editor Bluefish HTML-editor for Linux B http://bluefish.openoffice.nl/ id_a_f_126 BMCLIENT Part of ButtMan remote access tool seems to be a Trojan - see link http://www.glocksoft.com/trojan_list/ButtMan.htm id_a_f_120506_1 BMLAUNCHER Bookmark Express bookmark manager C Website is dead - was: http://www.bookmarkexpress.com/ id_a_f_127 Bobby/4.0.x RPT-HTTPClient/0.3-3E Bobby web accessibility desktop testing tool C http://bobby.watchfire.com/bobby/html/en/index.jsp id_a_f_128 boitho.com-dc/0.xx (http://www.boitho.com/dcbot.html) Boitho search (Norway) robot via 80.202.212.xx / 80.80.111.xx R http://www.boitho.com id_a_f_129 boitho.com-robot/1.x Boitho search (Norway) robot via 80.202.212.xx / 80.80.111.xx R http://www.boitho.com id_a_f_130 boitho.com-robot/1.x (http://www.boitho.com/bot.html) Boitho search (Norway) robot via 80.202.212.xx / 80.80.111.xx R http://www.boitho.com id_a_f_060806_1 Bookdog/x.x Bookdog - Mac bookmark manager C http://www.sheepsystems.com/bookdog/ id_a_f_131 Bookmark Buddy bookmark checker (http://www.bookmarkbuddy.net/) Bookmark Buddy - favorite bookmark manager C http://www.bookmarkbuddy.net id_a_f_132 Bookmark Renewal Check Agent [http://www.bookmark.ne.jp/] Favourites managing program C http://www.bookmark.ne.jp id_a_f_230807_1 Bookmark Renewal Check Agent [http://www.bookmark.ne.jp/] (Version 2.0beta) Favourites managing program C http://www.bookmark.ne.jp id_a_f_300408_1 BookmarkBase(2/;http://bookmarkbase.com) Bookmark Base bookmark manager C http://www.bookmarkbase.com/ id_a_f_133 Bot mailto:craftbot@yahoo.com cybercity.fr user robot / faked user agent ? id_a_f_160308_1 BPImageWalker/2.0 (www.bdbrandprotect.com) BD-Brandprotect copyright infringement crawler R 72.14.164.1xx http://www.bdbrandprotect.com/ http://www.bdbrandprotect.com/solutions_5.html id_a_f_134 BravoBrian bstop.bravobrian.it BravoBrian bSTOP parental control P R s. also BStop http://bstop.bravobrian.it/ id_a_f_135 BravoBrian SpiderEngine MarcoPolo Robot for BravoBrian bSTOP R s. also BStop http://bstop.bravobrian.it/ id_a_f_220508_1 BrightCrawler (http://www.brightcloud.com/brightcrawler.asp) BrightCloud web filtering for classifying websites P R http://www.brightcloud.com/brightcrawler.asp id_a_f_136 BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html) Webarchive Project Bruinbot crawler R http://webarchive.cs.ucla.edu id_a_f_137 BSDSeek/1.0 Inktomi (Hotbot-Lycos NBCi) robot R http://www.inktomi.com/ id_a_f_138 BStop.BravoBrian.it Agent Detector BravoBrian bSTOP parental control P R s. also BravoBrian .. http://bstop.bravobrian.it/ id_a_f_139 BTbot/0.x (+http://www.btbot.com/btbot.html) BitTorrent Search Engine btbot robot R http://www.btbot.com id_a_f_300408_2 BTWebClient/180B(9704) µTorrent BitTorrent Client D http://www.utorrent.com/ id_a_f_080407_2 BuildCMS crawler (http://www.buildcms.com/crawler) BuildCMS crawler - market monitoring project of BuildCMS R 194.24.253.xx http://www.buildcms.com/index.php http://www.buildcms.com/about_us/crawler id_a_f_051206_4 Bulkfeeds/r1752 (http://bulkfeeds.net/) Bulkfeeds: RSS directory link checking C 202.181.96.2xx http://bulkfeeds.net/ id_a_f_140 BullsEye BullsEye/Intelliseek robot R http://www.intelliseek.com/be/bullseye.htm id_a_f_141 bumblebee@relevare.com Relevare Portal software robot R http://www.relevare.com id_a_f_142 BunnySlippers Microsoft server information robot (see link) C http://www.webmasterworld.com/forum11/841.htm id_a_f_070606_1 BurstFindCrawler/1.1 (crawler.burstfind.com; http://crawler.burstfind.com; crawler@burstfind.com) BurstFind search crawler (64.34.172.xx) R http://www.burstfind.com/ id_a_f_143 Buscaplus Robi/1.0 (http://www.buscaplus.com/robi/) Buscaplus (Spain) robot R http://www.buscaplus.com id_a_f_050807_2 BW-C-2.0 Logitech Desktop Managers (LDM) Backweb (BW) update check D http://www.logitech.com/index.cfm/494/3041&cl=de,de id_a_f_140508_2 bwh3_user_agent Basic Web Hacking 3 fake user-agent from Hellbound Hackers challenges S http://www.hellboundhackers.org/forum/_basic_web_hacking_3-7-6960_0.html http://vuau.wordpress.com/2008/03/12/hbh-basic-web-3-switch-user-agent/ id_a_f_140508_3 Cabot/Nutch-0.9 (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com) Amfibi Search robot R 64.111.217.9x http://www.amfibi.com/ id_a_f_280508_1 Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com) Amfibi Search robot R 64.111.217.9x http://www.amfibi.com/ id_a_f_140506_4 CamelHttpStream/1.0 Evolution integrated mail solution Camel TCP stream class http://www.gnome.org/projects/evolution/ http://go-evolution.org/Camel.Stream id_a_f_144 Cancer Information and Support International; Some user agent id_a_f_110206_1 carleson/1.0 Cosmix project crawler (204.14.48.x / 38.113.234.xxx) R s. also - voyager/1.x - cfetch/1. http://www.cosmixcorp.com/ id_a_f_145 Carnegie_Mellon_University_Research_WebBOT-->PLEASE READ-->http://www.andrew.cmu.edu/~brgordon/webbot/index.html http://www.andrew.cmu.edu/~brgordon/webbot/index.html Carnegie Mellon University WebBOT R http://www.andrew.cmu.edu id_a_f_146 Carnegie_Mellon_University_WebCrawler http://www.andrew.cmu.edu/~brgordon/webbot/index.html Carnegie Mellon University WebBOT R http://www.andrew.cmu.edu id_a_f_170206_1 Catall Spider Catall.de search & web directory (Germany) R http://www.catall.de/ id_a_f_130807_3 CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com) UIUCs Cazoodle search based on MetaQuerier R 72.36.94.1xx http://www.cazoodle.com/ http://metaquerier.cs.uiuc.edu/ id_a_f_290308_1 CCBot/1.0 (+http://www.commoncrawl.org/bot.html) CommonCrawl Foundation search crawler R 38.103.63.1[6-8] http://www.commoncrawl.org/faq.htm id_a_f_147 ccubee/x.x Empyreum Ccubee (Czech) search engine solution R http://empyreum.com/technologies/ccubee id_a_f_010108_2 CDR/1.7.1 Simulator/0.7(+http://timewe.net) Profile/MIDP-1.0 Configuration/CLDC-1.0 Timewe mobile browser (WAP) simulator (Japan) B 61.142.xx.x[xx] http://timewe.net/ id_a_f_291105_6 CE-Preload Cisco Content Engine P http://www.cisco.com/en/US/products/hw/contnetw/index.html id_a_f_100408_4 CentiverseBot Nordic semantic search engine C 87.72.214.9x http://www.centiverse-project.net/post/Educating-the-bots.aspx id_a_f_180408_2 CentiverseBot - investigator Nordic semantic search engine C 87.72.214.9x http://www.centiverse-project.net/post/Educating-the-bots.aspx id_a_f_180408_3 CentiverseBot/3.0 (http://www.centiverse-project.net) Nordic semantic search engine C 87.72.214.9x http://www.centiverse-project.net/post/Educating-the-bots.aspx id_a_f_148 Ceramic Tile Installation Guide (http://www.floorstransformed.com) Floortransformed.com robot (link ckecking ??) R http://www.floorstransformed.com id_a_f_149 cfetch/1.0 Cosmix project crawler (204.14.48.x / 38.113.234.xxx) R s. also - voyager/1.x - carleson/1.x http://www.cosmixcorp.com id_a_f_150 CFNetwork/x.x MaxOS X CoreFoundation CFNetwork API http://www.cocoadev.com/index.pl?CFNetwork id_a_f_151 cg-eye interactive cg-eye CGI checker C http://www.htmlhelp.com id_a_f_152 Charon/1.x (Amiga) Charon Amiga download manager D http://tesla.rcub.bg.ac.yu/%7Eantony/Charon/ id_a_f_153 Checkbot/1.xx LWP/5.xx Checkbot link validation C http://degraaff.org/checkbot/ id_a_f_154 CheckLinks/1.x.x Checklinks - Perl link checker C http://www.jmarshall.com id_a_f_155 CheckUrl NTL user agent http://www.ntl.com id_a_f_156 CheckWeb CheckWeb link validation C http://p.duby.free.fr/chkweb.htm id_a_f_030206_2 Chilkat/1.0.0 (+http://www.chilkatsoft.com/ChilkatHttpUA.asp) Chilkat HTTP component user-agent http://www.chilkatsoft.com/HttpDotNet.asp id_a_f_230506_1 China Local Browse 2.6 Unknown spam bot from telekom.com.my (218.111.83.xxx) S see link: http://linuxreviews.org/webdesign/602_Apache_Webalizer/ id_a_f_170408_2 Chitika ContentHit 1.0 Chitika Inc. Blog advertising C 67.15.219.[x]x http://chitika.com/ id_a_f_157 ChristCRAWLER 2.0 Christcentral.com Christcrawler (was www.christcrawler.com) R - s. also Mozilla/4.0 (compatible; ChristCrawler..) id_a_f_010607_2 CHttpClient by Open Text Corporation CHttpClient - C++ class using WinInet http://www.codeproject.com/library/lyoulhttpclient.asp id_a_f_158 CipinetBot (http://www.cipinet.com/bot.html) CipinetBot -Cipinet Search Engine Web Crawler R http://www.cipinet.com id_a_f_130108_1 Cityreview Robot (+http://www.cityreview.org/crawler/) Cityreview regional search (Germany) link checking C 88.198.212.5x http://www.cityreview.de/ http://www.cityreview.org/crawler/ id_a_f_159 CJ Spider/ Commision Junction link checking spider C http://www.cj.com/ id_a_f_050606_1 CJB.NET Proxy CJB Net anonymous socks proxy service (216.194.70.x) P http://proxy.cjb.net/ id_a_f_160 ClariaBot/1.0 Claria (ex Gator) SearchScout robot (64.152.73.xx) R s. also Diamond http://www.searchscout.com id_a_f_161 Claymont.com Claymont Search robot R http://www.claymont.com id_a_f_100307_1 CloakDetect/0.9 (+http://fulltext.seznam.cz/) Seznam Search (Czech Republic) robot R 212.80.76.xx http://www.seznam.cz/ id_a_f_162 Clushbot/2.x (+http://www.clush.com/bot.html) Clush clustered search robot R http://www.clush.com id_a_f_163 Clushbot/3.x-BinaryFury (+http://www.clush.com/bot.html) Clush clustered search robot R http://www.clush.com id_a_f_164 Clushbot/3.xx-Ajax (+http://www.clush.com/bot.html) Clush clustered search robot R http://www.clush.com id_a_f_165 Clushbot/3.xx-Hector (+http://www.clush.com/bot.html) Clush clustered search robot R http://www.clush.com id_a_f_166 Clushbot/3.xx-Peleus (+http://www.clush.com/bot.html) Clush clustered search robot R http://www.clush.com id_a_f_070506_1 COAST WebMaster Pro/4.x.x.xx (Windows NT) COAST Webmaster - Web management and maintenance software C http://www.extablish.com/cwm.htm id_a_f_150306_1 CoBITSProbe Proposed Content-Based Image Tracking System (CoBITS) P2P crawler R C http://www.iis.sinica.edu.tw/~hungchi/CBIT/ id_a_f_167 Cocoal.icio.us/1.0 (v36) (Mac OS X; http://www.scifihifi.com/cocoalicious) Cocoa del.icio.us (social bookmarks manager) client for Mac OS X C http://www.scifihifi.com id_a_f_168 ColdFusion Cold Fusion server used by various IPs i.e.: - NetWORLD web catalogue link checking P C http://www.networld.com http://www.macromedia.com/software/coldfusion/ id_a_f_169 ColdFusion (BookmarkTracker.com) Cold Fusion server used by Bookmark Tracker - online favourites managment C http://www.bookmarktracker.com http://www.macromedia.com/software/coldfusion/ id_a_f_170 collage.cgi/1.xx WebCollage Syndicator graphics crawler/collector R D http://www.webcollage.com id_a_f_171 combine/0.0 Combine harvesting & indexing robot R 130.235.4.xx http://combine.it.lth.se/ id_a_f_080406_1 Combine/2.0 http://combine.it.lth.se/ Combine harvesting & indexing robot R 130.235.4.xx http://combine.it.lth.se/ id_a_f_160807_1 Combine/3 http://combine.it.lth.se/ Combine harvesting & indexing robot R 130.235.4.xx http://combine.it.lth.se/ id_a_f_172 Combine/x.0 Combine harvesting & indexing robot R 130.235.4.xx http://combine.it.lth.se/ id_a_f_280606_3 cometrics-bot, http://www.cometrics.de cometrics Web Content Mining solution - Germany R http://www.cometrics.de/ id_a_f_173 Commerce Browser Center Wildsoft Germany (closed) Internet client system user-agent (was www.oskarweb.de) B id_a_f_240107_1 complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~networks/crawl; nimakhaj@gmail.com) UCLA Complex Networks Groups Complex Network Analysis C 216.182.233.1xx http://cantor.ee.ucla.edu/~networks/ id_a_f_174 Computer_and_Automation_Research_Institute_Crawler crawler@ilab.sztaki.hu Hungarian Academy of Sciences data mining search R http://www.ilab.sztaki.hu/websearch/ id_a_f_010406_2 Comrite/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) ComRite Chinese Search Engine for Oversea Web Sites (69.248.26.xx) R powered by Nutch http://www.comrite.com/ http://meidong.comrite.com/ id_a_f_175 Contact unknown id_a_f_200307_1 ContactBot/0.2 Probably E-Mail harvesting robot - same as LMQueueBot S 64.124.152.xx id_a_f_291105_7 ContentSmartz ContentSmartz e-mail harvesting tools S id_a_f_176 contype Internet Explorer versions 4.x and 5 plugin content B NOT Contype mime type managment Perl script http://support.microsoft.com/default.aspx?scid=kb;en-us;293792 id_a_f_177 Convera Internet Spider V6.x Converas RetrievalWare Internet Spider (63.241.61.x) R s. also - infoConveraCrawler... - CrawlConvera ... http://www.convera.com id_a_f_178 ConveraCrawler/0.2 Converas RetrievalWare Internet Spider (63.241.61.x) R s. also - infoConveraCrawler... - CrawlConvera ... http://www.convera.com id_a_f_051205_1 ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl) Converas RetrievalWare Internet Spider (63.241.61.x) R s. also - infoConveraCrawler... - CrawlConvera ... http://www.convera.com/ id_a_f_179 ConveraMultiMediaCrawler/0.1 (+http://www.authoritativeweb.com/crawl) Converas RetrievalWare Internet Spider (63.241.61.x) R s. also - infoConveraCrawler... - CrawlConvera ... http://www.convera.com id_a_f_180 CoolBot Suchmaschine21 (Germany) robot R http://www.suchmaschine21.de/ id_a_f_040406_1 CoralWebPrx/0.1.1x (See http://coralcdn.org/) Coral Content Distribution Network P C http://coralcdn.org/ id_a_f_181 cosmos/0.8_(robot@xyleme.com) Xyleme SA France robot R http://www.xyleme.com/en/index.jsp id_a_f_182 cosmos/0.9_(robot@xyleme.com) Xyleme SA France robot R http://www.xyleme.com/en/index.jsp id_a_f_040607_1 CoteoNutchCrawler/Nutch-0.9 (info [at] coteo [dot] com) Coteo.com - local French directory link checking C http://www.coteo.com/ id_a_f_030206_3 CougarSearch/0.x (+http://www.cougarsearch.com/faq.shtml) Cougarsearch.com robot R http://www.cougarsearch.com/ id_a_f_280408_2 Covac TexAs Arachbot Covac Arachnid Web Crawler R s.also ArachBot http://www.covac-software.com/ id_a_f_183 Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) Naver Japan / Korea robot R s. also Python-urllib/1.15 - nabot - NaverBot & dloader http://www.naver.co.jp id_a_f_184 Cowbot-0.1.x (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) Naver Japan / Korea robot R s. also Python-urllib/1.15 - nabot - NaverBot & dloader http://www.naver.co.jp id_a_f_185 CrawlConvera0.1 (CrawlConvera@yahoo.com) Converas RetrievalWare Internet Spider R s. also - Convera Internet Spider .. - infoConveraCrawler... http://www.convera.com id_a_f_186 Crawler unknown robot via Level3.net id_a_f_187 Crawler (cometsearch@cometsystems.com) Cometsystems Comet Search robot via Findwhat (now Miva) R http://www.cometsystems.com http://www.miva.com/ id_a_f_188 Crawler admin@crawler.de Crawler.de / Abacho robot R http://www.crawler.de http://www.abacho.de/ id_a_f_189 Crawler V 0.2.x admin@crawler.de Crawler.de / Abacho robot R http://www.crawler.de http://www.abacho.de/ id_a_f_190 crawler@alexa.com Alexa crawler R http://www.alexa.com id_a_f_191 CrawlerBoy Pinpoint.com Pinpoint WAP search robot R http://www.pinpoint.com id_a_f_130506_1 Crawllybot/0.1 (Crawllybot; +http://www.crawlly.com; crawler@crawlly.com) Crawlly Beta search - Germany (72.232.194.2xx) R powered by Nutch http://www.crawlly.com/ id_a_f_192 CreativeCommons/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) Creatice Commons using Nutch open source robot R http://www.nutch.org id_a_f_193 CrocCrawler vx.3 [en] (http://www.croccrawler.com) (X11; I; Linux 2.0.44 i686) Croccrawler robot R http://www.croccrawler.com id_a_f_090306_2 csci_b659/0.13 Web mining project from CSCI 659 (computer science course) at Indiana Univerity R http://informatics.indiana.edu/fil/Class/b659/ id_a_f_180906_1 CSE HTML Validator Professional (http://www.htmlvalidator.com/) CSE HTML Validator for Windows C http://www.htmlvalidator.com/ id_a_f_194 Cuam Ver0.050bx Cuam - IE based browser B site is dead http://cuam.virtualave.net id_a_f_195 Cuasarbot/0.9b http://www.cuasar.com/spider_beta/ Cuasar (Spain) music / ringtone search spider R http://www.cuasar.com id_a_f_196 curl/7.10.x (i386-redhat-linux-gnu) libcurl/7.10.x OpenSSL/0.9.7a ipv6 zlib/1.1.4 Curl file transferring tool D http://curl.haxx.se/ id_a_f_197 curl/7.7.x (i386--freebsd4.3) libcurl 7.7.x (SSL 0.9.6) (ipv6 enabled) Curl file transferring tool D http://curl.haxx.se/ id_a_f_198 curl/7.8 (i686-pc-linux-gnu) libcurl 7.8 (OpenSSL 0.9.6) Curl file transferring tool D http://curl.haxx.se/ id_a_f_199 curl/7.9.x (win32) libcurl 7.9.x Curl file transferring tool D http://curl.haxx.se/ id_a_f_200 CurryGuide SiteScan 1.1 CurryGuide UK link check robot R http://uk.curryguide.com/ id_a_f_201 Custo x.x (www.netwu.com) Custo web site spidering tool (link checking) C s. also - - Mozilla/5.0 (compatible; Custo 3... http://www.netwu.com id_a_f_202 Custom Spider www.bisnisseek.com /1.0 Bisnisseek (was www.bisnisseek.com) robot R id_a_f_203 Cyberdog/2.0 (Macintosh; 68k) Cyberdog Mac Browser (was www.cyberdog.org) B id_a_f_260608_2 CyberPatrol SiteCat Webbot (http://www.cyberpatrol.com/cyberpatrolcrawler.asp) CyberPatrol LLC robot for web filtering software R 38.103.17.16x http://www.cyberpatrol.com/cyberpatrolcrawler.asp id_a_f_204 CyberSpyder Link Test/2.1.12 (admin@mspennyworth.com) CyberSpyder Link Test software C http://www.cyberspyder.com/cslnkts1.html id_a_f_205 CydralSpider/1.x (Cydral Web Image Search; http://www.cydral.com) Cydral image & site search spider R http://www.cydral.com id_a_f_060508_1 CydralSpider/3.0 (Cydral Image Search; http://www.cydral.com) Cydral image & site search spider R http://www.cydral.com id_a_f_206 DA 3.5 (www.lidan.com) Downloadaccelerator download manager D http://www.downloadaccelerator.com id_a_f_207 DA 4.0 Downloadaccelerator download manager D http://www.downloadaccelerator.com id_a_f_208 DA 4.0 (www.downloadaccelerator.com) Downloadaccelerator download manager D http://www.downloadaccelerator.com id_a_f_209 DA 5.0 Downloadaccelerator download manager D http://www.downloadaccelerator.com id_a_f_210 DA 7.0 Downloadaccelerator download manager D http://www.downloadaccelerator.com id_a_f_211 Dart Communications PowerTCP PowerTCP ActiveX control tool http://www.dart.com id_a_f_212 DataCha0s/2.0 Unknown bot from Kornet Korea (218.149.129.xxx) scans for Perl Awstats S id_a_f_101106_1 DataCha0s/2.0 Unknown UA looking for Awstats Perl components S from various IPs id_a_f_213 DataFountains/DMOZ Downloader INFOMINE/iVia Scholary Internet Resource Collections robot R 138.23.85.xx http://infomine.ucr.edu/ http://ivia.ucr.edu/useragents.shtml id_a_f_040307_1 DataFountains/Dmoz Downloader (http://ivia.ucr.edu/useragents.shtml) INFOMINE/iVia Scholary Internet Resource Collections robot R 138.23.85.xx http://infomine.ucr.edu/ http://ivia.ucr.edu/useragents.shtml id_a_f_110307_1 DataFountains/DMOZ Feature Vector Corpus Creator (http://ivia.ucr.edu/useragents.shtml) INFOMINE/iVia Scholary Internet Resource Collections robot R 138.23.85.xx http://infomine.ucr.edu/ http://ivia.ucr.edu/useragents.shtml id_a_f_050907_1 DataparkSearch/4.47 (+http://dataparksearch.org/bot) DataparkSearch open source search engine R http://www.dataparksearch.org id_a_f_214 DataparkSearch/4.xx (http://www.dataparksearch.org/) DataparkSearch open source search engine R http://www.dataparksearch.org id_a_f_080206_1 DataSpear/1.0 (Spider; http://www.dataspear.com/spider.html; spider@dataspear.com) DataSpear Directory robot (24.109.29.xx) R http://www.dataspear.com/ id_a_f_215 DataSpearSpiderBot/0.2 (DataSpear Spider Bot; http://dssb.dataspear.com/bot.html; dssb@dataspear.com) DataSpear Directory robot (24.109.29.xx) R http://www.dataspear.com/ id_a_f_161105_1 DatenBot( http://www.sicher-durchs-netz.de/bot.html) Sicher-durchs-Netz German security related search (62.75.220.xxx) R http://www.sicher-durchs-netz.de/suche id_a_f_216 DaviesBot/1.7 (www.wholeweb.net) Wholeweb robot R site is closed http://www.wholeweb.net id_a_f_217 daypopbot/0.x Daypop blog - weblog - online mag search spider R http://www.daypop.com/info/about.htm id_a_f_218 dbDig(http://www.prairielandconsulting.com) dbDig search engine R http://www.prairielandconsulting.com id_a_f_219 DBrowse 1.4b Some site scanning tool via diff. IPs i.e.: - wanweb.net (208.6.163.xxx) - cox.net (68.4.xxx.xxx) S UA sometimes - DSurf15a 01 - DBrowse 1.4d id_a_f_220 DBrowse 1.4d Some site scanning tool via diff. IPs i.e.: - pacbell.net (67.112.xxx.xxx) S see also DSurf15a 01 - DBrowse 1.4b id_a_f_221 DC-Sakura/x.xx DC-Sakura download manager D http://www.dc-sakura.com/ id_a_f_222 dCSbot/1.1 unknown divine/Openmarket.com robot R http://www.openmarket.com/ id_a_f_223 DDD some (website) downloading tool D id_a_f_224 dds explorer v1.0 beta Unknown user agent id_a_f_225 de.searchengine.comBot 1.2 (http://de.searchengine.com/spider) Searchengine.com (Germany) submission checking / robot (84.73.57.xx) R http://de.searchengine.com id_a_f_226 DeadLinkCheck/0.4.0 libwww-perl/5.xx DLC Perl HTTP link checking C http://dlc.sourceforge.net/ id_a_f_111206_3 Deep Link Calculator v1.0 Sootle Web Directory deep link checker C 216.89.111.x http://directory.sootle.com/deep-links/ id_a_f_227 deepak-USC/ISI deepak-USC/ISI robot from USC/Information Science Institute R http://www.isi.edu/%7Eravichan/deepak-usc-isi.html http://www.isi.edu/ id_a_f_228 DeepIndex Deepindex robot R http://www.deepindex.net/utilisateurs.php?referral=deepindex id_a_f_229 DeepIndex ( http://www.zetbot.com ) Zetbot search Belgium (213.41.128.xx) using Deepindex robot R http://www.zetbot.com/ http://www.deepindex.net/utilisateurs.php?referral=deepindex id_a_f_230 DeepIndex (www.en.deepindex.com) Deepindex robot R http://www.deepindex.net/utilisateurs.php?referral=deepindex id_a_f_231 DeepIndexer.ca Deepindex robot (via Paragon.net Canada) R http://www.deepindex.net/utilisateurs.php?referral=deepindex id_a_f_011006_2 del.icio.us-thumbnails/1.0 Mozilla/5.0 (compatible; Konqueror/3.4; FreeBSD) KHTML/3.4.2 (like Gecko) del.icio.us picture robot for thumbnail preview via Yahoo D 66.94.237.1xx http://del.icio.us/ id_a_f_250106_1 DeleGate/9.0.5-fix1 DeleGate application level gateway / proxy server P http://www.delegate.org/delegate/ id_a_f_232 Demo Bot DOT 16b Some site scanning tool from 217.34.59.xxx (btopenworld.com) S id_a_f_233 Demo Bot Z 16b Some site scanning tool from 68.154.96.xx (bellsouth.net) S appears also as - MFC Foundation Class Library - Full Web Bot 0516B id_a_f_234 Denmex websearch (http://search.denmex.com) Denmex Websearch robot/link checking R http://search.denmex.com id_a_f_235 Der große BilderSauger 2.00u DataBecker Bilder Sauger (discontinued) web graphics downloader D http://www.databecker.de id_a_f_230206_1 dev-spider2.searchpsider.com/1.3b Searchspider.com robot (72.245.225.xxx) R http://www.searchspider.com/ id_a_f_236 DevComponents.com HtmlDocument Object DevComponents HTMLDocument Class Library for Visual Studio.net http://www.devcomponents.com id_a_f_237 DiaGem/1.1 (http://www.skyrocket.gr.jp/diagem.html) DiaGem Japan web crawler R (site is offline) http://www.skyrocket.gr.jp id_a_f_238 Diamond/x.0 Claria (ex Gator) SearchScout robot (64.152.73.xx) R s. also Claria http://www.searchscout.com id_a_f_239 DiamondBot Claria (ex Gator) SearchScout robot (64.152.73.xx) R s. also Claria http://www.searchscout.com id_a_f_240 Digger/1.0 JDK/1.3.0rc3 Diggit! robot R http://www.diggit.com id_a_f_241 DigOut4U OpenPortal4U robot R http://www.arisem.com id_a_f_242 DIIbot/1.2 Findsame.com (site is offline) / Digital-Integrity robot R http://www.digital-integrity.com/ id_a_f_100206_1 Dillo/0.8.5-i18n-misc Dillo Web Browser B http://www.dillo.org/ id_a_f_243 Dillo/0.x.x Dillo Web Browser B http://www.dillo.org/ id_a_f_140306_1 DISCo Pump x.x DISCo Pump offline browser / website ripper D No active homepage http://www.arssoft.com/ http://www.filetransit.com/view.php?id=3870 id_a_f_300907_1 disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com) Unkown robot from Discovery Engine Corp. R 208.96.54.xx http://www.discoveryengine.com/ id_a_f_300907_2 disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com) Unkown robot from Discovery Engine Corp. R 208.96.54.xx http://www.discoveryengine.com/ id_a_f_244 DittoSpyder Ditto picture search robot R http://www.ditto.com id_a_f_245 dlman some download agent D Wildsoft Surfer id_a_f_246 dloader(NaverRobot)/1.0 Naver Japan / Korea robot R s. also Python-urllib/1.15 - nabot - NaverBot & Cowbot http://www.naver.co.jp id_a_f_100408_1 DNSRight.com WebBot Link Ckeck Tool. Report abuse to: dnsr@dnsright.com DNS Right - Online DNS tools C 203.161.71.17x http://www.dnsright.com/ id_a_f_247 DoCoMo/1.0/Nxxxi/c10 NTT DoCoMo (Japan) robot R http://www.nttdocomo.co.jp/ id_a_f_248 DoCoMo/1.0/Nxxxi/c10/TB NTT DoCoMo (Japan) robot R http://www.nttdocomo.co.jp/ id_a_f_249 DoCoMo/1.0/P502i/c10 (Google CHTML Proxy/1.0) Google (216.239.39.x) proxy server P id_a_f_250 DoCoMo/2.0 P900iV(c100;TB;W24H11) NTT DoCoMo (Japan) robot R http://www.nttdocomo.co.jp/ id_a_f_141205_1 DoCoMo/2.0 SH901iS(c100;TB;W24H12),gzip(gfe) (via translate.google.com) NTT DoCoMo (Japan) proxy server (210.136.161.1xx) P http://www.nttdocomo.co.jp/ id_a_f_081207_1 DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html) Yahoo Search Japan robot (203.216.197.xxx) R s. also Y!J-SRD/1.0 http://www.yahoo.co.jp/ id_a_f_271006_3 DoCoMo/2.0/SO502i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html) Yahoo Search Japan robot (203.216.197.xxx) R s. also Y!J-SRD/1.0 http://www.yahoo.co.jp/ id_a_f_251 DocZilla/1.0 (Windows; U; WinNT4.0; en-US; rv:1.0.0) Gecko/20020804 DocZilla - Mozilla-based SGML/XML/HTML- browser B http://www.doczilla.com id_a_f_210607_1 dodgebot/experimental unknown robot from AGMLAB Information Technologies (Information retrieval system ?) R 212.174.130.1xx http://www.agmlab.com/agmlab_eng.html id_a_f_180606_1 DonutP; Windows98SE Donut P - Japanese IE based browser B http://donutp.com/ id_a_f_250907_2 Doubanbot/1.0 (bot@douban.com http://www.douban.com) Unknown robot from douban search (China) - maybe image crawling 59.151.41.xx http://www.douban.com/ id_a_f_252 Download Demon/3.x.x.x Download Demon/Netzip download manager D http://www.netzip.com id_a_f_191105_4 Download Druid 2.x Download Druid IE plugin download manager D http://www.xemico.com/druid/index.html id_a_f_253 Download Express 1.0 Download Express download manager D http://www.metaproducts.com id_a_f_254 Download Master Download Master download manager D http://www.westbyte.com/dm/index.phtml id_a_f_191105_5 Download Ninja 3.0 Download Ninja download manager (Japan) D http://www.h-fd.org/mkro/mt/archives/2002/09/download_ninja_1.html id_a_f_255 Download Wonder Download Wonder download manager D http://www.forty.com id_a_f_256 Download-Tipp Linkcheck (http://download-tipp.de/) Download-Tipp Germany link checking R http://download-tipp.de/ id_a_f_257 Download.exe(1.1) (+http://www.sql-und-xml.de/freeware-tools/) download.exe .NET based downloading tool D http://www.sql-und-xml.de/freeware-tools/ id_a_f_300907_3 DownloadDirect.1.0 Download Direct download manager D http://senbit.com/ id_a_f_258 Dr.Web (R) online scanner: http://online.drweb.com/ Dr.WEB online virus scanner http://online.drweb.com id_a_f_259 Dragonfly File Reader Dragonfly CMS - Open Source content management system B http://www.cpgnuke.com id_a_f_260 Drecombot/1.0 (http://career.drecom.jp/bot.html) Drecom Japan (210.233.67.xxx) - Blog search ?? R http://www.drecom.co.jp id_a_f_3090906_1 Drupal (+http://drupal.org/) Drupal - open source content management platform C http://drupal.org/ id_a_f_261 DSurf15a 01 Some site scanning tool via diff. IPs i.e.: - cox.net (68.5.xxx.xxx) - pacbell.net (64.16x.xxx.xxx) S DBrowse 1.4d id_a_f_262 DSurf15a 71 Some site scanning tool via diff. IPs i.e.: - cox.net (68.4.xxx.xxx) S TRPMFHXE id_a_f_263 DSurf15a 81 Some site scanning tool via diff. IPs i.e.: - verizon.net (4.47.xxx.xxx) S WFRIKXVNFL id_a_f_264 DSurf15a VA Some site scanning tool via diff. IPs i.e.: - eastlink.ca (24.222.xxx.xxx) - cogeco.net (216.221.8x.xxx) S id_a_f_020506_2 DTAAgent DTAAgent Java object for data collecting R D http://www.ibr.cs.tu-bs.de/courses/ss00/sep-vs/gruppe1/jdoc/DTA/agent/DTAAgent.html id_a_f_265 dtSearchSpider dt Search Spider software R http://www.dtsearch.com/spider.html id_a_f_266 Dual Proxy Fourelle Venturi proxy server P http://www.fourelle.com/news/articles/148457.htm id_a_f_290308_2 DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html) Duck Duck Go search crawler R 72.94.249.34 http://duckduckgo.com/ http://duckduckgo.com/duckduckbot.html id_a_f_267 Dumbot(version 0.1 beta - dumbfind.com) DumbFind.com robot R http://www.dumbfind.com id_a_f_268 Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html) DumbFind.com robot R http://www.dumbfind.com id_a_f_269 Dumbot(version 0.1 beta) DumbFind.com robot R http://www.dumbfind.com id_a_f_270 e-sense 1.0 ea(www.vigiltech.com/esensedisclaim.html) Vigiltech e-Sense user research robot (website is offline) R id_a_f_271 e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) e-Society Project (Japan) crawler (133.9.238.xx) R http://www.yama.info.waseda.ac.jp id_a_f_280108_1 eApolloBot/2.0 (compatible; heritrix/2.0.0-SNAPSHOT-20071024.170148 +http://www.eapollo-opto.com) Global Opto's eApollo Flash based search engine - Taiwan R http://www.eapollo.com/eaSE.jsp?language=eng http://www.global-opto.com/ id_a_f_272 EARTHCOM.info/1.x [www.earthcom.info] Earthcom (Czech Republic) search robot (194.108.39.xx) R s. also Mozilla/5.0 (compatible; EARTHCOM.info/2.01 ... http://www.earthcom.info id_a_f_273 EARTHCOM.info/1.xbeta [www.earthcom.info] Earthcom (Czech Republic) search robot (194.108.39.xx) R s. also Mozilla/5.0 (compatible; EARTHCOM.info/2.01 ... http://www.earthcom.info id_a_f_274 EasyDL/3.xx Keywen Encyclopedia Bot R http://keywen.com/Encyclopedia/Bot/ id_a_f_275 EasyDL/3.xx http://keywen.com/Encyclopedia/Bot Keywen Encyclopedia Bot R http://keywen.com/Encyclopedia/Bot/ id_a_f_276 EBrowse 1.4b Some site scanning tool via diff. IPs i.e.: - swbell.net (65.66.xxx.xxx) S id_a_f_277 eCatch/3.0 eCatch (now Wysigot) offline browser B D http://www.ecatch.com id_a_f_278 EchO!/2.0 Echo.fr robot R http://echo.fr/ id_a_f_279 Educate Search VxB Some site scanning tool via diff. IPs i.e.: - cox.net (68.4.xxx.xxx) S s. also DSurf - 66.118.1xx.xxx (sagonet.com) - see also Full Web Bot - Industry Program 1.0.5 id_a_f_280 egothor/3.0a (+http://www.xdefine.org/robot.html) Xdefine text search engine robot - based on Egothor open source crawler R http://www.xdefine.com/ http://www.egothor.org/ id_a_f_281 EgotoBot/4.8 (+http://www.egoto.com/about.htm) Egoto Search robot R http://www.egoto.com id_a_f_211105_1 ejupiter.com eJupiter searcg robot (206.191.49.xx) R http://search.ejupiter.com/ id_a_f_282 EldoS TimelyWeb/3.x TimelyWeb web page monitoring tool C s. also TimelyWeb/... http://www.eldos.org/timelyweb/timelyweb.html id_a_f_283 elfbot/1.0 (+http://www.uchoose.de/crawler/elfbot/) Elftales crawler for uChoose theme based search (Germany) R http://www.uchoose.de id_a_f_300707_1 ELI/20070402:2.0 (DAUM RSS Robot, Daum Communications Corp.; +http://ws.daum.net/aboutkr.html) DAUMOA - RSS search robot of Daum R 211.115.109.1xx http://ws.daum.net/abouten.html id_a_f_284 ELinks (0.x.x; Linux 2.4.20 i586; 132x60) ELinks text mode browser B http://elinks.or.cz/ id_a_f_285 ELinks/0.x.x (textmode; NetBSD 1.6.2 sparc; 132x43) ELinks text mode browser B http://elinks.or.cz/ id_a_f_286 EmailSiphon Sonic E-mail collector S http://www.americaint.com/superstore/elist.html id_a_f_220508_2 EmailSpider EmailSpider E-mail harvesting software S http://www.emailspider.net/index.php?kat=11 id_a_f_287 EmailWolf 1.00 Trellian EMailWolf E-mail collector S http://www.trellian.com id_a_f_288 EmeraldShield.com WebBot EmeraldShield spam and web filtration services P R http://www.emeraldshield.com id_a_f_289 EmeraldShield.com WebBot (http://www.emeraldshield.com/webbot.aspx) EmeraldShield spam and web filtration services P R http://www.emeraldshield.com id_a_f_070606_2 EMPAS_ROBOT Empas search Korea robot (220.95.22x.xxx) R http://www.empas.com/ id_a_f_280208_1 EnaBot/1.x (http://www.enaball.com/crawler.html) Enabot - Enaball semantic search project crawler R http://www.enaball.com/crawler.html id_a_f_110506_4 endo/1.0 (Mac OS X; ppc i386; http://kula.jp/endo) endo - Mac news site and blog aggregator B http://kula.jp/software/endo/ id_a_f_290 Enfish Tracker Enfish Personal search tool R http://www.enfish.com id_a_f_291 Enterprise_Search/1.0 Enterprise Search engine software (64.202.165.xxx) R s. also - ES.NET_Crawler - InnerpriseBot http://www.innerprise.net id_a_f_292 Enterprise_Search/1.0.xxx Enterprise Search engine software (64.202.165.xxx) R s. also - ES.NET_Crawler - InnerpriseBot http://www.innerprise.net id_a_f_293 Enterprise_Search/1.00.xxx;MSSQL (http://www.innerprise.net/es-spider.asp) Enterprise Search engine software (64.202.165.xxx) R s. also - ES.NET_Crawler - InnerpriseBot http://www.innerprise.net id_a_f_230706_1 envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.php) Envolk Web Search robot R 70.169.191.x http://www.envolk.com/ id_a_f_294 envolk[ITS]spider/1.6(+http://www.envolk.com/envolkspider.html) Envolk Web Search robot R 70.169.191.x http://www.envolk.com/ id_a_f_295 EroCrawler EroCrawler adult search robot R http://www.erocrawler.com id_a_f_296 ES.NET_Crawler/2.0 (http://search.innerprise.net/) Enterprise Search engine software (64.202.165.xxx) R s. also - Enterprise_Search - InnerpriseBot http://search.innerprise.net id_a_f_297 eseek-larbin_2.6.2 (crawler@exactseek.com) ExactSEEK (Jayde Online) robot R see also ExactSeek Crawler / eseek-larbin / exactseek.com http://www.exactseek.com id_a_f_298 ESISmartSpider ESI Smart-Spider toolkit R http://www.smart-spider.com/ id_a_f_299 eStyleSearch 4 (compatible; MSIE 6.0; Windows NT 5.0) e-Style ISP search (Russia) robot (217.174.103.xxx) R http://www.e-styleisp.ru/ id_a_f_300 ESurf15a 15 Some site scanning tool via diff. IPs S s. also DSurf - PBrowse ... id_a_f_301 EuripBot/0.x (+http://www.eurip.com) GetFile Eurip.com - European Internet Portal robot R http://www.eurip.com id_a_f_302 EuripBot/0.x (+http://www.eurip.com) GetRobots Eurip.com - European Internet Portal robot R http://www.eurip.com id_a_f_303 EuripBot/0.x (+http://www.eurip.com) PreCheck Eurip.com - European Internet Portal robot R http://www.eurip.com id_a_f_220508_3 Eurobot/1.0 (http://www.ayell.eu) Ayell Euronet business directory robot R http://www.ayell.eu/ id_a_f_030206_4 EvaalSE - bot@evaal.com Evaal Search Engine robot R based on Nutch http://www.evaal.com/ http://search.evaal.com/en/about.html id_a_f_304 eventax/1.3 (eventax; http://www.eventax.de/; info@eventax.de) Eventax event search (Germany) R http://www.eventax.de id_a_f_291205_1 Everest-Vulcan Inc./0.1 (R&D project; host=e-1-24; http://everest.vulcan.com/crawlerhelp) Vulcan Inc. Everest crawler (in development) R http://www.vulcan.com/ id_a_f_305 Everest-Vulcan Inc./0.1 (R&D project; http://everest.vulcan.com/crawlerhelp) Vulcan Inc. Everest crawler (in development) R http://www.vulcan.com/ id_a_f_050406_2 Exabot-Images/1.0 Exalead Websearch image crawler (193.47.80.xx) R Exabot-Images only requests robots.txt - image crawling under UA NG/4.0.1229 http://www.exalead.com/search id_a_f_211206_1 Exabot-Test/1.0 Exalead (France) search robot (193.47.80.xx) R s. Harvest-NG/1.0.2 and Exalead NG... http://www.exabot.com/ id_a_f_190106_1 Exabot/2.0 Exalead (France) search robot (193.47.80.xx) R s. Harvest-NG/1.0.2 and Exalead NG... http://www.exabot.com/ id_a_f_211206_2 Exabot/3.0 Exalead (France) search robot (193.47.80.xx) R s. Harvest-NG/1.0.2 and Exalead NG... http://www.exabot.com/ id_a_f_306 ExactSearch eXact Search Bar for IE B http://www.exactsearchbar.com/exact04 id_a_f_307 ExactSeek Crawler/0.1 ExactSEEK (Jayde Online) robot R see also eseek-larbin / exactseek.com http://www.exactseek.com id_a_f_308 exactseek-crawler-2.63 (crawler@exactseek.com) ExactSEEK (Jayde Online) robot R see also eseek-larbin / exactseek.com http://www.exactseek.com id_a_f_309 exactseek-pagereaper-2.63 (crawler@exactseek.com) ExactSEEK (Jayde Online) robot R see also eseek-larbin / exactseek.com http://www.exactseek.com id_a_f_050606_2 exactseek.com ExactSEEK (Jayde Online) robot (69.9.181.1xx) R see also eseek-larbin http://www.exactseek.com id_a_f_310 Exalead NG/MimeLive Client (convert/http/0.120) Exalead (France) search robot (193.47.80.xx) R s. Harvest-NG/1.0.2 and NG/1.0 http://www.exabot.com id_a_f_311 Excalibur Internet Spider V6.5.4 Excalibur (now Convera) spider software R http://www.excalib.com id_a_f_191206_1 Execrawl/1.0 (Execrawl; http://www.execrawl.com/; bot@execrawl.com) Execrawl software search using nutch R 72.36.179.1xx http://www.execrawl.com/ id_a_f_060608_1 exooba crawler/exooba crawler (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com) exooba crawler for exooba search pre-alpha development R 216.195.184.xx http://www.exooba.com/ id_a_f_060608_2 exooba/exooba crawler (exooba; exooba) exooba crawler for exooba search pre-alpha development R 216.195.184.xx http://www.exooba.com/ id_a_f_312 ExperimentalHenrytheMiragoRobot Mirago UK Robot R http://www.mirago.co.uk id_a_f_313 Expired Domain Sleuth Expired Domain Sleuth domain name tool R C http://expireddomainsleuth.com/ id_a_f_314 Express WebPictures (www.express-soft.com) Express Web Pictures image browser B D http://www.express-soft.com/ id_a_f_315 ExtractorPro Extractor Pro e-mail collector S http://www.extractorpro.com id_a_f_170106_1 Extreme Picture Finder Exisoftware image grabber and downloading tool D http://www.exisoftware.com/picture_finder/ id_a_f_316 EyeCatcher (Download-tipp.de)/1.0 Download-Tipp Germany robot R http://download-tipp.de id_a_f_091006_1 Factbot 1.09 (see http://www.factbites.com/webmasters.php) Factbites search robot R 70.86.159.1xx http://www.factbites.com/ id_a_f_220906_1 factbot : http://www.factbites.com/robots Factbites search robot R 70.86.159.1xx http://www.factbites.com/ id_a_f_221006_1 FaEdit/2.0.x FaEdit Professional - Japanese bookmark manager C https://sw.vector.co.jp/swreg/step1.reserve?srno=SR040433&site=y id_a_f_317 FairAd Client FairAd user P http://www.fairad.de id_a_f_051206_1 FANGCrawl/0.01 Safe-t.net web filtering service P 63.167.160.1xx http://www.safe-t.net/ id_a_f_210206_1 FARK.com link verifier Drew Curtis' FARK.com link checking C http://www.fark.com/ id_a_f_318 Fast Crawler Gold Edition Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_319 FAST Enterprise Crawler 6 (Experimental) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_040206_1 FAST Enterprise Crawler 6 / Scirus scirus-crawler@fast.no; http://www.scirus.com/srsapp/contactus/ Fast Enterprise Crawler (66.151.181.xx) for Scirus scienctific information search R http://www.scirus.com/srsapp/ http://www.alltheweb.com id_a_f_320 FAST Enterprise Crawler 6 used by Cobra Development (admin@fastsearch.com) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_070507_1 FAST Enterprise Crawler 6 used by Comperio AS (sts@comperio.no) Comperio Web Miner based on Fast ESP R http://www.comperio.no/index.php?option=com_content&task=blogcategory&id=7&Itemid=27 id_a_f_321 FAST Enterprise Crawler 6 used by FAST (FAST) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_161106_1 FAST Enterprise Crawler 6 used by Pages Jaunes (pvincent@pagesjaunes.fr) Pages Jaunes business search (France) robot using Fast Enterprise Crawler R 193.252.242.xx http://www.pagesjaunes.fr/ id_a_f_322 FAST Enterprise Crawler 6 used by Sensis.com.au Web Crawler (search_comments\at\sensis\dot\com\dot\au) Fast/Alltheweb crawler for Sensis.com.au Australian search (66.151.181.xx) R http://www.sensis.com.au/ http://www.alltheweb.com id_a_f_251007_2 FAST Enterprise Crawler 6 used by Singapore Press Holdings (crawler@sphsearch.sg) SPH Search - Singapore related search using Fast crawler R 202.176.220.xx http://www.sphsearch.sg/ id_a_f_280606_2 FAST Enterprise Crawler 6 used by WWU (wardi@uni-muenster.de) FAST Enterprise Crawler used by WWU (University of Muenster - Germany) C 128.176.188.2xx http://www.uni-muenster.de/en/index.html http://www.alltheweb.com id_a_f_323 FAST Enterprise Crawler/6 (www.fastsearch.com) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_324 FAST Enterprise Crawler/6.4 (helpdesk at fast.no) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_325 FAST FirstPage retriever (compatible; MSIE 5.5; Mozilla/4.0) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.com id_a_f_326 FAST MetaWeb Crawler (helpdesk at fastsearch dot com) Fast/Alltheweb crawler (66.151.181.xx) R http://www.alltheweb.c
Disallow: /
User-agent: *
Disallow: /images
Disallow: /photogallery
Disallow: /stats