# # Standard robots.txt file for TPDirect, undated July, 2006 # # URL and Link Search Engine Bot Exclusions # The * tells all bots to follow the guidlines listed # Wildcards (*,?) are not allowed in Dissallow statements # Edit this file to exclude any folders needed or specific files # within allowed folders that are to be excluded. # Example File exclusion for nobots.html in the test folder # Disallow: /test/nobots.html # # Email Bots are disallowed from all folders and files within the site # and must be identified by the bot name. This list must be updated to # remain operational. # User-agent: Titan Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: * Disallow: /_db Disallow: /_dbfunctions.asp Disallow: /_siteconfig.asp Disallow: /_admin Disallow: /_autoscripts Disallow: /_backup Disallow: /_database Disallow: /_download Disallow: /_includes Disallow: /_template Disallow: /activeerror Disallow: /analog Disallow: /aspnet_client Disallow: /bin Disallow: /cgi-bin Disallow: /dbmail Disallow: /images Disallow: /scripts Disallow: /sendit Disallow: /shop Disallow: /test Disallow: /temp Disallow: /templates