# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism User-agent: * Disallow: /Admin Disallow: /tmp Disallow: /logs Disallow: /db Disallow: /logs Disallow: /bin Disallow: /tecknonerd/bin Disallow: /tecknonerd/app_data Disallow: /App_Data/ Disallow: /Content/Dynamic/ Disallow: /Content/Dynamic/imageGallery/ Disallow: /Content/Dynamic/imageGallery/large/ Disallow: /Content/Dynamic/imageGallery/medium/ Disallow: /Content/Dynamic/imageGallery/small/ Disallow: /Content/BabySite/Includes/ Disallow: /include/ Disallow: /jandj-online/admin/ Disallow: /jandj-online/App_Data/ Disallow: /jandj-online/Bin/ Disallow: /jandj-online/Content/Dynamic/ Disallow: /jandj-online/Content/Dynamic/imageGallery/ Disallow: /jandj-online/Content/Dynamic/imageGallery/large/ Disallow: /jandj-online/Content/Dynamic/imageGallery/medium/ Disallow: /jandj-online/Content/Dynamic/imageGallery/small/ Disallow: /jandj-online/Content/BabySite/Includes/