Robots.txt: Allow Directive
October 7th, 2009
No comments
Just a quick tip for writing a robots.txt file for your website, especially if you’re using the Google Webmaster Tools website to help you; the allow: /directory-name/ directive is not part of the standard protocol.
It’s definitely supported by Google but I’m not sure about other search engines.