This happens to so many people: you make a website and they don’t want just anyone to be able to access them. Unfortunately, once your website goes live, many search engines like MSN, Google, Yahoo, etc., are already way ahead of you in terms of indexing your work.
You can, however, create a command that will tell search engines to not include your website. First you have to create a robots.txt file. Then type into the pad “User-agent:” and then fill in that field the search engines you want blocked, if not all of them. After that type, “Disallow:” and then fill it with a “/” symbol. This will block those search engines from searching through your stuff.
You can also use the non-index Meta standard. Place this: <meta name=”robots” content=”noindex”> in the HEAD section of the webpage. This will have virtually the same effect.
For more information as to how to remove only specific parts of a website, more information is provided here.
Technorati Tags: robots.txt
Article written by SEOnotepad.com
No Comment Received
Sorry the comment area are closed for non registered users