Every other day we see some new bot crawling our site, but most of the time they are useless for us. Some of them are just sent out for some special purpose of either spamming or some type of content theft so you have to be careful and should be able to identify such unnecessary guest who have a bad purpose.

What are are Bad Robots ?
Bots who ignore robots.txt file.
Follow links through cgi scripts
Traverse the whole web site in seconds, which affect website speed.
Revisit the web site too often, even when there is no update in site.

How to prevent them for crawling your site?
You can ban all identified bad robots from getting into your site by inserting few lines at .htaccess
You can ban such bad bots by two ways
*) by banning all accesses from a particular site
*) Banning all accesses that use a specific id to access the server.

Usually having a search engine bot on your site is good for your site as they crawl and index your website faster but you should take care that you don’t allow bad bots crawling your site.