Do you have a question? Post it now! No Registration Necessary. Now with pictures!
- Posted on
- baidu spider and referer spam
June 5, 2005, 7:31 pm
rate this thread
as the referer in their header.
My stats page is Disallowed in robots.txt so referer spam doesn't work,
but I still dislike the idea that these guys are trying it. The spider
is crawling at a reasonable rate (just a couple of hits per hour at the
most), so bandwidth isn't an issue.
Any thoughts on this? Is putting a referer in the spider crawl spammy?
Should I ban these guys from my website?