Do you have a question? Post it now! No Registration Necessary. Now with pictures!
- Posted on
- how to ban yahoo mobile crawler
December 17, 2008, 12:58 am
rate this thread
forum was too much visited by this user agent:
Nokia6682/2.0 - 3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0
configuration/CLDC-1.1 UP.Link/22.214.171.124.0 - compatible;YahooSeeker/M1A1-R2D2
It takes along this URL that inform how to limit the spider's visits or
to ban it, by means of robots.txt:
I've tried to ban it by means of robots file and also .htaccess but with
What can I do? Anyone knows Yahoo mobile IPS?
Re: how to ban yahoo mobile crawler
robots.txt rules only help if the user agent is a spider, and that
spider respects the rules (many don't), which is interesting when some
sites have 200 robot.txt deny rules for spiders that don't respect them
anyway. You should either use .htaccess to block the user agent (or
relevant and unique portion of the UA string), or do so in the web
forum code itself (.htaccess is a better idea). That will solve your
problem. What method and rule/directive did you use in the .htaccess
file that didn't obtain the desired results for you? What web server
are you on? In what way didn't it work? Did it error or just not deny
Tim Greer, CEO/Founder/CTO, BurlyHost.com, Inc.
Shared Hosting, Reseller Hosting, Dedicated & Semi-Dedicated servers
and Custom Hosting. 24/7 support, 30 day guarantee, secure servers.
Industry's most experienced staff! -- Web Hosting With Muscle!
- » Is there any "quick & dirty" way to test for possible SQL injection vulnerability of our ...
- — Next thread in » Webmaster Forum