re google bot

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

Well their bot is BANNED on my domain together with the yahoo,msn and
other idiots who have bots and i haven't seen the google bot for a few
weeks now,thank goodness.i hope it is something to do with the court
case that google lost.Yet those other looser bots from yahoo and msn
just keep on coming back even they they get a 403 error

Re: re google bot


Quoted text here. Click to load it

That's nice.  Is there a reason for this or did you just decide to go on a
bot-banning spree?

Infinity has its limits.

Re: re google bot

David Starr wrote:

Quoted text here. Click to load it

Was is much harder than adding

User-agent: *
Disallow: /

To your robots.txt ?

It's funny though but you might have just disabled the ones that actually
obey robot.txt and gave your site off to rogue bots exclusively. Smooth!

See Site Sig Below

Article posted with Web
Developer's USENET Archive
Web and RSS gateway
to your favorite newsgroup -
comp.infosystems.www.authoring.html - 19933
messages and counting!

Site Timeline