Do you have a question? Post it now! No Registration Necessary. Now with pictures!
- Posted on
May 10, 2005, 3:55 pm
rate this thread
Re: allow googlebot but disallow others from 1 folder
ITYM "don't want any other bots to index the files", and "the robots.txt
This isn't HTML, but anyway:
The robots.txt format is explained at
If you do want to *prevent* other robots from visiting the directory in
question, you would have to use real access lists instead of relying on the
robots in question to present themselves in an honest fashion.