Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
Hi Folk

I am trying to reduce the number of pages that are indexed on my site.

If I were to put

User-agent: *
Disallow: /*/*.php
Disallow: /*/*.html

in the robots.txt file then would that mean that any files that are not in
the root www directory will be ignored by robots?

Thanks in advance (AKA TIA)

- Nicolaas

Re: robots.txt

windandwaves wrote:

Quoted text here. Click to load it

No. Most robots do not support globbing, except for the special case
of "User-agent: *"

Toby A Inkster BSc (Hons) ARCS
Contact Me  ~

Re: robots.txt

Toby Inkster wrote:

Quoted text here. Click to load it

Would it work for Google?  They do support globbing, but I am not sure if
the syntax is correct.



Site Timeline