Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View


    Is it usual for the googlebot to spider orphan folders which have
no references to them anywhere?

    For example, last night I put up a proposed new index page for a
client to look at, in a newly made entirely unreferenced folder on a
website that google spiders fairly regularly. I recently did a similar
thing for another proposed page.

    In both cases, the googlebot spidered them on its next visit. How
does it find such folders? Is it normal?




Re: Googlebot...

Fleeing from the madness of the  jungle
and said:

Quoted text here. Click to load it

How do you do?

Quoted text here. Click to load it

If that were true (no references) then it's unlikely, but ...

Quoted text here. Click to load it

  ... do you have the google tool-bar loaded?  or perhaps a version of  
Opera that still displays context sensitive ads?  or maybe your client  

perhaps you have a dynamic site-map script?

Check your logs - you'll see some access you don't recognise right there  
between your browsing the pages and the goog-bot slurping it up.

William Tasso

whither a trophy?

Re: Googlebot...

Will wrote:
Quoted text here. Click to load it

Yes, this is fairly common. It's caused by previously visiting those
pages (either you or your customer) with the Google toolbar active.

They're definitely getting evil....

Re: Googlebot... says...
Quoted text here. Click to load it

    Hi, thanks to both you and William, I do have some parts of the
google toolbar enabled, so I guess that's it.




Site Timeline