Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

My site has been visited constantly for weeks by crawler but I'm not
seeing any results when I search at Anyone know why?? thanks

Re: crawler

Quoted text here. Click to load it

Who knows? Don't have anything to go by like a URL to have a look-see. Also,
bots can crawl your site without it being indexed, and you might be indexed
just not in the top 100 to 1000.


Re: crawler

Bruce Lee wrote:

Quoted text here. Click to load it

Man, I feel your pain!
This Teoma/Ask crawler alone is responsible for 5.6Gb of traffic on one of
my sites just last month, and the number of indexed pages did not move a
bit in several months! They have, however, added about 3000 pages about a
day or so ago. Besides, traffic from is laughable compared to
Google. I'm SO-O-O pissed, I'm getting ready to disallow  this bot from my
sites altogether.  

On the other hand: maybe they supply results to someone worthwhile and not
only AskJeeves?  

Check out my CGI scripts installation offer

Article posted with Web Developer's USENET Archive
no-spam read and post WWW interface to your favorite newsgroup - - 10351 messages and counting!

Site Timeline