Creating web directory

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
Hello to all,

I have a spider written in java and i now want to create a web
directory to store it in my database.
My question is where do i start my spider?.

There is a possibility that i may not go through a lot of sites.
how can i make sure that i should reduce that possibility.

thank you in advance,

Re: Creating web directory

I don't know anything but if I had a spider I'd start it off at dmoz or
google's directory

Re: Creating web directory

Another good spot to start a spider would be on servers of popular
internet service providers...and maybe the yahoo directory

aaron wall

Quoted text here. Click to load it

Re: Creating web directory (aaron wall) wrote in message
Quoted text here. Click to load it

Thank you all for Helping me out.

I love groups.

Site Timeline