spider job

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

Thanking Jim for his reponse, I'd like to ask the NG the question that I'm
afraid is obvious, but nor for me.
Usually a search engine robot or spider, OPENS the web sites like a user
would do with a browser, or is able to READ in some way "offline" the
website files, without requesting them with a browser?
I have clues for both ways and don't know how it really happens. I need this
info to better understand how to realize a website and give it better

Thank you

Re: spider job

Max wrote:

Quoted text here. Click to load it

The first way -- it requests files from the site like a normal user would.
Except that the "browser" is built into the spider itself. And normally
the "browser" doesn't support Javascript / Flash / etc.

Toby A Inkster BSc (Hons) ARCS
Contact Me ~ http://tobyinkster.co.uk/contact
Geek of ~ HTML/SQL/Perl/PHP/Python*/Apache/Linux

* = I'm getting there!

Site Timeline