web crawler using POST method

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

Present web crawlers don't seem to use the HTTP POST method. Is this
unethical or even illegal? It may sure cause some problems in the
beginning, but in the end it may make more information accessible
through search engines. What do you think?


Re: web crawler using POST method

Quoted text here. Click to load it

What would they POST to the web site? A SOAP request?
Philip Baker

Site Timeline