Automated HTML code extraction and documenting

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
I have an interesting problem on hand. My goal is to find all pages
within our institution's web domain that contain a certain length of
form posting code , then report a list of all pages that contain the

The reason for searching for the code is that we are looking for all
pages that links to our campus' search engine. Any user who makes a
web page on campus can add the search engine easily with the
appropriate code; in turn it is uncertain the extent of the search
engine implementation.  However, we will be changing the way the search
engine is accessed and in turn we must find and correct all instances
of the old code.

The files that lie under our domain are uncentralized; lying on many
different web servers throughout the university, so a query on one web
server will not work.

My thoughts are that we need a HTML spidering software to crawl
throughout the entire domain, while at the same time searching for the
code segment specified and documenting its location.

If anyone has a better solution, or can possibly point me in the
direction that I need to go in, it would be much appreciated.

Re: Automated HTML code extraction and documenting wrote:

Quoted text here. Click to load it

Much easier: make your SE backwards compatible, or use mod_rewrite.
If you log the use of the "old way", you get your badly behaving pages.

Oh, and I do write crawling software, see sig :-)

John                       Perl SEO tools:
                 Experienced (web) developer:
Get a SEO report of your site for just 100 USD:

Site Timeline