Click here to get back home

Website test module

 HomeNewsGroups | Search | About
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Website test module Ignoramus9014 02-19-2008
---> Re: Website test module Joost Diepenmaa...02-19-2008
Get Chitika Premium
Posted by Ignoramus9014 on February 19, 2008, 2:37 pm
Please log in for more thread options
I would like to know what are the "most recommended" perl modules for
testing websites. For example, my typical task would be

- connect to IP address A
- request webpage B http://B/page.html
- check that it contacts a keyword C


What is important is that address A (my test or backup website) does
not necessarily match virtual host name B.

I read on Test::WWW::Mechanize, which I like a lot as I am very
familiar with WWW::Mechanize, but it does not seem to offer this
ability (connect to a given IP but use a unrelated virtual address).

Thanks

i

Posted by Joost Diepenmaat on February 19, 2008, 2:51 pm
Please log in for more thread options

> I would like to know what are the "most recommended" perl modules for
> testing websites. For example, my typical task would be
>
> - connect to IP address A
> - request webpage B http://B/page.html
> - check that it contacts a keyword C

I'm not sure what this means.

> What is important is that address A (my test or backup website) does
> not necessarily match virtual host name B.
>
> I read on Test::WWW::Mechanize, which I like a lot as I am very
> familiar with WWW::Mechanize, but it does not seem to offer this
> ability (connect to a given IP but use a unrelated virtual address).

So you want to access site A but with the hostname set to that of B?
The first thing in my mind is to set the IP address of B with the name
of A in the /etc/hosts file (there are similar files in various versions
windows, should you need to).

You probably do need root/administrator priviliges to set the hosts
file, though.

Another way could be to use a local HTTP proxy and connect through
that. I'm confident HTTP::Proxy will let you write something that
translates hostnames in a dozen lines or so - see its example scripts.

--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/

Posted by Ignoramus9014 on February 19, 2008, 3:10 pm
Please log in for more thread options
>
>> I would like to know what are the "most recommended" perl modules for
>> testing websites. For example, my typical task would be
>>
>> - connect to IP address A
>> - request webpage B http://B/page.html
>> - check that it contacts a keyword C
>
> I'm not sure what this means.
>
>> What is important is that address A (my test or backup website) does
>> not necessarily match virtual host name B.
>>
>> I read on Test::WWW::Mechanize, which I like a lot as I am very
>> familiar with WWW::Mechanize, but it does not seem to offer this
>> ability (connect to a given IP but use a unrelated virtual address).
>
> So you want to access site A but with the hostname set to that of B?
> The first thing in my mind is to set the IP address of B with the name
> of A in the /etc/hosts file (there are similar files in various versions
> windows, should you need to).

You got it, yes, I want to use a certain virtual hostname, but access
an IP that does not match it (because I am testing a replacement
server).

For example, I am testing a replacement server for www.algebra.com.
The IP address for real algebra.com is 65.182.171.162 and it is in a
datacenter.

The replacement server is 10.1.2.3 and is in my basement, being built
and tested.

I want to access 10.1.2.3 and say "give me http://www.algebra.com/page.html".

i

> You probably do need root/administrator priviliges to set the hosts
> file, though.
>
> Another way could be to use a local HTTP proxy and connect through
> that. I'm confident HTTP::Proxy will let you write something that
> translates hostnames in a dozen lines or so - see its example scripts.
>

OK, that, in fact, is a great start, if I can use a proxy in this
manner I will be happy. Thanks

i

Posted by Ben Morrow on February 19, 2008, 3:39 pm
Please log in for more thread options

> I would like to know what are the "most recommended" perl modules for
> testing websites. For example, my typical task would be
>
> - connect to IP address A
> - request webpage B http://B/page.html
> - check that it contacts a keyword C
>
> What is important is that address A (my test or backup website) does
> not necessarily match virtual host name B.
>
> I read on Test::WWW::Mechanize, which I like a lot as I am very
> familiar with WWW::Mechanize, but it does not seem to offer this
> ability (connect to a given IP but use a unrelated virtual address).

You should be able to subclass LWP::Protocol::http, override
->_new_socket to map B->A and call SUPER::_new_socket, and then use
LWP::Protocol::implementor to set your new class as the implementation
to use for http. Then you should be able to just use WWW::Mech normally.
Something like (completely untested):

use LWP;
use WWW::Mechanize;

{
package My::LWP::Protocol::http;

our @ISA = 'LWP::Protocol::http';

my %fake = (
'B' => '192.168.1.2',
);

sub _new_socket {
my ($self, $host, $port, $timeout) = @_;
$fake and $host = $fake;
return $self->SUPER::_new_socket($host, $port, $timeout);
}

LWP::Protocol::implementor http => __PACKAGE__;
}

my $M = WWW::Mechanize->new(...);

...

Ben


Posted by Ignoramus9014 on February 19, 2008, 4:49 pm
Please log in for more thread options
>
>> I would like to know what are the "most recommended" perl modules for
>> testing websites. For example, my typical task would be
>>
>> - connect to IP address A
>> - request webpage B http://B/page.html
>> - check that it contacts a keyword C
>>
>> What is important is that address A (my test or backup website) does
>> not necessarily match virtual host name B.
>>
>> I read on Test::WWW::Mechanize, which I like a lot as I am very
>> familiar with WWW::Mechanize, but it does not seem to offer this
>> ability (connect to a given IP but use a unrelated virtual address).
>
> You should be able to subclass LWP::Protocol::http, override
> ->_new_socket to map B->A and call SUPER::_new_socket, and then use
> LWP::Protocol::implementor to set your new class as the implementation
> to use for http. Then you should be able to just use WWW::Mech normally.
> Something like (completely untested):
>
> use LWP;
> use WWW::Mechanize;
>
> {
> package My::LWP::Protocol::http;
>
> our @ISA = 'LWP::Protocol::http';
>
> my %fake = (
> 'B' => '192.168.1.2',
> );
>
> sub _new_socket {
> my ($self, $host, $port, $timeout) = @_;
> $fake and $host = $fake;
> return $self->SUPER::_new_socket($host, $port, $timeout);
> }
>
> LWP::Protocol::implementor http => __PACKAGE__;
> }
>
> my $M = WWW::Mechanize->new(...);
>
> ...
>
> Ben
>

Ben, thanks, it is a little over my head, but I spent some time
trying.

I got to the point of running queries, but the _new_socket function
was never called.

Here's my actual code:

######################################################################

#!/usr/bin/perl

use LWP;
use WWW::Mechanize;
use Data::Dumper;

{
package My::LWP::Protocol::http;
use Data::Dumper;
use LWP::Protocol::http;

our @ISA = 'LWP::Protocol::http';

my %fake = (
'B' => '127.0.0.1',
);

sub _new_socket {
my ($self, $host, $port, $timeout) = @_;
print STDERR "HOST=$host\n";
exit 0;
$fake and $host = $fake;
return $self->SUPER::_new_socket($host, $port, $timeout);
}

sub new {
return LWP::Protocol::http->new;
}

LWP::Protocol::implementor http => __PACKAGE__;
}

my $M = WWW::Mechanize->new();

my $resp = $M->get( 'http://www.algebra.com/robots.txt' );

print $resp->content;

Similar ThreadsPosted
Test::Harness and test values. August 7, 2006, 4:06 pm
website hosting January 3, 2005, 3:27 pm
Website scraper September 24, 2005, 8:59 am
Classic website March 21, 2006, 7:32 am
faq on perl website May 9, 2006, 2:19 pm
Website Interface August 15, 2006, 1:25 pm
This website is for sale! August 24, 2007, 9:45 pm
Logging onto a Website August 27, 2007, 12:03 am
generate different website out of database. January 20, 2005, 4:32 am
share a session between two website April 29, 2006, 8:59 am

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap