Click here to get back home

Selective Redirect with LWP::Parallelel::Useragent
 Home | NewsGroups | Search | About
Login Password
Register Now!
 comp.lang.perl.modules    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Selective Redirect with LWP::Parallelel::Useragent jonas.soderstrom 01-20-2006
Get Chitika Premium
Posted by jonas.soderstrom on January 20, 2006, 8:01 am
Please log in for more thread options

Hi I have a problem.

I use LWP::Parallel::Useragent to fetch prices to my Meta Search
Engine.

It usually works great but now i have a problem.
There is one site on which it does not work if redirects are enabled.
At the moment I use simple_request and works my way through every step
and it works great.

But I want to be able to use the Register method so its truly parallel.
I know how to turn Redirect of but several of the other sites need
Redirect to work.

Do you see my problem.
I want to be able to turn of Redirect on one site but not on the
others.

I really need help on this one.


Posted by xhoster on January 20, 2006, 3:29 pm
Please log in for more thread options

jonas.soderstrom@gmail.com wrote:
> Hi I have a problem.
>
> I use LWP::Parallel::Useragent to fetch prices to my Meta Search
> Engine.

It is UserAgent, not Useragent. Case often matters.

>
> It usually works great but now i have a problem.
> There is one site on which it does not work if redirects are enabled.
> At the moment I use simple_request and works my way through every step
> and it works great.
>
> But I want to be able to use the Register method so its truly parallel.
> I know how to turn Redirect of[f] but several of the other sites need
> Redirect to work.
>
> Do you see my problem.
> I want to be able to turn of Redirect on one site but not on the
> others.

I'm not sure if I see the problem or not. The docs for $ua->redirect tells
you to "See $ua->register for how to change the behaviour for particular
requests only." So it seems like all you have to do is decide, before
registering each request, whether the request is to that specific site or
not, and register it accordingly.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Posted by jonas.soderstrom on January 24, 2006, 2:19 am
Please log in for more thread options


Hi xho

Thanks for you answer.

This is exactly what I wanted to hear. I have been looking in the
documentation for this solution but I can't seem to find the proper
argument/parameters to send. Maybe I am looking in the wrong place.

I will looking deeper this time now that I have your answer. If anybody
knows what arguments/parameters so send to $ua->register to turn of the
redirect for the specific request please let me know. Either here or by
email (or both)

Jonas


Posted by xhoster on January 24, 2006, 11:37 am
Please log in for more thread options


jonas.soderstrom@gmail.com wrote:
> Hi xho
>
> Thanks for you answer.
>
> This is exactly what I wanted to hear. I have been looking in the
> documentation for this solution but I can't seem to find the proper
> argument/parameters to send. Maybe I am looking in the wrong place.
>
> I will looking deeper this time now that I have your answer. If anybody
> knows what arguments/parameters so send to $ua->register to turn of the
> redirect for the specific request please let me know. Either here or by
> email (or both)
>
> Jonas

Hi Jonas,

Perhaps you are using an older version which does not have this feature?
This is from the docs for my version:

$ua->register ( $request [, $arg [, $size [, $redirect_ok]]] )
Registers the given request with the User Agent. In case of an
error, a "HTTP::Request" object containing the HTML-Error
message is returned. Otherwise (that is, in case of a success)
it will return undef.
...
Specifying $redirect_ok will alter the redirection behaviour for
this particular request only. '1' or any other true value will
force Parallel::UserAgent to follow redirects, even if the
default is set to 'no_redirect'. (see "$ua-"redirect>) '0' or
any other false value should do the reverse. See LWP::UserAgent
for using an object's "requests_redirectable" list for
fine-tuning this behav- ior.

I don't know how to describe it better than that (other than to say that it
should read "...'0' or any other false but defined value should do the
reverse").


Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Similar ThreadsPosted
Not able to use LWP::UserAgent::Authenticated December 26, 2005, 9:11 pm
LWP::UserAgent and HTTP_ACCEPT January 24, 2006, 7:03 pm
LWP:Simple/LWP:UserAgent (newbie) February 24, 2005, 12:11 pm
A pipe or FH equivalent of LWP::UserAgent ? February 25, 2005, 3:12 pm
URGENT: no response - LWP::UserAgent February 26, 2005, 12:22 pm
Using HTTPS with LWP::UserAgent = Bad Service March 30, 2005, 6:45 am
installing LWP::UserAgent on windows June 21, 2005, 10:37 pm
newbie LWP::UserAgent questions August 13, 2006, 8:10 pm
CGI & LWP::UserAgent - forwarding cookies - how? January 15, 2007, 3:25 pm
LWP::UserAgent hangs up infinitively February 26, 2007, 9:05 am

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap