Setting the LWP::Parallel::UserAgent chunk size in the callback method

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!


I want to use LWP::Parallel::UserAgent to handle each request when it
has returned.

I've tried the wait() method. The problem with the wait() method is
that I can parse each response first when all the other requests are

My second try was to use a callback function. Since I want to handle
the whole response data at the same time, I've set the chunk size to be
very large (100 MB). I was thinking that this would make each chunk to
be the whole page. However, that didn't work. Each cunk becomes about
1460 bytes anyway.

My third try was to use a callback function and put the chunks for a
certain request in a buffer. So that when the whole page is loaded, I
can parse it. However - I can't figure out how to know when the page is
fully loaded. As staded in the docs, the length($content) == 0 doesn't
mean the connection is closed.

Is the some other way to handle each request when it has returned?

Site Timeline