|
Posted by Richard Lawrence on March 17, 2005, 4:28 pm
Please log in for more thread options
Hi all,
My script requests http://foo.bar.com/ with code that looks a little
like this:
my $ua = LWP::UserAgent->new;
my $cookie_jar = HTTP::Cookies->new(file => $cookie_path);
$cookie_jar->load($cookie_path);
$ua->cookie_jar($cookie_jar);
my $req = HTTP::Request->new(GET => "http://foo.bar.com/");
$cookie_jar->add_cookie_header($req);
# Make request
my $res = $ua->request($req);
# HTML back
if ($res->is_success)
{
$cookie_jar->extract_cookies($res);
$cookie_jar->save();
}
This works great, however the site sends back this:
Set-Cookie: name=fred; domain=.bar.com; path=/
which for some reason doesn't get saved in the cookie jar.
I'm not sure if this is because the set-cookie header is badly formed
or non-standard but since Firefox and IE are both happy with it I
really need to make my code happy with it.
Have I done something wrong or is there a way to get this to work?
Many thanks in advance,
Richard
|
|
Posted by Gunnar Hjalmarsson on March 18, 2005, 1:39 am
Please log in for more thread options
Richard Lawrence wrote:
> My script requests http://foo.bar.com/ with code that looks a little
> like this:
<code snipped>
> This works great, however the site sends back this:
>
> Set-Cookie: name=fred; domain=.bar.com; path=/
>
> which for some reason doesn't get saved in the cookie jar.
Have you possibly finished the printing of CGI headers prematurely? If
you don't understand what I mean by that, please post a *short* but
*complete* script that illustrates the issue.
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
|
|
Posted by Richard Lawrence on March 18, 2005, 3:21 am
Please log in for more thread options
Gunnar Hjalmarsson wrote:
> Richard Lawrence wrote:
> > My script requests http://foo.bar.com/ with code that looks a
little
> > like this:
>
> <code snipped>
>
> > This works great, however the site sends back this:
> >
> > Set-Cookie: name=fred; domain=.bar.com; path=/
> >
> > which for some reason doesn't get saved in the cookie jar.
>
> Have you possibly finished the printing of CGI headers prematurely?
If
> you don't understand what I mean by that, please post a *short* but
> *complete* script that illustrates the issue.
You're right in the fact that I don't understand :) Also following a
bit more investigation, it doesn't look like what I originally thought
was the problem isn't the problem.
Here is a script that demonstrates the code although sadly I have to
mask the site and the details sent. It's worth noting I have no control
over the site I connect to and that the one I use below is different
from the original one as it's easier to show the problem:
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request;
my $received_html;
# Note: horrible wrapping caused by groups-google.com
&get_page("http://www.somedomain.com/process_logon.jsp",
"referring_page=sms.jsp&email_address=my%40email%2Eaddress&password=mypassword",
"http://www.somedomain.com/process_logon.jsp");
&get_page("http://www.somedomain.com/sms.jsp", "",
"http://www.somedomain.com/process_logon.jsp");
sub get_page
{
my ($url, $data, $referer) = @_;
my $cookie_path = "./cookies";
# If $data exists then it's POST, otherwise GET
# Create the user agent
my $ua = LWP::UserAgent->new(env_proxy => 1);
$ua->agent("Mozilla/4.0");
push @{ $ua->requests_redirectable }, 'POST';
# Create the cookie jar
my $cookie_jar = HTTP::Cookies->new(file => $cookie_path);
$cookie_jar->load($cookie_path);
$ua->cookie_jar($cookie_jar);
# If we have a data= section, then this is a POST
my $req;
if ($data)
{
$req = HTTP::Request->new(POST => $url);
$req->content_type('application/x-www-form-urlencoded');
$req->content($data);
}
else
{
$req = HTTP::Request->new(GET => $url);
}
$req->header('Accept' => 'text/*');
# Add a referer if required
$req->referer($referer) if ($referer);
# Add cookies
$cookie_jar->add_cookie_header($req);
print "-- Start of sent headers --n";
print $req->as_string();
print "n-- End of sent headers --n";
# Connect!
my $res = $ua->request($req);
$received_html = "";
die "stop: Error sending request - " . $res->status_line . "n" if
(!$res->is_success);
# Since we're here, we know that it went ok
print "-- Start of received headers and HTML --n";
print $res->as_string . "n";
print "-- End of received headers and HTML --nn";
$received_html = $res->content;
# Extract cookies
$cookie_jar->extract_cookies($res);
$cookie_jar->save();
}
Here is the output I get:
-- Start of sent headers --
POST http://www.somedomain.com/process_logon.jsp
Accept: text/*
Referer: http://www.somedomain.com/process_logon.jsp
Content-Type: application/x-www-form-urlencoded
referring_page=sms.jsp&email_address=my%40email%2Eaddress&password=mypassword
-- End of sent headers --
-- Start of received headers and HTML --
HTTP/1.1 200 OK
Cache-Control: no-cache="set-cookie,set-cookie2"
Connection: close
Date: Fri, 18 Mar 2005 10:07:10 GMT
Via: 1.1 www.somedomain.com
Server: Microsoft-IIS/5.0
Content-Length: 103
Content-Type: text/html; charset=ISO-8859-1
Expires: Thu, 01 Dec 1994 16:00:00 GMT
Client-Date: Fri, 18 Mar 2005 10:07:22 GMT
Client-Peer: 10.10.142.190:80
Client-Response-Num: 1
Refresh: 2;url="sms.jsp"
Set-Cookie: registered=1
Set-Cookie: jsessionid=2380421411141442392;path=/
X-Powered-By: ASP.NET
[html snipped]
-- End of received headers and HTML --
As you can see here, two cookies have been set however in the next
send...
-- Start of sent headers --
GET http://www.somedomain.com/sms.jsp
Accept: text/*
Referer: http://www.somedomain.com/process_logon.jsp
-- End of sent headers --
These aren't sent back to the server.
Many thanks for any suggestions!
Richard
|
|
Posted by Gunnar Hjalmarsson on March 19, 2005, 4:11 am
Please log in for more thread options
Richard Lawrence wrote:
> Gunnar Hjalmarsson wrote:
>> Richard Lawrence wrote:
>>> My script requests http://foo.bar.com/ with code that looks a little
>>> like this:
>>
>> <code snipped>
>>
>>> This works great, however the site sends back this:
>>>
>>> Set-Cookie: name=fred; domain=.bar.com; path=/
>>>
>>> which for some reason doesn't get saved in the cookie jar.
>>
>> Have you possibly finished the printing of CGI headers prematurely? If
>> you don't understand what I mean by that, please post a *short* but
>> *complete* script that illustrates the issue.
>
> You're right in the fact that I don't understand :)
Well, I obviously didn't read your question carefully enough, sorry.
Please disregard that remark.
A cookie without a valid expires parameter is not saved to file by
default. That makes perfect sense, since such a cookie is just a session
cookie which automatically expires at end of session. It gets not saved
to disk by browsers either.
Use the "ignore_discard" parameter to still have it saved to file:
my $cookie_jar =
HTTP::Cookies->new(file => $cookie_path, ignore_discard => 1);
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
|
|
Posted by Richard Lawrence on March 21, 2005, 5:38 am
Please log in for more thread options
Gunnar Hjalmarsson wrote:
> Well, I obviously didn't read your question carefully enough, sorry.
> Please disregard that remark.
Not to worry :)
> A cookie without a valid expires parameter is not saved to file by
> default. That makes perfect sense, since such a cookie is just a
session
> cookie which automatically expires at end of session. It gets not
saved
> to disk by browsers either.
>
> Use the "ignore_discard" parameter to still have it saved to file:
>
> my $cookie_jar =
> HTTP::Cookies->new(file => $cookie_path, ignore_discard => 1);
Ahhh, I didn't realise I needed that because I failed to spot it was a
session cookie. Works like a charm now.
Thanks very much!
Richard
|
| Similar Threads | Posted | | Apache2::Cookie | August 23, 2005, 12:52 am |
| CGI Cookie Path Question | December 26, 2005, 10:53 am |
| Extract Session Cookie | February 23, 2007, 3:29 am |
| mod_perl2 and APR::Request::Cookie error | October 29, 2006, 3:22 pm |
| Perl module for cookie/logon/session management with Apache | May 10, 2007, 12:40 pm |
| File::Store::Hierarchical | April 9, 2007, 11:20 am |
| ANNOUNCE: OOPS 1.004 - Object Oriented Persistent Store | July 4, 2006, 1:53 pm |
| Win32::OLE and CAPICOM to find a certificate in certificate store will raise exception | July 20, 2006, 2:44 am |
| XML-RPC over HTTP with SSL | January 17, 2005, 10:08 pm |
| Using HTTP::Proxy | September 23, 2004, 5:41 am |
|
XML Sitemap