Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

lwp+w/o jsessionid

by opensourcer (Monk)
on Sep 20, 2008 at 18:53 UTC ( #712762=perlquestion: print w/ replies, xml ) Need Help??
opensourcer has asked for the wisdom of the Perl Monks concerning the following question:


how do i disable jsession id or change the headers using lwp. I'm writing a scraper for ISP login page and i found out the header something like this using Live HTTP headers plugin of firefox +xxxxxxx& +lcwmm4m',
and lwp sends something like the following
'location' => ' +esh/home;jsessionid=z6toaqyuja0p?user=xxxxxx& +cx2pnr',

I want to remove the jsession.
my code
$uri = "" +me"; use LWP::UserAgent; use HTTP::Cookies; my $ua = LWP::UserAgent->new(keep_alive => 300); $ua->agent("MyBrowser 8.0"); my $cookie_jar = HTTP::Cookies->new( file => $Config::cookie, autosave => 1, ); $ua->cookie_jar($cookie_jar); print "1\n"; my $req = $ua->post($uri, ["user" => $username, "username" => $username, "password" => $password, "ServerList" => "", ]); die unless $req->is_success; }

Comment on lwp+w/o jsessionid
Select or Download Code
Replies are listed 'Best First'.
Re: lwp+w/o jsessionid
by ikegami (Pope) on Sep 21, 2008 at 01:57 UTC

    One way:

    Create a subclass of LWP::UserAgent that overrides request to remove the jsessionid from the requested url.

    use strict; use warnings; package My::UserAgent; BEGIN { our @ISA = 'LWP::UserAgent'; } use LWP::UserAgent qw( ); use URI qw( ); use URI::QueryParam qw( ); sub request { my $self = shift; my $request = shift; my $uri = URI->new( $request->uri() ); $uri->query_param_delete( 'jsessionid' ); $request->uri( $uri ); return $self->SUPER::request( $request, @_ ); } 1;

    Beware that this might cause infinite loops if the server insists on you using jsessionid. Well, not quite infinite since LWP::UserAgent stops redirecting after max_redirect redirects (default is 7).

    Update: Fixed bug mentioned in reply. Added use strict; and use warnings;.

      throws error "Undefined subroutine &main:: called at line 16."
      I named package as UserAgent
        $uri->query_param_delete->( 'jsessionid' );
        should be
        $uri->query_param_delete( 'jsessionid' );


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://712762]
Approved by ikegami
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (12)
As of 2015-12-02 08:03 GMT
Find Nodes?
    Voting Booth?

    My keyboard shows this many letters:

    Results (37 votes), past polls