Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Using WWW::Scripter with NTLM authentication

by rowdog (Curate)
on Feb 02, 2011 at 17:47 UTC ( #885804=note: print w/ replies, xml ) Need Help??


in reply to Using WWW::Scripter with NTLM authentication

my $w = WWW::Scripter->new( keep_alive => 1);

WWW::Scripter is a subclass of WWW::Mechanize which is a subclass of LWP::UserAgent which is where you'll find the explaination of the keep_alive option. Basically, it's the number of objects to be stored in the LWP::ConnCache. You need a lot more than 1.


Comment on Re: Using WWW::Scripter with NTLM authentication
Select or Download Code
Replies are listed 'Best First'.
Re^2: Using WWW::Scripter with NTLM authentication
by LambethBoy (Initiate) on Feb 03, 2011 at 10:55 UTC

    Sorry, my example wasn't very helpful in that respect. Yes, you are right about it controlling the cache capacity. However, even when I set it to large values (100, 1000, 100000 etc) it makes no difference.

    After some digging around in the code for LWP::UserAgent (5.835) I found this

    sub clone { my $self = shift; my $copy = bless { %$self }, ref $self; # copy most fields delete $copy->{handlers};
        delete $copy->{conn_cache};
    # copy any plain arrays and hashes; known not to need recursive co +py for my $k (qw(proxy no_proxy requests_redirectable)) { next unless $copy->{$k}; if (ref($copy->{$k}) eq "ARRAY") { $copy->{$k} = [ @{$copy->{$k}} ]; } elsif (ref($copy->{$k}) eq "HASH") { $copy->{$k} = { %{$copy->{$k}} }; } } if ($self->{def_headers}) { $copy->{def_headers} = $self->{def_headers}->clone; } # re-enable standard handlers $copy->parse_head($self->parse_head); # no easy way to clone the cookie jar; so let's just remove it for + now $copy->cookie_jar(undef); $copy; }

    So it looks like the conn_cache data is not inherited when the UserAgent is cloned.

    If I comment this line out (delete $copy->{conn_cache};), my problems go away. Could a more enlightened member tell me whether or not this looks like a bug, or am I just barking mad and fiddling with something with terrible unintended consequences?

    Cheers
    LB

      I don't see where or why you're cloning the ua but it looks pretty easy to get the ConnCache from the original.

      my $ua2 = $ua->clone; $ua2->conn_cache($ua->conn_cache);

        I'm not explicitly cloning, it happens when WWW::Scripter starts reading referenced JavaScript files (when using the JavaScript plugin). The oroginal HTML document loads fine, I get the NTLM authorisation errors when the .js files are being read.

        Cheers,
        LB.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://885804]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (11)
As of 2015-07-29 11:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (263 votes), past polls