Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Re^2: Using WWW::Scripter with NTLM authentication

by LambethBoy (Initiate)
on Feb 03, 2011 at 10:55 UTC ( [id://885968]=note: print w/replies, xml ) Need Help??


in reply to Re: Using WWW::Scripter with NTLM authentication
in thread Using WWW::Scripter with NTLM authentication

Sorry, my example wasn't very helpful in that respect. Yes, you are right about it controlling the cache capacity. However, even when I set it to large values (100, 1000, 100000 etc) it makes no difference.

After some digging around in the code for LWP::UserAgent (5.835) I found this

sub clone { my $self = shift; my $copy = bless { %$self }, ref $self; # copy most fields delete $copy->{handlers};
    delete $copy->{conn_cache};
# copy any plain arrays and hashes; known not to need recursive co +py for my $k (qw(proxy no_proxy requests_redirectable)) { next unless $copy->{$k}; if (ref($copy->{$k}) eq "ARRAY") { $copy->{$k} = [ @{$copy->{$k}} ]; } elsif (ref($copy->{$k}) eq "HASH") { $copy->{$k} = { %{$copy->{$k}} }; } } if ($self->{def_headers}) { $copy->{def_headers} = $self->{def_headers}->clone; } # re-enable standard handlers $copy->parse_head($self->parse_head); # no easy way to clone the cookie jar; so let's just remove it for + now $copy->cookie_jar(undef); $copy; }

So it looks like the conn_cache data is not inherited when the UserAgent is cloned.

If I comment this line out (delete $copy->{conn_cache};), my problems go away. Could a more enlightened member tell me whether or not this looks like a bug, or am I just barking mad and fiddling with something with terrible unintended consequences?

Cheers
LB

Replies are listed 'Best First'.
Re^3: Using WWW::Scripter with NTLM authentication
by rowdog (Curate) on Feb 03, 2011 at 21:49 UTC

    I don't see where or why you're cloning the ua but it looks pretty easy to get the ConnCache from the original.

    my $ua2 = $ua->clone; $ua2->conn_cache($ua->conn_cache);

      I'm not explicitly cloning, it happens when WWW::Scripter starts reading referenced JavaScript files (when using the JavaScript plugin). The oroginal HTML document loads fine, I get the NTLM authorisation errors when the .js files are being read.

      Cheers,
      LB.

        Hrm, I hadn't realized that WWW::Mechanize::FAQ says that

        Mech keeps a history of every page, and the state it was in. It actually keeps a clone of the full Mech object at every step along the way.

        So, there's a clone for each step. I'm kind of in over my head here but I don't see anything wrong with your workaround, as long as you're not sending requests from a bunch of different threads or something silly like that.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://885968]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (6)
As of 2024-03-19 09:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found