Re: Using WWW::Scripter with NTLM authentication

[rowdog]

my $w = WWW::Scripter->new( keep_alive => 1);

WWW::Scripter is a subclass of WWW::Mechanize which is a subclass of LWP::UserAgent which is where you'll find the explaination of the keep_alive option. Basically, it's the number of objects to be stored in the LWP::ConnCache. You need a lot more than 1.

[LambethBoy]

Sorry, my example wasn't very helpful in that respect. Yes, you are right about it controlling the cache capacity. However, even when I set it to large values (100, 1000, 100000 etc) it makes no difference.

After some digging around in the code for LWP::UserAgent (5.835) I found this

sub clone
{
    my $self = shift;
    my $copy = bless { %$self }, ref $self;  # copy most fields

    delete $copy->{handlers};

    delete $copy->{conn_cache};

    # copy any plain arrays and hashes; known not to need recursive co
+py
    for my $k (qw(proxy no_proxy requests_redirectable)) {
        next unless $copy->{$k};
        if (ref($copy->{$k}) eq "ARRAY") {
            $copy->{$k} = [ @{$copy->{$k}} ];
        }
        elsif (ref($copy->{$k}) eq "HASH") {
            $copy->{$k} = { %{$copy->{$k}} };
        }
    }

    if ($self->{def_headers}) {
        $copy->{def_headers} = $self->{def_headers}->clone;
    }

    # re-enable standard handlers
    $copy->parse_head($self->parse_head);

    # no easy way to clone the cookie jar; so let's just remove it for
+ now
    $copy->cookie_jar(undef);

    $copy;
}

So it looks like the conn_cache data is not inherited when the UserAgent is cloned.

If I comment this line out (delete $copy->{conn_cache};), my problems go away. Could a more enlightened member tell me whether or not this looks like a bug, or am I just barking mad and fiddling with something with terrible unintended consequences?

Cheers
LB

[rowdog]

I don't see where or why you're cloning the ua but it looks pretty easy to get the ConnCache from the original.

my $ua2 = $ua->clone;
$ua2->conn_cache($ua->conn_cache);

[LambethBoy]

I'm not explicitly cloning, it happens when WWW::Scripter starts reading referenced JavaScript files (when using the JavaScript plugin). The oroginal HTML document loads fine, I get the NTLM authorisation errors when the .js files are being read.

Cheers,
LB.