Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: WWW::Mechanize link limit?

by johnnywang (Priest)
on Dec 18, 2005 at 08:00 UTC ( #517575=note: print w/ replies, xml ) Need Help??


in reply to WWW::Mechanize link limit?

shouldn't be any size limit, here's the method in WWW::Mechanize that parses the links:

sub _extract_links { require WWW::Mechanize::Link; my $self = shift; my $p = HTML::TokeParser->new(\$self->{content}); $self->{links} = []; while (my $token = $p->get_tag( keys %urltags )) { my $tag = $token->[0]; my $url = $token->[1]{$urltags{$tag}}; my $text; my $name; if ( $tag eq "a" ) { $text = $p->get_trimmed_text("/$tag"); $text = "" unless defined $text; my $onClick = $token->[1]{onclick}; if ( $onClick && ($onClick =~ /^window\.open\(\s*'([^']+)'/) ) + { $url = $1; } } if ( $tag ne "area" ) { $name = $token->[1]{name}; } next unless defined $url; # probably just a name link or <AR +EA NOHREF...> push( @{$self->{links}}, WWW::Mechanize::Link->new( $url, $tex +t, $name, $tag, $self->base ) ); } # Old extract_links() returned a value. Carp if someone expects # this version to return something. if ( defined wantarray ) { my $func = (caller(0))[3]; $self->warn( "$func does not return a useful value" ); } return; }


Comment on Re: WWW::Mechanize link limit?
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://517575]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (8)
As of 2014-11-27 17:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (186 votes), past polls