Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

$mechanize->get($url) connexion timeout stalls my script

by mnem00 (Initiate)
on Oct 26, 2009 at 17:33 UTC ( [id://803316]=perlquestion: print w/replies, xml ) Need Help??

mnem00 has asked for the wisdom of the Perl Monks concerning the following question:

Hi noble monks, i got a loop which freezes after a $mechanize->get($url) if this instruction returns a connexion timeout. i'd like to bypass this, i tried an eval{expr} or next but that doesn't work. is it possible ? (i could Net::Ping each host before getting their content but it would slow my script :/) here's the code
#!/usr/bin/perl -w use strict; my($database,$hostname,$port,$user,$password) = @ARGV; use WWW::Mechanize; use DBI; print"usage : xmlrpc.pl databaseName DBhostname DBportNumber DBusernam +e DBlogin\n"; my $dsn = "DBI:mysql:database=$database;host=$hostname;port=$port"; my $dbh = DBI->connect($dsn, $user, $password) or die "can't connect t +o database"; my $drh = DBI->install_driver("mysql") or die "can't install mysql dri +ver"; my @databases = DBI->data_sources("mysql"); my $sth = $dbh->prepare("select url,username,password from blogs where + type like \"%pattern%\";") or die ("\ncan't get rows"); $sth->execute(); my @row; while ( @row = $sth->fetchrow_array ) { print "@row\n"; my $urlbase = $row[0]; my $pwd = $row[2]; my $login = $row[1]; my $mech = WWW::Mechanize->new(cookie_jar => {}) o +r next; eval{$mech->get ($urlbase)} or next;#the bug's her +e eval{$mech->form_number(0)} or next; eval{$mech->set_fields(log=>$login,pwd=>$pwd)}or +next; eval{$mech->submit} or next; eval{$mech->form_number(0)}or next; eval{$mech->tick( "enable_xmlrpc", "1" )} or next; eval{$mech->submit} or next; }

Replies are listed 'Best First'.
Re: $mechanize->get($url) connexion timeout stalls my script
by zwon (Abbot) on Oct 26, 2009 at 19:13 UTC

    You can set timeout by passing timeout parameter to WWW::Mechanize->new. But if you have to process a lot of URLs it is better to perform several requests in parallel. You may do this using threads or using something like AnyEvent::HTTP (I personally prefer the later).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://803316]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (5)
As of 2024-04-19 13:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found