Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: How to download html with threads?

by Trizor (Pilgrim)
on Jul 30, 2007 at 20:47 UTC ( [id://629663]=note: print w/replies, xml ) Need Help??

Help for this page

Select Code to Download


  1. or download this
    sub ReadURLS {
      my ($queue,$filename) = @_;
    ...
      close $outFH;
      return 1;
    }
    
  2. or download this
    my $nr_workers = 5; #set this value for the number of side by side dow
    +nloaders and parsers. Better yet, take it as an argument
    my $urlfile = "url_planets.txt"; # see comment about arguments
    ...
    }
    
    # At this point, barring some horrible catastrophe, the specified $out
    +file should have the desired output.
    

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://629663]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (3)
As of 2024-04-18 22:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found