Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: Automating downloads with Perl

by mpeters (Chaplain)
on Feb 21, 2005 at 14:55 UTC ( #433067=note: print w/ replies, xml ) Need Help??


in reply to Automating downloads with Perl

If I understand you correctly, then the answer is a most definite yes! This is typically called screen scraping or spidering. The best tool I've found for the job is WWW::Mechanize. It's pretty self explanatory.

If you want to create WWW:::Mechanize scripts automatically, and you know that the sites you are using are not dependent on Javascript to function, you can use HTTP::Recorder to create a proxy that will record your actions with a website so that you can repeat them.

As far as code samples go, the above linked documentation provides lots of examples, but WWW::Mechanize::Examples is another good resource if you need it.


Comment on Re: Automating downloads with Perl

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://433067]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2014-09-21 23:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (176 votes), past polls