Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Automating downloads with Perl

by mpeters (Chaplain)
on Feb 21, 2005 at 14:55 UTC ( #433067=note: print w/ replies, xml ) Need Help??


in reply to Automating downloads with Perl

If I understand you correctly, then the answer is a most definite yes! This is typically called screen scraping or spidering. The best tool I've found for the job is WWW::Mechanize. It's pretty self explanatory.

If you want to create WWW:::Mechanize scripts automatically, and you know that the sites you are using are not dependent on Javascript to function, you can use HTTP::Recorder to create a proxy that will record your actions with a website so that you can repeat them.

As far as code samples go, the above linked documentation provides lots of examples, but WWW::Mechanize::Examples is another good resource if you need it.


Comment on Re: Automating downloads with Perl

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://433067]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (14)
As of 2015-07-03 12:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (53 votes), past polls