Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Downloading html content

by pupun (Initiate)
on Jan 02, 2008 at 08:36 UTC ( #659930=perlquestion: print w/replies, xml ) Need Help??

pupun has asked for the wisdom of the Perl Monks concerning the following question:

Hi All, Actually I am a newbie in this world of perl. I have been assigned a task where I need to write codes that will make the system to download a attached files from the browser automatically and will then save it into a directory. just like the crontab jobs we have in shell scripting. But here we are having a windows environment(windows 2000).please guide me. thanks in advnace

Replies are listed 'Best First'.
Re: Downloading html content
by marto (Archbishop) on Jan 02, 2008 at 09:00 UTC
Re: Downloading html content
by GrandFather (Sage) on Jan 02, 2008 at 08:52 UTC

    LWP is the tool that is generally used. WWW::Mechanize gets quite a lot of use for that sort of thing too.


    Perl is environmentally friendly - it saves trees
Re: downloading files from websites automatically in windows
by Punitha (Priest) on Jan 02, 2008 at 09:09 UTC
Re: downloading files from websites automatically in windows
by kirillm (Friar) on Jan 02, 2008 at 09:48 UTC

    Hi,

    Use LWP::Simple to fetch files from the WWW easily. You could automatically save them to the local filesystem using the mirror() function. E.g. (this is right out of my head, not tested):

    use LWP::Simple; # An array of array refs whith remote url and local file my @files = (["http://foo.bar/baz1.html", "C:/.../baz1.html"], ["http://bar.foo/zab2.html", "C:/.../zab2.html"]); for my $duplet (@files) { mirror($duplet->[0], $duplet->[1]); }

    A more detailed/specific recipe will require that you tell more about what you are trying to achieve.

      hi, thanks for ur help. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data(its a link specifying download data item) and then save it in a network drive. OS- windows 2000
Re: Downloading html content
by blazar (Canon) on Jan 02, 2008 at 21:07 UTC
    I have been assigned a task where I need to write codes that will make the system to download files from the browser automatically and will then save it into a directory.

    I personally believe that you just mean "to download files from the web" and in that case the answers you already got are perfect. But if you really mean "from the browser", in the acceptation of "from the web, by means of a browser," then the following modules may help:

    --
    If you can't understand the incipit, then please check the IPB Campaign.
      Thanks for your reply. Let me put it more clearly. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data and then save it in a network drive. OS- windows 2000
Re: Downloading html content
by Win (Novice) on Jan 02, 2008 at 16:53 UTC
    pupun,

    You've probably got all you need. However, if not, the following bit of code might help.
    #!/usr/bin/perl use warnings; use strict; use LWP::UserAgent; use HTTP::Request::Common; my @web_addresses = ("http://www.web1.com/page", "http://www.web2.co.uk/page.htm"); my $count = 0; foreach (@web_addresses) { $count++; my $ua = LWP::UserAgent->new; $ua->get("$_", ':content_file' => "$count"); }
    I expect that the code may work without  use HTTP::Request::Common;. Not sure, you will have to try that out if you wish to know.
      Thanks for your reply. Let me put it more clearly. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data and then save it in a network drive. OS- windows 2000

        Let me put it more clearly. If you looked at the modules suggested by other monks you would find out that the task is fairly easy even if you have to login. You can either use WWW::Mechanize to navigate to the login page, fill in the form, submit it and then navigate to that "particular link" and save content. Or you can use LWP::UserAgent to send the login information to the same URL the login form would send it making sure the cookies sent with the response are remembered by the LWP::UserAgent object and then download that particular link.

        But I admit it would require that you look at the documentation of those modules and try to do something yourself.

        Show us you tried and come back if you have problems, but don't expect to get a full script doing exactly what you ask for. This is not a free script service.

      A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://659930]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (2)
As of 2020-07-04 18:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?