Downloading html content

[pupun]

Hi All, Actually I am a newbie in this world of perl. I have been assigned a task where I need to write codes that will make the system to download a attached files from the browser automatically and will then save it into a directory. just like the crontab jobs we have in shell scripting. But here we are having a windows environment(windows 2000).please guide me. thanks in advnace

[marto]

Hi pupun,

The task you have been given sounds achievable using LWP's lwp-download or perhaps lwp-rget depending on the exact requirements you have.

I notice you have posted this question twice (as Reaped: downloading files from websites automatically in windows), please read the PerlMonks FAQ and How do I post a question effectively? if you have not already done so. Since you tell us that you are new to Perl perhaps you should start by working your way through the tutorials section of this site.

Hope this helps

Martin

[GrandFather]

LWP is the tool that is generally used. WWW::Mechanize gets quite a lot of use for that sort of thing too.

[Punitha]

Hi

Some of the discussions about this can help you, please refer

how to download files from the web
Downloading files from a Secured Site

Punitha

[kirillm]

Hi,

Use LWP::Simple to fetch files from the WWW easily. You could automatically save them to the local filesystem using the mirror() function. E.g. (this is right out of my head, not tested):

use LWP::Simple;

# An array of array refs whith remote url and local file
my @files = (["http://foo.bar/baz1.html", "C:/.../baz1.html"],
             ["http://bar.foo/zab2.html", "C:/.../zab2.html"]);

for my $duplet (@files) {
    mirror($duplet->[0], $duplet->[1]);
}

A more detailed/specific recipe will require that you tell more about what you are trying to achieve.

[pupun]

hi, thanks for ur help. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data(its a link specifying download data item) and then save it in a network drive. OS- windows 2000

[blazar]

I have been assigned a task where I need to write codes that will make the system to download files from the browser automatically and will then save it into a directory.

I personally believe that you just mean "to download files from the web" and in that case the answers you already got are perfect. But if you really mean "from the browser", in the acceptation of "from the web, by means of a browser," then the following modules may help:

• Win32::IE::Mechanize
• Mozilla::Mechanize

[pupun]

Thanks for your reply. Let me put it more clearly. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data and then save it in a network drive. OS- windows 2000

[Win]

pupun,

You've probably got all you need. However, if not, the following bit of code might help.

#!/usr/bin/perl

use warnings;
use strict;


use LWP::UserAgent;
use HTTP::Request::Common;


my @web_addresses = ("http://www.web1.com/page",
"http://www.web2.co.uk/page.htm");


my $count = 0;

foreach (@web_addresses) {

  $count++;

  my $ua = LWP::UserAgent->new;
  $ua->get("$_", ':content_file' => "$count");

}

I expect that the code may work without  use HTTP::Request::Common;. Not sure, you will have to try that out if you wish to know.

[pupun]

Thanks for your reply. Let me put it more clearly. actually, it's like this; there is a particular secured client site in that particular site through the script we have to give the user id and password and once we enter the site the script will access a partcular link and from which the script has to download data and then save it in a network drive. OS- windows 2000

[Jenda]

Let me put it more clearly. If you looked at the modules suggested by other monks you would find out that the task is fairly easy even if you have to login. You can either use WWW::Mechanize to navigate to the login page, fill in the form, submit it and then navigate to that "particular link" and save content. Or you can use LWP::UserAgent to send the login information to the same URL the login form would send it making sure the cookies sent with the response are remembered by the LWP::UserAgent object and then download that particular link.

But I admit it would require that you look at the documentation of those modules and try to do something yourself.

Show us you tried and come back if you have problems, but don't expect to get a full script doing exactly what you ask for. This is not a free script service.