Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Reports Downloading

by aartist (Scribe)
on Jul 18, 2011 at 16:27 UTC ( #915218=perlquestion: print w/ replies, xml ) Need Help??
aartist has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
This is not stictly a Perl question, but I like to use Perl to do this if possible. have to download the CSV reports based on input parameters in secure environment. What are the good ways for this task? Currently I am doing it manually.
Thanks.

Comment on Reports Downloading
Re: Reports Downloading
by jethro (Monsignor) on Jul 18, 2011 at 16:43 UTC
    Well, you don't give much detail. Take a look at Text::CSV for CSV handling.
      Here, report downloading has to do with going to webserver, filling the appropriate parameters etc.., rather than reading CSV files.
Re: Reports Downloading
by zentara (Archbishop) on Jul 18, 2011 at 17:52 UTC
    download the CSV reports based on input parameters in secure environment.

    What do you mean secure environment? Does that mean you are behind a firewall on a secure intranet?, or using https?, or do you need to setup the secure channel yourself?

    If you know where the file is on the remote filesystem, and can use ssh, Net::SSH2 might be useful, and it has a builtin sftp command, so your file is encrypted over the network. Google has many Perl ssh examples free for a search.

    If you are going over a web browser, you can use Mechanize as mentioned above, or if the link is known, you probably can use the simpler LWP::Simple, and here is a generic example that includes any html auth you might need.

    #!/usr/bin/perl -w #adapted from lwpcook use strict; use LWP::UserAgent; my $user = 'mini_me'; my $pass = 'foobar'; my $ua = LWP::UserAgent->new; my $URL = 'https://your_target_site.com/CSV/latest.csv'; my $filename = substr( $URL, rindex( $URL, "/" ) + 1 ); #print "$filename\n"; open( IN, ">$filename" ) or die $!; print "Fetching $URL\n"; my $expected_length; my $bytes_received = 0; my $req = HTTP::Request->new(GET => $URL); $req->authorization_basic($user, $pass); my $res = $ua->request($req, sub { my ( $chunk, $res ) = @_; $bytes_received += length($chunk); unless ( defined $expected_length ) { $expected_length = $res->content_length || 0; } if ($expected_length) { printf STDERR "%d%% - ", 100 * $bytes_received / $expected +_length; } print STDERR "$bytes_received bytes received\n"; # XXX Should really do something with the chunk itself print IN $chunk; } ); print $res->status_line, "\n"; close IN; exit;

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://915218]
Approved by Marshall
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (14)
As of 2014-08-01 12:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (18 votes), past polls