Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Perl Module Not Working In Crontab

by dailyoliver (Initiate)
on Jan 05, 2012 at 22:13 UTC ( #946486=perlquestion: print w/ replies, xml ) Need Help??
dailyoliver has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks - a little help here would be highly valued..

OS NAME: Red Hat Enterprise Linux 5 RHEL5

Perl version: v5.8.8 built for x86_64-linux-thread-multi

Module Version: Archive::Extract v?

Situation: A bash script 'my_bash.txt' invokes my_perl.pl - both scripts enclosed.

TWO Problems:

1. If I don't manually invoke 'my_bash.txt' from the same dir that it resides in - for example /usr/my_dir/my_bash.txt -

the Archive::Extract module in /usr/my_dir/my_perl.pl extracts the output to the root folder '/' and not the correct working directory /usr/my_dir/.

2. If I invoke /usr/my_dir/my_bash.txt via root crontab - the Archive::Extract module will not work.

Please could someone help as I have been at this maddening process for days..

BASH SCRIPT BELOW (this invokes my_perl.pl)

#!/bin/sh #call perl script to download MODIS greening each decad /usr/bin/perl /work/locust1/swarmsdata/RasterData/greening/modis_green +ing.pl

PERL SCRIPT BELOW (is called by bash script above)

#!/usr/bin/perl use Archive::Extract; $Archive::Extract::PREFER_BIN=1; #if $inputZip exists unzip, store file correctly and remove date from +dekadalGet.txt if (-e $inputZip) { #unzip file print $timeStamp.' -New zip file successfully downloaded, unzipping fi +le: '."\n"."\n"; # create an archive object my $archive = Archive::Extract->new(archive => $inputZip); #extract the contents to the current working directory $archive->extract() or die "Can't extract: " . $archive->error; print $timeStamp.' -New zip file = '.$inputZip.' for '.$j.' successful +ly unzipped -> deleting file.'."\n"."\n"; #delete the zip file unlink($inputZip) or die "Zip file cannot be deleted."; unless (-e $inputZip) { print $timeStamp.' -New zip file successfully deleted: '."\n"."\n"; }

Comment on Perl Module Not Working In Crontab
Select or Download Code
Re: Perl Module Not Working In Crontab
by Anonymous Monk on Jan 05, 2012 at 22:23 UTC
Re: Perl Module Not Working In Crontab
by ww (Bishop) on Jan 05, 2012 at 23:24 UTC
    Don't know, but line 20 of your shell script looks fishy to me. It doesn't invoke the Perl executable; merely references its location, so that part is either redundant or wrong.

    And despite your assertion that the modis_greening.pl runs but fails to use the module, I have to wonder if you have CHMODed your Perl script appropriately. Crontab won't be running under your id and you haven't cited any evidence that "the script works," whatever that means when you're telling us it doesn't do what you intended.

    Note that I haven't even looked closely at most of modis_greening.pl. As the advice on posting questions notes, post a minimal sample that reproduces the problem. 400 lines, even with generous whitespace is more than I'll bother to download, absent some far more intriguing problem description. The prior respondent may be onto sometime, but that suggests s/he did at least run your script against perl -c ....

Re: Perl Module Not Working In Crontab
by i5513 (Monk) on Jan 06, 2012 at 00:49 UTC
Re: Perl Module Not Working In Crontab
by graff (Chancellor) on Jan 06, 2012 at 07:46 UTC
    The two key suggestions above that are most important: (1) create a minimal script that focuses on the main problem (what differs between interactive use and cron-job use), and (2) use strict.

    Apart from that, your script suffers from too much repetition of path information, ugly use of "." for string concatenation where a single double-quoted string will do, too many lines of code that could be removed by using a couple more core modules (e.g. POSIX, which provides the very handy "strftime()" function), and probably some errors that you aren't aware of.

    For example, you have some lines like this:

    $geoTiff_delete_all = </work/locust1/swarmsdata/RasterData/greening/*. +tif>; $zip_delete_all = </work/locust1/swarmsdata/RasterData/greening/*.zip> +;
    Are you aware that if there are many *.tif and *.zip files, only one file name will be assigned to each of those variables? Is that your intention? Try this set of shell commands to see what I'm talking about:
    cd /tmp touch foo1.foo foo2.foo foo3.foo perl -le '$f=<*.foo>; print "scalar: $f"; @f=<*.foo>; print "array: @f +"'

    UPDATE: Just to give you an idea of how coding style relates to ease of coding and maintenance, the following snippet of 48 lines should be functionally the same as (or better than) the first 186 lines of the OP perl script; I'm using strictures, consistent indentation, simplest possible quoting, no repetition of literal strings, fewer variables, no redundant comments, and a "for" loop that eliminates several lines of repeated code:

    I stopped at line 186 in your script ("Starting ... collection procedure") because after that point it was less clear to me what you were really trying to do (i.e. I'm not sure the OP code was actually doing what you intended). But I would expect you could get a similar (4-to-1) reduction for the rest of the script.

Re: Perl Module Not Working In Crontab
by dailyoliver (Initiate) on Jan 09, 2012 at 18:34 UTC
    Thank you all for your kind generosity and help - I have made the code a bit clearer and ONLY add the bits that are not working - basically the Perl module Archive::Extract works manually but will not work via crontab - there has been mention of environmental variables but I am not sure what I need to do here - any further suggestions with the simplified code would be great! thanks to you all-
Re: Perl Module Not Working In Crontab
by dailyoliver (Initiate) on Jan 11, 2012 at 22:26 UTC

    Thank you all for your comments -

    The Archive::Extract module works fine if invoked manually. However it fails to extract using crontab.

    As suggested I have turned debugging on $Archive::Extract::DEBUG=1;

    <BELOW> are the results - would anybody have an idea what this means? I see the term 'write error (disk full?)' comes up several times??? Any help would be appreciated.. thanks again..

    # Extracting with ->_unzip_bin Running [/usr/bin/unzip -Z -1 /work/locust1/swarmsdata/RasterData/gree +ning/MCD_GreenArea_20111211_LocustArea.zip]... MCD_GreenArea_20111211_LocustArea.tif Running [/usr/bin/unzip -qq -o /work/locust1/swarmsdata/RasterData/gre +ening/MCD_GreenArea_20111211_LocustArea.zip]... MCD_GreenArea_20111211_LocustArea.tif: write error (disk full?). Con +tinue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated Unable to unzip '/work/locust1/swarmsdata/RasterData/greening/MCD_Gree +nArea_20111211_LocustArea.zip': MCD_GreenArea_20111211_LocustArea.tif +: write error (disk full?). Continue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 1106 Archive::Extract::_unzip_bin('Archive::Extract=HASH(0x1f96b990)') +called at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 415 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 # Extraction method failed # Extracting with ->_unzip_az IO error: write error during copy : No space left on device at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 Extraction of 'MCD_GreenArea_20111211_LocustArea.tif' from '/work/locu +st1/swarmsdata/RasterData/greening/MCD_GreenArea_20111211_LocustArea. +zip' failed at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line + 1166 Archive::Extract::_unzip_az('Archive::Extract=HASH(0x1f96b990)') c +alled at /usr/lib/perl5/site_perl/5.8.8/Archive/Extract.pm line 415 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 # Extraction method failed Extract failed due to errors at /usr/lib/perl5/site_perl/5.8.8/Archive +/Extract.pm line 439 Archive::Extract::extract('Archive::Extract=HASH(0x1f96b990)') cal +led at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl + line 340 at /work/locust1/swarmsdata/RasterData/greening/modis_greening.pl lin +e 340 Can't extract: Unable to unzip '/work/locust1/swarmsdata/RasterData/gr +eening/MCD_GreenArea_20111211_LocustArea.zip': MCD_GreenArea_20111211 +_LocustArea.tif: write error (disk full?). Continue? (y/n/^C) warning: MCD_GreenArea_20111211_LocustArea.tif is probably truncated Extraction of 'MCD_GreenArea_20111211_LocustArea.tif' from '/work/locu +st1/swarmsdata/RasterData/greening/MCD_GreenArea_20111211_LocustArea. +zip' failed Extract failed due to errors at /work/locust1/swarmsdata/RasterData/gr +eening/modis_greening.pl line 340.

      What do you think this message is trying to tell you

      MCD_GreenArea_20111211_LocustArea.tif: write error (disk full?). Continue? (y/n/^C)

      ...

      IO error: write error during copy : No space left on device

      Are you ready to stop blaming Perl Module Archive::Extract?

        Thanks you for your response - I eventually found out that I needed to add 'cd /my/working/directory/' into the bash script. This was a privilege issue - Archive::Extract module could not extract the file to the required directory and adding 'cd ...' to the calling script managed to solve it! Thank you to the guy that mentioned TURN ON DEBUG

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://946486]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (10)
As of 2014-09-19 20:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (145 votes), past polls