http://www.perlmonks.org?node_id=1147668

fasoli has asked for the wisdom of the Perl Monks concerning the following question:

Hello wise monks,

I have been trying to write a foreach loop in order to untar multiple tar.gz files that I (will) have in a directory. The script I've written is

#/bin/perl/ use strict; use warnings; # First, setting the directories we're interested in. my $inputs = "~/inputs"; # inputs we upload to the server. my $results = "~/results"; # results coming in. my @array; # tarballs of the incoming results. chomp (@array = system "ls $results\/*.tar.gz"); # putting all tar fil +es in an array # tar can't accept multiple tar.gz files as input # arguments after the tarball filename are expected to be files to unt +ar from the tarball. foreach (@array) { system "tar -tvf $results\/$_"; }

And this is the output I'm getting:

/home/user/results/results_back.tar.gz (my comment -> that's the ls c +ommand output, so it is finding one tar.gz file, I only have one for +now) tar: 0: Cannot open: No such file or directory tar: Error is not recoverable: exiting now

So there seems to be something wrong but I can't find what. As you can see, for now I'm only listing the tar.gz contents, if I get it working I'll add the real extract command.

Have I made a very obvious mistake? It would be great if you could give me a hint about what might be happening.

Thank you! :) I'd really appreciate any help :)

Replies are listed 'Best First'.
Re: foreach loop to untar multiple files
by Paladin (Vicar) on Nov 13, 2015 at 16:25 UTC
    system doesn't return the output of the command run, it returns the exit code of what's run, in your case 0 hence the 0 in your output "tar: 0: Cannot open: No such file or directory". Take a look at the glob() perl command to get the listing of files in a directory.
Re: foreach loop to untar multiple files
by toolic (Bishop) on Nov 13, 2015 at 16:26 UTC
    The problem is that the output of system is the exit value of the ls command, not the output of the ls command. You could use qx to capture the ls command output.

    Another approach is to use glob:

    my $results = "~/results"; # results coming in. for (glob "$results/*.tar.gz") { system "tar -tvf $_"; }
Re: foreach loop to untar multiple files
by hippo (Bishop) on Nov 13, 2015 at 16:28 UTC
    chomp (@array = system "ls $results\/*.tar.gz");

    The system function does not return its output - only its exit code. You probably want to use backticks instead. eg:

    chomp (@array = `ls ${results}/*.tar.gz`);

    Update: obviously a slow typing day for me :-) As my learned brethren have pointed out there are much better ways of doing this than shelling out at all.

Re: foreach loop to untar multiple files
by stevieb (Canon) on Nov 13, 2015 at 16:37 UTC

    Other Monks have pointed out the errs in your ways, so I won't go there, I'll just expand... I would personally use Archive::Extract so your code doesn't have to shell out. Said module has been part of perl's core since pre-v5.10. Also note that I use a file glob (<*.ext>) to slurp in the file names, which retain their full path information (this eliminates another system call (ls)). Here's an example:

    use warnings; use strict; use Archive::Extract; my $inputs = "~/inputs"; my $results = "/home/steve/results"; my @files = <$inputs/*.tar.gz>; for (@files){ my $ae = Archive::Extract->new(archive => $_); $ae->extract(to => $results) or die "couldn't decompress the damned $_ file!: " . $ae->error; print "decompressed $_\n"; }

    For some reason, extract() wouldn't recognize the homedir shortcut, so I just entered in the full path. I'm sure it can expand it somehow, I just don't have the time to troubleshoot.

Re: foreach loop to untar multiple files
by fasoli (Beadle) on Nov 13, 2015 at 17:09 UTC

    Thank you all, I will be doing the relevant reading (on glob) and modifying the script accordingly.

    I've already copied and tried this

    my $results = "~/results"; # results coming in. for (glob "$results/*.tar.gz") { system "tar -tvf $_"; }

    ...and it does what I want.

    I've tried the following modification in my script though, and something funny happens again.

    foreach (@tar) { `tar -tvf $results\/$_`; }

    This gives me the following error:

    tar: /home/user/results//home/user/results/results_back.tar.gz: Cannot open: No such file or directory

    So the path to the tar.gz file is messed up obviously. But then my hopes build up because I see that it finds the tar.gz file and that's when I stupidly try to be "clever" by removing $results completely from the command and only keeping $_. In that case I'm just getting nothing, it fails quietly. Why is that?

      why don't you print out $_ and see what it really contains?

      Also, see Basic debugging checklist to find out how to debug your code when it goes wrong.

      Carefully look at your output... the file path is being specified twice... tar: /home/user/results//home/user/results/results_back.tar.gz. Try this:

      for (@tar){ print `tar -tvf $_`; }

      ps. I'd recommend you look at my Archive::Extract example here, to avoid having to shell out.

        for (@tar){ print `tar -tvf $_`; }

        Quite dangerous. Imaginge a file named "foo ; rm -rf * ; bar.tar.gz", injected into @tar by ls or glob. (And yes, that's a legal filename.)

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

        Thanks for your answer. I noticed that the file path is specified twice, that's why I said that I tried to delete $results and only go for $_. However this was obviously wrong as it hangs for a few seconds and fails quietly and I was wondering if anybody knew why. The same happens with the script you provided :(

        I will look at the Extract example in more detail, it just scared me a bit as it looked a little more complicated.