Opening multiple files

[mcasspj]

Back to Perl after a long, too long break! I wish to make a csv data file (for R) and I have a series of files consisting of lists of single items. Rather than open the first file, outputing the first line to a new (output) file then opening the second file, reading the first item and appending it to the first line of the output file and so on until I run out of input files then doing the same for the second line I'm sure there must be a more elegant way. I know the files all have the same number of lines so don't need to worry about this. Somehow I'm guessing the solution uses
while (<>) {}
but not sure how to handle the handles (no pun intended!) Cheers Paul

[Eliya]

You could do something like this:

#!/usr/bin/perl -w
use strict;

my @fh;
# open all files
for my $i (0..$#ARGV) {
    open $fh[$i], "<", $ARGV[$i] or die $!;
}

while (1) {
    my @lines;
    # read one line from each file
    push @lines, scalar readline $fh[$_] for 0..$#ARGV;
    
    last unless defined $lines[0];
    chomp @lines;
    print join(",", @lines), "\n";
}

$ ./950649.pl infile1 infile2 infile3 ... > outfile

(Note that you have to use readline with "complex" file handle expressions like $fh[$i], as <$fh[$i]> wouldn't work here.)

[kielstirling]

Hi,

Maybe I'm still asleep but, will this ever return out of the while loop?

-Kiel R Stirling

[Eliya]

will this ever return out of the while loop?

Yes, due to the

    last unless defined $lines[0];

where $lines[0] becomes undef when EOF of the first file has been reached.

[kielstirling]

Hi,

If I understand correctly what you are wanting to do .. this is how I would do it.

Have fun !!

-Kiel R Stirling

#!/usr/bin/perl -w

use strict;
use IO::File;

my $output = pop @ARGV;
my @merge;
for my $file (@ARGV) {
        my $input_fh = IO::File->new($file);
        die "failed to open $file\n"
                unless defined $input_fh;
        $merge[$.-1] .= $_
                while (defined ($_ = <$input_fh>) and chomp);
        $input_fh->close;
}

my $output_fh = IO::File->new("> $output");
die "failed to open $output\n"
        unless defined $output_fh;

$" = "\n";
print $output_fh "@merge";
$output_fh->close;

./merge.pl file1 file2 file3 output_file

[mcasspj]

That's exactly what I wanted, just tweaked it and it's perfect!

Many Thanks Paul

[Anonymous Monk]

Um, paste -d, one two three four > onetofour.csv

$ perl -e " for my $l ( a..c ){ open $fh, '>', $l; print $fh qq[$l$_\n
+] for 1 .. 3; } "

$ paste -d, a b c
a1,b1,c1
a2,b2,c2
a3,b3,c3

[Anonymous Monk]

#!/usr/bin/perl --
use strict; use warnings;
use Text::CSV;
use autodie qw/ open close /;
Main( @ARGV );
exit( 0 );
sub Main {
    return Usage() unless @_ > 2;
    
    my( $outfile, @infiles ) = @_;
    my $csv_out = Text::CSV->new( {
        always_quote => 1,
        binary       => 1,
        eol          => $/,
    } ) or die Text::CSV->error_diag();
    
    open $outfile, '>:raw', $outfile; # autodie
    
    $csv_out->print( $outfile, \@infiles ) or die $csv_out->error_diag
+ ;
    
    @infiles = map { open my $fh, '<:raw', $_; $fh  } @infiles;
    
    while( not( grep eof, @infiles ) ){
        my @lines = map { scalar readline( $_ ) } @infiles;
#~         chomp( @lines );
#~         no warnings 'uninitialized';
#~         s/[\r\n]+$// for @lines;
        s/[\r\n]+$// for grep defined, @lines;
        $csv_out->print( $outfile, \@lines ) or die $csv_out->error_di
+ag ;
    }
    
    undef $csv_out;
    close $outfile; # autodie
} ## end sub Main


sub Usage {
    print <<"__USAGE__";

$0    
$0    outFile.csv   inOne.csv inTwo.csv inThree.csv ...


EXAMPLE SESSION

    \$ perl csv.paste.pl    out.csv  ta.csv  tb.csv  tc.csv  td.csv

    \$ cat out.csv
    "ta.csv","tb.csv","tc.csv","td.csv"
    "a1","b1","c1","d1""st(i)n,ker"""
    "a2","b2","c2","d2'st(i)n,ker'"
    "a3","b3","c3","d3""'st(i)n,ker'"""
    ,,,"d4 stinker unquoted "

__USAGE__
} ## end sub Usage

[Anonymous Monk]

Correction

-  while( not( grep eof, @infiles ) ){

+  while( not( grep \&eof, @infiles ) ){

[mcasspj]

Cheers, the reasons I didn't want to use paste are:
1. The people who may use this wont be on linux/unix boxes or have cygwin.
2. I want to include some pre-processing in the program, it's Russian Election Data scraped from the web and needs a little cleaning i.e. numbers are formatted with spaces.
3. To learn some Perl.
Many Thanks
Paul