Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

Appending new column data to CSV file?

by rnaeye (Pilgrim)
on Aug 04, 2013 at 15:12 UTC ( #1047802=perlquestion: print w/replies, xml ) Need Help??
rnaeye has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys, I have following code that takes multiple files, split fields, do simple arithmetic operation, and write results of each file into a separate output file. Now, I need to modify the code, and print the result of each input file as a seperate column in single output file (instead of multiple output files). I was wondering if you guys can give me a pointer. Thank you.

#!/usr/bin/perl use warnings; use 5.12.4; # USAGE: <perlscript> <*.txt> my (%hash, $key, $value); my ($col1, $col5, $sum, $rowdata, $line, $outfile, $infile, @column5); foreach my $file (@ARGV) { #name output file $outfile = $file . '.csv'; #file check die "Oops! A file called '$outfile' already exists.\n" if -e $outf +ile; open (IN, "<", "$file" ) or die "Cannot open file $!"; open (OUT, ">", "$outfile") or die "Cannot open file: $outfile + $!"; while (<IN>) { chomp ($line = $_); next if $line =~ /NoCoordinateCount.*/; next if $line =~ /^$/; ($col1, undef, undef, undef, $col5 ) = split(/\s/, $line); $sum += $col5; $key = $col1; $value = $col5; $hash{$key} = $value; } foreach (sort keys %hash) { print OUT $_, "," , $hash{$_}/$sum, "\n"; } $sum = 0; #reset sum for each file }

Replies are listed 'Best First'.
Re: Appending new column data to CSV file?
by moritz (Cardinal) on Aug 04, 2013 at 16:44 UTC

    Here is a pointer: Instead of printing the data, store it in a data structure that is declared outside the outer foreach loop. Then when you are done with all input files, write the output into a single file.

    If you haven't yet worked with nested data structures in Perl, I can recommend perlreftut, perldsc and perllol (the latter for arrays of arays).

Re: Appending new column data to CSV file?
by Cristoforo (Deacon) on Aug 04, 2013 at 20:47 UTC
    In the while loop, these 2 statements:
    $sum += $col5; push @numbers, $col5;
    $sum and @numbers need to be declared before the while loop that reads the file. Then after each file is read, statements:
    while (my ($i, $num) = each @numbers) { push @{ $output[$i] }, $num/$sum; }
    You would need to declare @output before the foreach loop begins.

    Then after all the files have been read and the info collected, you could print it out like: print OUT join(",", @$_), "\n" for @output;

Re: Appending new column data to CSV file?
by Laurent_R (Abbot) on Aug 04, 2013 at 19:06 UTC

    Instead of printing your hash values divided by $sum, change directly your hash and push a reference to it into a global array of hashes. Once you have readd all your files, use that data structure to print your data structure in the required format.

    $_ = $_/$sum foreach (values %hash); push @AoH, \%hash;

    It would probably be better to declare your hash within the "foreach my file (@ARGV)" loop to make sure you get a brand new and empty hash each time, but, on the other side, @AoH needs to be declared at the top of your program.

    At the end, you just need to go through the @AoH structure to print your output. (Of course, this is just one possibility, there are quite a few others.)

Re: Appending new column data to CSV file?
by sundialsvc4 (Abbot) on Aug 05, 2013 at 12:35 UTC

    Aside from suggesting Text::CSV, consider the very-useful join() function, which can do simple things like join("," 1, 2, 3); with ease.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1047802]
Approved by Old_Gray_Bear
robby_dobby adds fortune cookies to the platter on the sideboard.
robby_dobby adds raisin cookies to the platter on the sideboard.
robby_dobby takes a handful of cookies from the platter on the sideboard.

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (9)
As of 2017-04-26 13:29 GMT
Find Nodes?
    Voting Booth?
    I'm a fool:

    Results (477 votes). Check out past polls.