Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Appending new column data to CSV file?

by rnaeye (Friar)
on Aug 04, 2013 at 15:12 UTC ( #1047802=perlquestion: print w/replies, xml ) Need Help??
rnaeye has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys, I have following code that takes multiple files, split fields, do simple arithmetic operation, and write results of each file into a separate output file. Now, I need to modify the code, and print the result of each input file as a seperate column in single output file (instead of multiple output files). I was wondering if you guys can give me a pointer. Thank you.

#!/usr/bin/perl use warnings; use 5.12.4; # USAGE: <perlscript> <*.txt> my (%hash, $key, $value); my ($col1, $col5, $sum, $rowdata, $line, $outfile, $infile, @column5); foreach my $file (@ARGV) { #name output file $outfile = $file . '.csv'; #file check die "Oops! A file called '$outfile' already exists.\n" if -e $outf +ile; open (IN, "<", "$file" ) or die "Cannot open file $!"; open (OUT, ">", "$outfile") or die "Cannot open file: $outfile + $!"; while (<IN>) { chomp ($line = $_); next if $line =~ /NoCoordinateCount.*/; next if $line =~ /^$/; ($col1, undef, undef, undef, $col5 ) = split(/\s/, $line); $sum += $col5; $key = $col1; $value = $col5; $hash{$key} = $value; } foreach (sort keys %hash) { print OUT $_, "," , $hash{$_}/$sum, "\n"; } $sum = 0; #reset sum for each file }

Replies are listed 'Best First'.
Re: Appending new column data to CSV file?
by moritz (Cardinal) on Aug 04, 2013 at 16:44 UTC

    Here is a pointer: Instead of printing the data, store it in a data structure that is declared outside the outer foreach loop. Then when you are done with all input files, write the output into a single file.

    If you haven't yet worked with nested data structures in Perl, I can recommend perlreftut, perldsc and perllol (the latter for arrays of arays).

Re: Appending new column data to CSV file?
by Cristoforo (Curate) on Aug 04, 2013 at 20:47 UTC
    In the while loop, these 2 statements:
    $sum += $col5; push @numbers, $col5;
    $sum and @numbers need to be declared before the while loop that reads the file. Then after each file is read, statements:
    while (my ($i, $num) = each @numbers) { push @{ $output[$i] }, $num/$sum; }
    You would need to declare @output before the foreach loop begins.

    Then after all the files have been read and the info collected, you could print it out like: print OUT join(",", @$_), "\n" for @output;

Re: Appending new column data to CSV file?
by Laurent_R (Canon) on Aug 04, 2013 at 19:06 UTC

    Instead of printing your hash values divided by $sum, change directly your hash and push a reference to it into a global array of hashes. Once you have readd all your files, use that data structure to print your data structure in the required format.

    $_ = $_/$sum foreach (values %hash); push @AoH, \%hash;

    It would probably be better to declare your hash within the "foreach my file (@ARGV)" loop to make sure you get a brand new and empty hash each time, but, on the other side, @AoH needs to be declared at the top of your program.

    At the end, you just need to go through the @AoH structure to print your output. (Of course, this is just one possibility, there are quite a few others.)

Re: Appending new column data to CSV file?
by sundialsvc4 (Abbot) on Aug 05, 2013 at 12:35 UTC

    Aside from suggesting Text::CSV, consider the very-useful join() function, which can do simple things like join("," 1, 2, 3); with ease.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1047802]
Approved by Old_Gray_Bear
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (9)
As of 2018-11-13 19:56 GMT
Find Nodes?
    Voting Booth?
    My code is most likely broken because:

    Results (158 votes). Check out past polls.