Re: Efficient way to sum columns in a file

[tlm]

in reply to Efficient way to sum columns in a file

By testing eof() instead of eof in dave0's one-liner, you can extend its functionality to sum across multiple files. Like this:

# First we make to test files
% perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i
+n1.csv
% perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i
+n2.csv
% head -3 in1.csv
94,434,249,267,649,367,572,579,498,369
452,735,420,543,832,198,28,86,67,382
801,339,978,859,85,719,758,89,191,377

# Now we run dave0's original script on them
perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof' in1.csv in2
+.csv
Sum is 23789
Sum is 48874

# Now let's try a small change --------------------------.
#                                                        |
#                                                        V
perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof()' in1.csv i
+n2.csv
Sum is 48874

What happens is that eof (no parens!) returns true at the end of each file in @ARGV, while eof() is true only at the end of the last one (see eof). In the case of a single input file, the behavior is the same as before.

To get subtotals, you'd do

perl -lanF, -e '$total += $F[6]; $sub += $F[6]; print "Subtotal: $sub"
+ and $sub = 0 if eof; print "Total: $total" if eof()' in1.csv in2.csv
Subtotal: 23789
Subtotal: 25085
Total: 48874

An alternative to testing for eof(), is to put everything in an END. e.g.

perl -lanF, -e '$sum += $F[6]; END{ print "Sum is $sum" }' in1.csv in2
+.csv

Update: Bug in last one-liner corrected.

[Anonymous Monk]

I don't think the '$total += $sub += $F6;' part is doing what you expect it to... Actually, it doesn't add $F6 to both $total and $sub, but rather add it to $sub and then add $sub to $total, which generates a wrong total.

[tlm]

The totals was obviously wrong; that should have alerted me to the error. Thank you very much!