Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number

Re: Efficient way to sum columns in a file

by tlm (Prior)
on Apr 13, 2005 at 11:55 UTC ( #447331=note: print w/ replies, xml ) Need Help??

in reply to Efficient way to sum columns in a file

By testing eof() instead of eof in dave0's one-liner, you can extend its functionality to sum across multiple files. Like this:

# First we make to test files % perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i +n1.csv % perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i +n2.csv % head -3 in1.csv 94,434,249,267,649,367,572,579,498,369 452,735,420,543,832,198,28,86,67,382 801,339,978,859,85,719,758,89,191,377 # Now we run dave0's original script on them perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof' in1.csv in2 +.csv Sum is 23789 Sum is 48874 # Now let's try a small change --------------------------. # | # V perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof()' in1.csv i +n2.csv Sum is 48874
What happens is that eof (no parens!) returns true at the end of each file in @ARGV, while eof() is true only at the end of the last one (see eof). In the case of a single input file, the behavior is the same as before.

To get subtotals, you'd do

perl -lanF, -e '$total += $F[6]; $sub += $F[6]; print "Subtotal: $sub" + and $sub = 0 if eof; print "Total: $total" if eof()' in1.csv in2.csv Subtotal: 23789 Subtotal: 25085 Total: 48874

An alternative to testing for eof(), is to put everything in an END. e.g.

perl -lanF, -e '$sum += $F[6]; END{ print "Sum is $sum" }' in1.csv in2 +.csv

Update: Bug in last one-liner corrected.

the lowliest monk

Replies are listed 'Best First'.
Re^2: Efficient way to sum columns in a file
by Anonymous Monk on Apr 13, 2005 at 13:48 UTC
    I don't think the '$total += $sub += $F6;' part is doing what you expect it to... Actually, it doesn't add $F6 to both $total and $sub, but rather add it to $sub and then add $sub to $total, which generates a wrong total.

      The totals was obviously wrong; that should have alerted me to the error. Thank you very much!

      the lowliest monk

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://447331]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2016-08-27 15:20 GMT
Find Nodes?
    Voting Booth?
    The best thing I ever won in a lottery was:

    Results (380 votes). Check out past polls.