Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re: Efficient way to sum columns in a file

by tlm (Prior)
on Apr 13, 2005 at 11:55 UTC ( #447331=note: print w/replies, xml ) Need Help??

in reply to Efficient way to sum columns in a file

By testing eof() instead of eof in dave0's one-liner, you can extend its functionality to sum across multiple files. Like this:

# First we make to test files % perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i +n1.csv % perl -le 'print join(",", map int(rand(1000)), 1..10) for 1..50' > i +n2.csv % head -3 in1.csv 94,434,249,267,649,367,572,579,498,369 452,735,420,543,832,198,28,86,67,382 801,339,978,859,85,719,758,89,191,377 # Now we run dave0's original script on them perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof' in1.csv in2 +.csv Sum is 23789 Sum is 48874 # Now let's try a small change --------------------------. # | # V perl -lanF, -e '$sum += $F[6]; print "Sum is $sum" if eof()' in1.csv i +n2.csv Sum is 48874
What happens is that eof (no parens!) returns true at the end of each file in @ARGV, while eof() is true only at the end of the last one (see eof). In the case of a single input file, the behavior is the same as before.

To get subtotals, you'd do

perl -lanF, -e '$total += $F[6]; $sub += $F[6]; print "Subtotal: $sub" + and $sub = 0 if eof; print "Total: $total" if eof()' in1.csv in2.csv Subtotal: 23789 Subtotal: 25085 Total: 48874

An alternative to testing for eof(), is to put everything in an END. e.g.

perl -lanF, -e '$sum += $F[6]; END{ print "Sum is $sum" }' in1.csv in2 +.csv

Update: Bug in last one-liner corrected.

the lowliest monk

Replies are listed 'Best First'.
Re^2: Efficient way to sum columns in a file
by Anonymous Monk on Apr 13, 2005 at 13:48 UTC
    I don't think the '$total += $sub += $F6;' part is doing what you expect it to... Actually, it doesn't add $F6 to both $total and $sub, but rather add it to $sub and then add $sub to $total, which generates a wrong total.

      The totals was obviously wrong; that should have alerted me to the error. Thank you very much!

      the lowliest monk

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://447331]
[stevieb]: I found that win10 broke a C# library I was using for one project while enhancing tests for a Perl dist, which breaks other Perl dists, and I'm about to throw my hands up on berrybrew. win2k12 broke one thing, win10 breaks something...
[stevieb]: ...unrelated which requires replacing a lot of code and a whole lib. I'm about to go nix only ffs
[shmem]: stevieb: what you're doing sounds afwully complex. Too much for me this evening to provide brighter insight ;-)
[stevieb]: I don't even own a Windows computer. Both my girl and I have a laptop each with Linux. I'm supporting Windows in some of my projects and I can't even guage whether it's worth it or not.
[stevieb]: shmem It's something I desired to have years ago, which is why I took over berrybrew. Cross-platform build/test automation locally, or over the network Test::BrewBuild
[shmem]: sounds good.
[shmem]: but I'm crumbling smaller stones. remember...
[stevieb]: I'm working on it to fatten it up and make it more reliant so I can finalize my Raspberry Pi automated build system for that software :) It's all well and fun, until I try to make it work with Windows lol
[shmem]: "debugging a program is more difficult than to write it in the first place. If you code your program as smart as you are, you are, by definition, too dumb to debug it."

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2017-03-28 22:07 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (342 votes). Check out past polls.