Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

sumof - attempting to sum a column from each file

by ricky5ive (Initiate)
on Sep 14, 2012 at 15:30 UTC ( #993745=perlquestion: print w/ replies, xml ) Need Help??
ricky5ive has asked for the wisdom of the Perl Monks concerning the following question:

Hello all - I'm fairly new to Perl, I've been learning for the last year. I'm getting confused by this script I've been working on and was looking to see if someone can help me. I want to sum every file. I wrote this script originally using one specific file and it worked flawless, but now I want it to do this with each file in a specific directory and return the sum for each file for the specific column. First script that only uses one file:
open (FILE ,"<", "Myfile"); while($lines=<FILE>) { ###-reads each line from the file and splits it by multiple spaces @kbused = split(/\s+/,$lines); push(@kilo, $kbused[2]); } ####-takes the number from column2 and removes the KB from the end foreach $aline (@kilo) { $aline =~ s/\D//g; push (@onlynums, $aline); } $sumof += $_ for @onlynums; print "$sumof\n"; close FILE;
The script I'm working on now that will do it for each file in the directory:
@allfiles = <C:/Users/user/Desktop/DOwork/filez/nabillingscript/09_14_ +2012/nas/*>; foreach $file (@allfiles) { open (FILE ,"<", "$file"); while($lines=<FILE>) { my $sumof = 0; @kbused = split(/\s+/,$lines); push(@kilo, $kbused[2]); foreach $aline (@kilo) { $aline =~ s/\D//g; push (@onlynums, $aline); $sumof += $_ for @onlynums; } } print "$sumof\n"; } close FILE;
I can't figure this out on my own for some reason, but I can't learn without asking for help. Thank you.

Comment on sumof - attempting to sum a column from each file
Select or Download Code
Re: sumof - attempting to sum a column from each file
by choroba (Abbot) on Sep 14, 2012 at 15:52 UTC
    You are resetting and redeclaring $sumof inside the loop. Therefore, it cannot hold the sum value when you print it.
    You can use strict and warnings to make Perl guard these (and many more) errors.
    لսႽ ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
      I think I'm going to rewrite my script from scratch to get a better hang of the suggestions and see the whole picture as I write it. I appreciate all your input, as this has greatly helped me.
Re: sumof - attempting to sum a column from each file
by roboticus (Canon) on Sep 14, 2012 at 15:53 UTC

    ricky5ive:

    You're setting your count to 0 after each line read. You want to do it only once per file. So move the my $sumof=0; to just before or after the open statement.

    You're also summing up the items *while you're still reading the file*. Finish reading the lines before doing the summing operation.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Re: sumof - attempting to sum a column from each file
by Rudolf (Monk) on Sep 14, 2012 at 15:53 UTC

    $sumof needs to be outside of the while loop otherwise it will keep getting reset to zero and you should be using strict and warnings or even better in my opinion a v string (v5.14). Use local variables.. that is my $x; my @z;

    hope this helps, - Rudolf

    use v5.14; my @allfiles = <C:/Users/user/Desktop/DOwork/filez/nabillingscript/09_ +14_2012/nas/*>; for my $file (@allfiles) { open (FILE ,'<', $file) or die $!; my $sumof = 0; while(<FILE>){ my $lines = $_; my @kbused = split(/\s+/,$lines); push(my @kilo, $kbused[2]); for my $aline (@kilo){ $aline =~ s/\D//g; push (my @onlynums, $aline); $sumof += $_ for @onlynums; } } print "$sumof\n"; } close FILE;
      This code you posted works perfectly. Appreciate you taking the time!

        No problem at all, you should consider learning more Perl if you have the time - it is a valuable investment. Not only a powerful tool but a fun artistic language! good luck

Re: sumof - attempting to sum a column from each file
by frozenwithjoy (Curate) on Sep 14, 2012 at 15:55 UTC
    Well, there are issues with your second script that others have pointed out, but why not just put the first one (which you say works) inside a loop that loops through each file?
    #!/usr/bin/env perl use strict; use warnings; use autodie; @allfiles = <C:/Users/user/Desktop/DOwork/filez/nabillingscript/09_14_ +2012/nas/*>; foreach $file (@allfiles) { open (FILE ,"<", "Myfile"); ###-reads each line from the file and splits it by multiple spaces while($lines=<FILE>) { @kbused = split(/\s+/,$lines); push(@kilo, $kbused[2]); } ####-takes the number from column2 and removes the KB from the end foreach $aline (@kilo) { $aline =~ s/\D//g; push (@onlynums, $aline); } $sumof += $_ for @onlynums; print "$sumof\n"; close FILE; }
      Thank you very much. I'm going to review each of your responses and soak it all in and then reply with more.
      Using strict and warnings has a lot of issues with my variable names. Not something I usually use, but obviously I need to get into the practice of.
        You should just always use them. Even if you rarely write Perl, they will probably save you hours of your life.
Re: sumof - attempting to sum a column from each file
by CountZero (Bishop) on Sep 14, 2012 at 20:48 UTC
    Can you show us a few lines from your datafiles? I think that your program can be written much shorter.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

    My blog: Imperial Deltronics
      It's actually output from a Netapp. I like the idea of simpler ... helps to learn the ways of the jedi.
      <df -k> Filesystem total used avail capacity Mounted + on /vol/vol0/ 142606336KB 5274244KB 137332092KB 4% /vol/v +ol0/ /vol/vol0/.snapshot 35651584KB 923524KB 34728060KB 3% /vol/vo +l0/.snapshot /vol/vobstore/ 734003200KB 6335232KB 727667968KB 1% /vol/ +vobstore/ /vol/vobstore/.snapshot 183500800KB 260200KB 183240600KB 0% / +vol/vobstore/.snapshot /vol/ccase/ 125829120KB 21814420KB 104014700KB 17% /vol/ +ccase/ /vol/ccase/.snapshot 31457280KB 90572KB 31366708KB 0% /vol/c +case/.snapshot /vol/viewstore/ 943718400KB 6492604KB 937225796KB 1% /vol/ +viewstore/ /vol/viewstore/.snapshot 235929600KB 137500KB 235792100KB 0% +/vol/viewstore/.snapshot /vol/viewstore_win/ 67108864KB 480KB 67108384KB 0% /vol/vi +ewstore_win/ /vol/viewstore_win/.snapshot 16777216KB 2372KB 16774844KB 0% + /vol/viewstore_win/.snapshot </df -k>
        Try this for your summing code:
        use Modern::Perl; my $sum_of_used; { no warnings qw/numeric uninitialized/; $sum_of_used += (split /\s+/)[2] while <DATA>; } say $sum_of_used; __DATA__ <df -k> Filesystem total used avail capacity Mounted + on /vol/vol0/ 142606336KB 5274244KB 137332092KB 4% /vol/v +ol0/ /vol/vol0/.snapshot 35651584KB 923524KB 34728060KB 3% /vol/vo +l0/.snapshot /vol/vobstore/ 734003200KB 6335232KB 727667968KB 1% /vol/ +vobstore/ /vol/vobstore/.snapshot 183500800KB 260200KB 183240600KB 0% / +vol/vobstore/.snapshot /vol/ccase/ 125829120KB 21814420KB 104014700KB 17% /vol/ +ccase/ /vol/ccase/.snapshot 31457280KB 90572KB 31366708KB 0% /vol/c +case/.snapshot /vol/viewstore/ 943718400KB 6492604KB 937225796KB 1% /vol/ +viewstore/ /vol/viewstore/.snapshot 235929600KB 137500KB 235792100KB 0% +/vol/viewstore/.snapshot /vol/viewstore_win/ 67108864KB 480KB 67108384KB 0% /vol/vi +ewstore_win/ /vol/viewstore_win/.snapshot 16777216KB 2372KB 16774844KB 0% + /vol/viewstore_win/.snapshot </df -k>
        It can be so simple because Perl knows how to make a number out of "almost" numbers. In other words, Perl simply disregards the 'KB' which is attached to the number. Everything which does not contain numbers will become 0, such as the header line. A perfect example of Perlish "Do What I Mean".

        Of course I had to switch of the "not numeric" and "not initialized" warnings, but that is OK if you know what you are doing.

        CountZero

        A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

        My blog: Imperial Deltronics
        And your whole program can be as short as this:
        use Modern::Perl; my $sum_of_used; @ARGV = <C:/Users/user/Desktop/DOwork/filez/nabillingscript/09_14_2012 +/nas/*>; { no warnings qw/numeric uninitialized/; $sum_of_used += (split /\s+/)[2] while <ARGV>; } say $sum_of_used;
        It uses the magic attached to the ARGV filehandle which automatically iterates over all lines of all filenames in @ARGV.

        CountZero

        A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

        My blog: Imperial Deltronics

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://993745]
Approved by herveus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2014-09-17 08:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (66 votes), past polls