Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Memeory usage output KB to MB conversion

by Perlistan (Initiate)
on Jun 05, 2021 at 15:57 UTC ( #11133559=perlquestion: print w/replies, xml ) Need Help??

Perlistan has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks
Looking to format the text output of db2pd command to convert memory values from KB to MB.
input file is as below:
Database Member 0 -- Active -- Up 26 days 13:26:26 -- Date 2020-07-10- +10.11.32.332869 Database Member Memory Controller Statistics Controller Automatic: N Controller License Limit: N Controller Limit Enforced: Y Memory Limit: 26597404 KB Current usage: 25768448 KB HWM usage: 26003008 KB Cached memory: 3352576 KB Individual Memory Consumers: Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== APPL-PB1 196352 601792 38272 DBMS-db2pb1 210752 210752 4736 FMP_RESOURCES 22528 22528 20736 PRIVATE 1377280 1424640 843968 DB-PB1 23961536 24333568 2444864
Desired output
>perl -nl db2mem.pl db2mem.PB1 Memory Limit : 25,974 MB Current usage : 25,164 MB HWM usage : 25,393 MB Name Mem Used (MB) HWM Used (MB) Cached (MB) ======================================================== APPL-PB1 191 587 37 FMP_RESOURCES 22 22 20 PRIVATE 1,345 1,391 824 DB-PB1 23,399 23,763 2,387
Here is my ugly code.
wondering if there is better way for it and having parameter that can be given as MB or GB since our servers have huge memory all in GBs
#use strict; #use warnings; #-- Format memory in MB from db2pd -dbptnmem #-- Run as db2pd -dbptnmem | perl -nl db2mem.pl #-- or perl -nl db2mem.pl db2mem.PB1 sub commify { my $text = reverse $_[0]; $text =~ s/(\d\d\d)(?=\d)(?!\d*\.)/$1,/g; return scalar reverse $text; } print if ($. == 1); if (/Name|===/) { s/KB/MB/g; print; } if (/Memory Limit|Current usage:|HWM usage:| Cached memory:/ && length +($_) > 0 ){ ($name, $value) =split /:/; $value =~ s/KB//g; printf "%-14s: %10s MB\n", $name, commify(int $value/1024); if ($name eq "HWM usage") { printf "\n"; } } if (/APPL-|DB-|FMP|PRI/) { ($name, $used, $hwm, $free) =split /\s+/; printf "%-15s %10s %10s %10s\n", $name, commify(int $used/1024), c +ommify( int $hwm/1024), commify( int $free/1024); }

Replies are listed 'Best First'.
Re: Memeory usage output KB to MB conversion
by Fletch (Chancellor) on Jun 06, 2021 at 01:13 UTC

    You may be interested in Number::Bytes::Human or Number::Format from CPAN. Never used the latter personally but it looks like it has some support for "unformatting" numbers which you could use to parse your input and then reformat things back out with whatever different suffix and/or commaification. I have a shell function using the former that I keep on hand for hyoomaning up arbitrary numbers in the shell.

    format_bytes () { perl -MNumber::Bytes::Human=format_bytes -lE 'for( @ARGV ) { say for +mat_bytes( $_ ); }' "$@" }

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

Re: Memeory usage output KB to MB conversion
by tomasz (Acolyte) on Jun 05, 2021 at 19:53 UTC
    This should help you find a less chaotic direction. The code is not complete and some bits are not along your wish, but the code is plastic.

    You need to improve the value transformation and commation-commification function (rc in the code). 1024 MB is 1073741824 bytes (1024 ** 3), not 1024 * 1000 * 1000.

    As for the comma or commas, there might be such a printf option. I'm not sure, but wouldn't be surprised.

    This function will also integrate nicely with translation to other measure bases (GB or whatever).
    sub rc { my $n = int shift()/1024; $n =~ s/(\d+?)(\d{3})$/$1,$2/; return $n; } s/([^:]+:)\s+(\d+)\s+KB/ sprintf "%-14s%10s MB", $1, rc($2)/e; s/^\=+$/"=" x 68/e; my $IMCfmt = "%-17s%17s%17s%17s"; s/^(Name)\s+(Mem Used)\s+\(KB\)\s+(HWM Used)\s+\(KB\)\s+(Cached)\s+\(K +B\)/ sprintf $IMCfmt, $1, $2." (MB)", $3." (MB)", $4." (MB)"/e; s/^([APPL-|DBMS-|FMP_|PRIVATE|DB-]\S*)\s+(\d+)\s+(\d+)\s+(\d+)/ sprintf $IMCfmt, $1, rc($2), rc($3), rc($4)/e;
    I FORGOT: I used the -p switch on the command line instead of the -n. (That's where all the prints are.)
Re: Memeory usage output KB to MB conversion
by Marshall (Canon) on Jun 06, 2021 at 00:12 UTC
    Your code works, except: (1) You are just moving the decimal so, divide by 1,000 instead of 1024, (2) you should round up, I used the ceil() function for that.

    Here is a "hack it with regex" approach:
    This doesn't exactly replicate your formatting. But to my eye, it looks "close enough".
    I just started adding one regex at a time until I got something "close to desired".
    Mileage varies. I used "just a complicated enough" regex for your example. I didn't consider other examples.
    Again, mileage varies. But maybe you will see something useful in this hack.
    Also, I considered running 5 regexes per line to be of no performance consequence at all.

    ADDED: One complication that I did see was, what if Name like "DBMS-db2pb13345" appears? A more complex regex would be needed to skip that. Also from your problem statement, it sounds like sometimes this might need to go KB->GB instead of KB->MB? I'm not sure that that would mean. "Commifiying" the numbers adds a lot of readability. Your sub for that is fine and right out of Perl Cookbook!

    use strict; use warnings; use POSIX 'ceil'; #-- Format memory in MB from db2pd -dbptnmem #-- Run as db2pd -dbptnmem | perl -nl db2mem.pl #-- or perl -nl db2mem.pl db2mem.PB1 sub commify { my $text = reverse $_[0]; $text =~ s/(\d\d\d)(?=\d)(?!\d*\.)/$1,/g; return scalar reverse $text; } while (<DATA>){ print, next if ($.==1); #don't process first line s/KB/MB/g; # all KB->MB s|(\d{2,})|commify(ceil($1/1000));|ge; # move decimal point s| (\d+,)|$1|g; # allow for one comma in al +ignment s| (Mem Used)|$1|; # adjust columns to left 3 +spaces s|===||; # shorten separator 3 colum +ns print; } =OUTPUT Database Member 0 -- Active -- Up 26 days 13:26:26 -- Date 2020-07-10- +10.11.32.332869 Database Member Memory Controller Statistics Controller Automatic: N Controller License Limit: N Controller Limit Enforced: Y Memory Limit: 26,598 MB Current usage: 25,769 MB HWM usage: 26,004 MB Cached memory: 3,353 MB Individual Memory Consumers: Name Mem Used (MB) HWM Used (MB) Cached (MB) ===================================================== APPL-PB1 197 602 39 DBMS-db2pb1 211 211 5 FMP_RESOURCES 23 23 21 PRIVATE 1,378 1,425 844 DB-PB1 23,962 24,334 2,445 =cut __DATA__ Database Member 0 -- Active -- Up 26 days 13:26:26 -- Date 2020-07-10- +10.11.32.332869 Database Member Memory Controller Statistics Controller Automatic: N Controller License Limit: N Controller Limit Enforced: Y Memory Limit: 26597404 KB Current usage: 25768448 KB HWM usage: 26003008 KB Cached memory: 3352576 KB Individual Memory Consumers: Name Mem Used (KB) HWM Used (KB) Cached (KB) ======================================================== APPL-PB1 196352 601792 38272 DBMS-db2pb1 210752 210752 4736 FMP_RESOURCES 22528 22528 20736 PRIVATE 1377280 1424640 843968 DB-PB1 23961536 24333568 2444864
Re: Memeory usage output KB to MB conversion
by Perlistan (Initiate) on Jun 10, 2021 at 13:33 UTC
    Thank you Monks for helping me widen my perspective on this problem. My code is much better now.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11133559]
Approved by marto
Front-paged by erzuuli
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2021-06-12 15:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)












    Results (53 votes). Check out past polls.

    Notices?