Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

undefined hash elements

by MrTEE (Novice)
on Dec 18, 2013 at 15:40 UTC ( #1067703=perlquestion: print w/replies, xml ) Need Help??

MrTEE has asked for the wisdom of the Perl Monks concerning the following question:

I have a script that parses df into something that perl can use.

1 #!/usr/bin/perl 2 use strict; 3 use warnings; 4 5 my @headers = qw(name size used free capacity mount); 6 my @df = `df -k`; 7 shift @df; # get rid of the header 8 9 my %devices; 10 for my $line (@df) { 11 my %info; 12 @info{@headers} = split /\s+/, $line; # note the hash slice 13 $info{capacity} = _percentage_to_decimal($info{capacity}); 14 $devices{ $info{mount} } = \%info; 15 } 16 17 # Change 12.3% to .123 18 sub _percentage_to_decimal { 19 my $percentage = shift; 20 $percentage =~ s{%}{}; 21 return $percentage / 100; 22 } 23 # Now the information for each device is in a hash of hashes. 24 25 # Show how much space is free in device /dev/ad4s1e 26 print $devices{"/production/log"}{free} ; 27 print "\n"; 28 for my $info (values %devices) { 29 # Skip to the next device if its capacity is not over 60%. 30 next unless $info->{capacity} > .10; 31 32 # Print some info about each device 33 printf "%s is at %d%% with %dK remaining.\n", 34 $info->{mount}, $info->{capacity}*100, $info->{free}; 35 }

However I keep getting these warnings.

Use of uninitialized value in substitution (s///) at ./get_df line 21. Use of uninitialized value in division (/) at ./get_df line 22. Use of uninitialized value in hash element at ./get_df line 15. Use of uninitialized value in substitution (s///) at ./get_df line 21. Use of uninitialized value in division (/) at ./get_df line 22. Use of uninitialized value in hash element at ./get_df line 15. Use of uninitialized value in substitution (s///) at ./get_df line 21. Use of uninitialized value in division (/) at ./get_df line 22. Use of uninitialized value in hash element at ./get_df line 15. Use of uninitialized value in substitution (s///) at ./get_df line 21. Use of uninitialized value in division (/) at ./get_df line 22. Use of uninitialized value in hash element at ./get_df line 15. Use of uninitialized value in substitution (s///) at ./get_df line 21. Use of uninitialized value in division (/) at ./get_df line 22. Use of uninitialized value in hash element at ./get_df line 15. 9006792 /production/log is at 70% with 9006792K remaining. / is at 37% with 17037532K remaining. /production is at 11% with 13171728K remaining. /export/home is at 24% with 11199904K remaining. /production/archive is at 18% with 8095796K remaining. /boot is at 28% with 68351K remaining.

I looked at the DF module on CPAN last night at home, but I would have to get sysadmin approval to get it installed. On the df the Filesystem is too long, so it gets printed to another line. I can df -k | grep -v out the long filenames. The long file names on the extra line messed up the data dumper print out - some of the hash values get labeled undef.

casper@casperbox]:~/.wjohnson> df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 28313732 9816924 17035356 37% / /dev/sda1 101086 27516 68351 29% /boot tmpfs 2987896 0 2987896 0% /dev/shm /dev/mapper/VolGroupPROD-ExportHome 15481840 3495504 11199904 24% /export/home /dev/mapper/VolGroupPROD-Production 15481840 1523692 13171716 11% /production /dev/mapper/VolGroupPROD-ProdLog 30963708 20410952 8979892 70% /production/lo +g /dev/mapper/VolGroupPROD-ProdArchive 10313016 1693640 8095500 18% /production/ar +chive [casper@casperbox]:~/.wjohnson> [casper@casperbox]:~/.wjohnson> [casper@casperbox]:~/.wjohnson> [casper@casperbox]:~/.wjohnson> [casper@casperbox]:~/.wjohnson> df -k | grep -v dev Filesystem 1K-blocks Used Available Use% Mounted on 28313732 9816924 17035356 37% / 15481840 3495504 11199904 24% /export/home 15481840 1523692 13171716 11% /production 30963708 20410952 8979892 70% /production/lo +g 10313016 1693640 8095500 18% /production/ar +chive [casper@casperbox]:~/.wjohnson>

From Data::Dumper - many of the hash values are coming up as undefined. is there a way that I could predefine the values of the hash. I want to learn to get rid of them. grepping out the long filenames works but it seems like a hack around a problem that i have very often in Perl - the undefined values.

$VAR1 = {}; Use of uninitialized value in substitution (s///) at ./get_df.just_cap +acity line 24. Use of uninitialized value in division (/) at ./get_df.just_capacity l +ine 25. Use of uninitialized value in hash element at ./get_df.just_capacity l +ine 17. $VAR1 = { '' => { 'free' => undef, 'mount' => undef, 'used' => undef, 'name' => '/dev/mapper/VolGroup00-LogVol00', 'capacity' => '0', 'size' => undef } }; $VAR1 = {}; $VAR1 = { '' => { 'free' => undef, 'mount' => undef, 'used' => undef, 'name' => '/dev/mapper/VolGroup00-LogVol00', 'capacity' => '0', 'size' => undef },

this is resolved by using df -k | grep -v var - but there has to be a better way. i tried an if statment checking that if a hash element was not defined to assign it a 1, but it kept on throwing out more errors.

Replies are listed 'Best First'.
Re: undefined hash elements
by marinersk (Priest) on Dec 18, 2013 at 19:03 UTC

    Summary of Problems: Blindly passing bad data around.
     

    • You are passing bad data to _percentage_to_decimal(), which is not doing any defensive data checking, and;
    • You are unconditionally using a hash key without checking its existence first.

    I am able to reproduce your problem using the input data you specified (thank you for providing that). My line numbers are off by one from yours but otherwise it looks live I've matched your error:

    I apologize for using my homespun debug module instead of Data::Dumper. Feel free to convert the debugging lines at your convenience, but you probably won't need them going forward.

    In this debug version of the code, look in the results for where $percentage = ''. You pass bad data in to the subroutine, and it blindly starts trying to compute against it.

    The results follow -- WARNING: This is LONG.

    I then modified _percentage_to_decimal to include defensive coding:

    # Change 12.3% to .123 sub _percentage_to_decimal { my $percentage = shift; if (!defined $percentage) { $percentage = 0; } $percentage =~ s{%}{}; return $percentage / 100; }

    That reduced the errors a bit:

    C:\Steve\Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef>testdf4.pl Use of uninitialized value $info{"mount"} in hash element at C:\Steve\ +Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef\testdf4.pl line 14. Use of uninitialized value $info{"mount"} in hash element at C:\Steve\ +Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef\testdf4.pl line 14. Use of uninitialized value $info{"mount"} in hash element at C:\Steve\ +Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef\testdf4.pl line 14. Use of uninitialized value $info{"mount"} in hash element at C:\Steve\ +Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef\testdf4.pl line 14. Use of uninitialized value $info{"mount"} in hash element at C:\Steve\ +Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef\testdf4.pl line 14. 8979892 /production/log is at 70% with 8979892K remaining. / is at 37% with 17035356K remaining. /boot is at 28% with 68351K remaining. /export/home is at 24% with 11199904K remaining. /production is at 11% with 13171716K remaining. /production/archive is at 18% with 8095500K remaining.

    So I added a trap in the main loop:

    if (!defined $info{mount}) { my $displayLine = $line; chomp $displayLine; print "\$info{mount} undefined when \$line = [$displayLine]\ +n"; } else { $devices{ $info{mount} } = \%info; }

    Now, the only "errors" displayed are the ones from the trap:

    C:\Steve\Dev\PerlMonks\P-2013-12-18@1604-Hash-Undef>testdf5.pl $info{mount} undefined when $line = [/dev/mapper/VolGroup00-LogVol00] $info{mount} undefined when $line = [/dev/mapper/VolGroupPROD-ExportHo +me] $info{mount} undefined when $line = [/dev/mapper/VolGroupPROD-Producti +on] $info{mount} undefined when $line = [/dev/mapper/VolGroupPROD-ProdLog] $info{mount} undefined when $line = [/dev/mapper/VolGroupPROD-ProdArch +ive] 8979892 /production/log is at 70% with 8979892K remaining. / is at 37% with 17035356K remaining. /production is at 11% with 13171716K remaining. /export/home is at 24% with 11199904K remaining. /production/archive is at 18% with 8095500K remaining. /boot is at 28% with 68351K remaining.

    I should think at this point the resulting code would help you figure out how to get rid of your logic errors:

    #!/usr/bin/perl use strict; use warnings; my @headers = qw(name size used free capacity mount); my @df = &readDFfile('test.dat'); shift @df; # get rid of the header my %devices; for my $line (@df) { my %info; @info{@headers} = split /\s+/, $line; # note the hash slice $info{capacity} = _percentage_to_decimal($info{capacity}); if (!defined $info{mount}) { my $displayLine = $line; chomp $displayLine; print "\$info{mount} undefined when \$line = [$displayLine]\ +n"; } else { $devices{ $info{mount} } = \%info; } } # Change 12.3% to .123 sub _percentage_to_decimal { my $percentage = shift; if (!defined $percentage) { $percentage = 0; } $percentage =~ s{%}{}; return $percentage / 100; } # Now the information for each device is in a hash of hashes. # Show how much space is free in device /dev/ad4s1e print $devices{"/production/log"}{free} ; print "\n"; for my $info (values %devices) { # Skip to the next device if its capacity is not over 60%. next unless $info->{capacity} > .10; # Print some info about each device printf "%s is at %d%% with %dK remaining.\n", $info->{mount}, $info->{capacity}*100, $info->{free}; } exit; sub readDFfile { my ($rdofnm, @arglst) = @_; if (!defined $rdofnm) { $rdofnm = ''; } # -----[ Was command, now reading from file for testing purposes +]-------------- # my @df = `df -k`; # --------------------------------------------------------------- +--------------- my @df = (); if (open RDOFIL, '<', $rdofnm) { @df = <RDOFIL>; close RDOFIL; } # --------------------------------------------------------------- +--------------- return @df; } __END__

    Holler if you need any additional assistance.

      thank you very much - I really like the term you use "defensive coding". that sums up just what I was trying to achive here. In my perl coding i want to develop more defensive coding parctices. This is alot of beef to chew on - it should keep me feed for days.

        This particular form of defensive programming is sometimes called "fortressing", meaning it tries to keep bad data from getting in. The reciprocal function is called, if memory serves, "dungeoning", which is trapping bad data where it can do no harm. I don't know if I can think of any examples of where that approach serves much use -- which probably means I don't know as much about it as I thought I did. :-)
Re: undefined hash elements
by Random_Walk (Prior) on Dec 18, 2013 at 15:58 UTC

    On my current OS, Linux Mint, the following code works. Note I take the headers from the first line of df and use them. I then have a small abstraction of a name hash to map mount and percent to appropriate fields. I think your list of headers came from another OS than the one you are running on, and that is leading down the wrong path.

    #!/usr/bin/perl use strict; use warnings; use Data::Dumper; # a couple of headers that vary from df to df my %name = ( percent => 'Use%', mount => 'Mounted', ); # my @headers = qw(name size used free capacity mount); my @df = `df -k`; my @headers = split /\s+/, shift @df; my %devices; for my $line (@df) { print $line; my %info; @info{@headers} = split /\s+/, $line; # note the hash slice $info{capacity} = _percentage_to_decimal($info{$name{percent}}); $devices{ $info{$name{mount}} } = \%info; } # Change 12.3% to .123 sub _percentage_to_decimal { my $percentage = shift; $percentage =~ s{%}{}; return $percentage / 100; } print Dumper \%devices;

    Cheers,
    R.

    Pereant, qui ante nos nostra dixerunt!

      Yeah It works for me at home as well. the DF has long values at work, they get kicked to another line. That is just what DF does. . The df -k | grep -v var cleans up - but i have gotten these undefined errors before. it is a linux box.

      [capsper@casperbox .wjohnson]$ uname -a Linux casper25s 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 + x86_64 x86_64 x86_64 GNU/Linux

      This is the data dumper - is there a way to get rid of the undef's ?

      [icapsper@casperbox .wjohnson]$ ./get_df.just_capacity $VAR1 = {}; Use of uninitialized value in substitution (s///) at ./get_df.just_cap +acity line 25. Use of uninitialized value in division (/) at ./get_df.just_capacity l +ine 26. Use of uninitialized value in hash element at ./get_df.just_capacity l +ine 18. $VAR1 = { '' => { 'free' => undef, 'mount' => undef, 'used' => undef, 'name' => '/dev/mapper/VolGroup00-LogVol00', 'capacity' => '0', 'size' => undef } };

        I dump out the %info and %devices hashes

        use Data::Dumper ; my %devices; for my $line (@df) { use Data::Dumper ; my %info; print Dumper (\%info); @info{@headers} = split /\s+/, $line; # note the hash slice $info{capacity} = _percentage_to_decimal($info{capacity}); $devices{ $info{mount} } = \%info; print Dumper(\%devices); }
Re: undefined hash elements
by roboticus (Chancellor) on Dec 18, 2013 at 15:52 UTC

    MrTEE:

    I don't know how df decides to break the lines up, but if it uses the terminal size, then you might be able to change the terminal width environment variable (COLUMNS, I'd guess) before running it.

    $ENV{COLUMNS}=4096; my @array = `df -k`;

    If that doesn't work, you could check for the problem and join with the next line if required. For example, you might do something like:

    # Fix array: If line is too short, append to previous line { my @tmp; while (@array) { my $line = shift @array; $line .= shift @array if length($line)<50; push @tmp, $line; } @array = @tmp; }

    Note: Untested, as I don't have perl on this computer.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Re: undefined hash elements
by rmcgowan (Sexton) on Dec 18, 2013 at 16:25 UTC

    What operating system are you using? I have access to Linux and Solaris systems, but cannot duplicate your problem. I get multiple lines on tty output (to the terminal) but when I run the 'df' in backquotes, all the data is on one line, for each device. This is also true if I pipe 'df' output through 'grep', just as you found.

    What I don't understand is why the same doesn't happen for you with the backquotes.

    As an untested suggestion, you could do your 'split' into an explicit variable declared outside the loop, and check it's size:

    my ($tmp, $prev); # ...loop here if (@tmp == 1) { @prev = @tmp; next; }

    You could then merge the two arrays to process with the slice. Perhaps not the most elegant solution, but it has the advantage of not failing if the df output ever changes to be single lines in all cases. Since this is done before creating the hash, it should fix at least that part of the problem.

      I am using linux. maybe I am too hung up on this undefined hash values thing. maybe it is better to masage the data going in. i have seen this undef error so many times when using hashes that to rememdy i just take off 'strict' - which ultimately causes even more headaches. hey check this out

      df -Ph | perl -ne 'chomp; printf "\n%-40s %8s %8s %8s %8s %-20s", spli +t / +/, $_, 6 ; '
        mayb ei am using the wrong df :

        check this out. this is df -k

        [icapsper@casperbox .wjohnson]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 28G 9.4G 17G 37% / /dev/sda1 99M 27M 67M 29% /boot tmpfs 2.9G 0 2.9G 0% /dev/shm /dev/mapper/VolGroupPROD-ExportHome 15G 3.4G 11G 24% /export/home /dev/mapper/VolGroupPROD-Production 15G 1.5G 13G 11% /production /dev/mapper/VolGroupPROD-ProdLog 30G 20G 8.4G 71% /production/log /dev/mapper/VolGroupPROD-ProdArchive 9.9G 1.7G 7.8G 18% /production/archive

        and this is df -P (which is better ) the df -P does not put the mounted files on a separate line

        [casper@casperbox .wjohnson]$ df -P Filesystem 1024-blocks Used Available Capacity Mounted on /dev/mapper/VolGroup00-LogVol00 28313732 9812576 17039704 37% + / /dev/sda1 101086 27516 68351 29% /boot tmpfs 2987896 0 2987896 0% /dev/shm /dev/mapper/VolGroupPROD-ExportHome 15481840 3495504 11199904 + 24% /export/home /dev/mapper/VolGroupPROD-Production 15481840 1523736 13171672 + 11% /production /dev/mapper/VolGroupPROD-ProdLog 30963708 20621072 8769772 71 +% /production/log /dev/mapper/VolGroupPROD-ProdArchive 10313016 1693936 8095204 + 18% /production/archive [icapfo@usprvfotr25s .wjohnson]$

        maybe I am using wrong df

Re: undefined hash elements
by MrTEE (Novice) on Dec 18, 2013 at 15:57 UTC
    Is there a way to assing a value to the hash elements before they get assigned an elements so they are not 'undef'. I tried an if (!defined) thing but it was thowing errors.

      defined() works. You might like to show us the code and error and we can fix that.

      If you want to check if a hash key exists at all, the function exists() is the right one to use

      Don't do that. Almost always undef warnings point to a logic error. If you mask the warning you are ignoring the logic error. Better to figure out how you get into the state where the warnings are generated then fix the root cause than to paper over the cracks until the whole thing falls down around your ears.

      True laziness is hard work

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1067703]
Approved by taint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (2)
As of 2022-06-27 07:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (87 votes). Check out past polls.

    Notices?