Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Stepping through a hash while modifiying its content

by Gorgarian (Initiate)
on Dec 28, 2012 at 04:48 UTC ( #1010615=perlquestion: print w/replies, xml ) Need Help??
Gorgarian has asked for the wisdom of the Perl Monks concerning the following question:

Hi all in the monastry -- hope you are not all sworn to silence during the festive season.

I have a small challenge, being fairly new to perl. I wish to step through a hash with

while ( ($key, $value) = each %hash ){ }

but then modify the hash by deleting key,value pairs from the hash after some processing if they meet a certain condition, with the while loop in progress.

I have this nagging worry that modifiying the contents of the hash on the fly will stuff up the orderly progression of the while loop and that some records may be missed.

Can anyone with a deeper knowledge of perl advise on this?

A workaround would be to establish a companion array

@array = keys %hash; foreach $key (@array){ # Access the hash, and do the deed including any hash # key, value deletions. }

but this doubles the memory requirements, and my files are huge. I would like to avoid this if it is not necessary.

Looking forward to hearing from you. My first post.

Replies are listed 'Best First'.
Re: Stepping through a hash while modifiying its content
by Anonymous Monk on Dec 28, 2012 at 04:58 UTC
    If you add or delete a hash's elements while iterating over it, entries may be skipped or duplicated--so don't do that

    perlfaq4#What happens if I add or remove keys from a hash while iterating over it?

    (contributed by brian d foy)

    The easy answer is "Don't do that!"

    If you iterate through the hash with each(), you can delete the key most recently returned without worrying about it. If you delete or add other keys, the iterator may skip or double up on them since perl may rearrange the hash table. See the entry for each() in perlfunc.

      OK, thank you for confirming my suspicions, and pointing out the documentation that covers this. I can probably get away with only deleting the most recently returned entry, and not the second entry I reference,though with some considerable increase in runtime.

      Have a good new year.

Re: Stepping through a hash while modifiying its content
by Anonymous Monk on Dec 28, 2012 at 05:02 UTC

    # Access the hash, and do the deed including any hashkey, value deletions

    You should replace that comment with some code :)

    You could delay deletion :)

    { my @goners; while( my( $k, $v ) = each %hash ){ push @goners, $k; } delete @hash{ @goners }; undef @goners; }

      That is a good suggestion too -- I could run the code in blocks, saving up the deletions, and doing a clean-up after each 1M iterations, then restarting the while loop. I will see how much longer it takes to run after restricting myself to deleting only deleting the most recent iteration reference, and if it is a problem, move to your idea as a Plan B.

      Thanks for your help. Learning ... :)

Re: Stepping through a hash while modifiying its content
by karlgoethebier (Prior) on Dec 28, 2012 at 14:43 UTC

    Perhaps you can do it with List::Pairwise instead?

    use List::Pairwise qw(grepp); # grep hash subset my %subset1 = grepp {$a =~ /^ba/} %hash; my %subset2 = grepp {$b < 5} %hash;

    See List::Pairwise

    "...and my files are huge"

    I don't know how your files look. Perhaps you can provide an example...

    I'm just guessing. Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1010615]
Approved by muba
Front-paged by Lotus1
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (5)
As of 2018-01-20 19:20 GMT
Find Nodes?
    Voting Booth?
    How did you see in the new year?

    Results (227 votes). Check out past polls.