Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^3: why won't the hash go away?

by davidrw (Prior)
on Jan 12, 2008 at 23:49 UTC ( #662125=note: print w/ replies, xml ) Need Help??


in reply to Re^2: why won't the hash go away?
in thread why won't the hash go away?

You could set it up so that each shuffle is done after a fork, and just wait for that one to end/die before going to the next. Something like (untested; off top of head & docs, but i think is the general outline):

my @deck = create_deck(); while(1){ # or whatever; loop over specific deals, or do just N deals +, etc. my $pid = fork; die unless defined $pid; if($pid){ # parent wait; # report on $? if desired next; } do_heavy_lifting(\@deck); exit; # w/an error code based on result if desired } sub do_heavy_lifting { ... my %positions = ....; ... }
So the %positions isn't created/populated (made huge) until in the fork'd process, so when that exits, the memory will get freed to the system, but your program is still going from the parent process.


Comment on Re^3: why won't the hash go away?
Select or Download Code
Managing re-exec (was: Re^4: why won't the hash go away?
by WoodyWeaver (Scribe) on Jan 13, 2008 at 00:29 UTC
    Its a good plan. I tend to prefer a watchdog approach -- a second program that periodically wakes up, scans the process table, sees if enough copies of (whatever) are running, and if not launch enough additional to bring it up to snuff. But in any event, it sounds like you are suggesting re-exec as the best approach.
Re^4: why won't the hash go away?
by kyle (Abbot) on Jan 13, 2008 at 02:35 UTC

    Memory management via fork is a good idea; I've used it in several situations (usually when I'm going to leak memory).

    The problem WoodyWeaver may have is that when the child exits normally, it will still go through the ten hours of data structure destruction, and the parent will wait the whole time. Taking some code from davidrw and some from my own node,

    use Set::Light; my $pid = fork; die "Can't fork: $!" if ! defined $pid; if($pid){ # parent my $start_time = time; wait; printf "Child exited after %d seconds\n", time - $start_time; } else { # child my $start_time = time; my $set = Set::Light->new(); for( my $i = 0; $i < 10_000_000; $i++ ) { $set->insert( $i ); } printf "Child done after %d seconds\n", time - $start_time; exit; } __END__ Child done after 36 seconds Child exited after 163 seconds

    The solution to this is not to let the child exit normally (i.e., kill it). In the above code, if I change "exit" to "kill 9 => $$", the OS does the memory reclamation, and the output looks like this:

    Child done after 32 seconds Child exited after 32 seconds
      I haven't seen the problem you describe, oddly enough. That might be OS / version specific. My routines in the past would exit because of time or state limits exceeded (I'd have them start over after four hours or four billion states examined) or signals (I'd have them print and die if I sent them an INT or HUP). They would exit quickly and normally on the latter (some flavor of linux and ActiveState, 5.8.something.)

      I did take the excellent advice presented here. My code now "uses memory management via fork".

      A run shows the following:
      Mon Jan 14 12:36:28 2008 Launched process to play cards. user 0.031 system 0.031 children 0, 0 Mon Jan 14 16:29:34 2008 Child process finished. user 9774.546 system 556.765 children 0, 0 Mon Jan 14 16:29:35 2008 Launched process to play cards. user 9774.546 system 556.781 children 0, 0
      The "Child process finished" is after the waitpid, so that should mean that my kid is dead. There is a caveat on times() that says "times for children are included only after they terminate" which should have occurred. Why didn't I see any value on the line for 16:29:34 (and similarly for 16:29:35). Is this some sort of race condition? Or is my OS (oh, look at the Vista out this window!) so brain dead that the feature isn't implemented?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://662125]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2014-09-01 19:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (15 votes), past polls