Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Multialarm loop

by Excalibor (Pilgrim)
on Oct 10, 2005 at 14:38 UTC ( [id://498816]=perlquestion: print w/replies, xml ) Need Help??

Excalibor has asked for the wisdom of the Perl Monks concerning the following question:

Greetings, dear brothers in the Monastery

To-day I seek you to illuminate on a complex matter I cannot find a satisfactory way to solve...

I have a process creating objects on a database that must be garbage collected by another process. Those object live in process 1 core memory as avatars of the DB objects. but their demise must not mean the deletion from the DB (thus I cannot user DESTROY() nor END()). I have a second process that will do garbage collection of the DB objects. This GC is based on object status (a field in the DB object) and on time (since creation time, last access by other processeses, etc).

I have three different collecting priorities, and thus I thought a kind of generational GC could be "nice": phase_1 objects have been marked by a process to be collected ASAP (really ASAP. they are occupying resources that must be liberated to the program resources pool manager so other processes can use them (IP addresses, and the like)); phase_2 object are marked for recollection "when the system can, no rush"; phase_3 objects are all that surpass a given timeout, independently of their status.

Now, phase_1 time is circa 10 smaller than phase_2, which is about 100 times smaller than phase_3 times... phase 1 can be around 30 seconds, more or less...

So, I loop over all objects, locate those belonging to each "generation" and proceed to recollection, then sleep... What I'd actually like is to have three timers, that lauch the appropriate kind of recolection, but SIGALRM won't do...

Suggestions, please?

Note: I tried to read through EventServer, but I don't understand it... I basically need 3 "concurrent" loops, that's "all"...

Thanks, today's a hard day, I'm obscure...

--
our $Perl6 is Fantastic;

Replies are listed 'Best First'.
Re: Multialarm loop
by Roy Johnson (Monsignor) on Oct 10, 2005 at 14:54 UTC
    You could spawn a job for each phase. The job for phase 1 would sleep 30 seconds and exit; the phase 2 and phase 3 for 300 and 30000 (?!) seconds, respectively. The master program would wait for some process to finish. As each one does, the master program re-spawns the job and handles the collection.

    Or you could just keep an array of phase-time pairs and set an alarm for the next-expiring one:

    my @increments = (undef, 30, 300, 30000); my @times = map {($_, time + $increments[$_])} 1..3; alarm($times[0][1]); #... # then in sigalarm, handle and reschedule the next phase my $this_phase = (shift(@times))->[0]; @times = sort { $a->[1] <=> $b[1] } (@times, [$this_phase, $increments +[$this_phase]); alarm($times[0][1]); my_garbage_collect($this_phase);

    Caution: Contents may have been coded under pressure.
Re: Multialarm loop
by Zaxo (Archbishop) on Oct 10, 2005 at 14:52 UTC

    You only get one alarm timer, so you're going to have to set up a scheduler of some kind. Decide which reaper is to be called next and when, then set alarm to that interval. A global variable holding a code reference to the next reaper sub would be handy.

    A priority queue, aka heap, is a handy data structure for that kind of thing.

    After Compline,
    Zaxo

Re: Multialarm loop
by samizdat (Vicar) on Oct 10, 2005 at 15:07 UTC
    What kind of system are you on, Excalibor? If it's a *NIX box, using a daemon that runs your checking loops is the way to go. Since each is a multiple of its smaller brethren, you have the daemon wake up once every 25 seconds (~) to perform phase_1 cleanup. After doing that, it checks a pair of countdown counters to see if 10 cycles of phase_1 have passed and it's time for a phase_2 sweep, and if 100 phase_1 cycles (or ten sets of phase_2) have passed and it's time for a phase_3 sweep. The daemon will remain resident, so you don't need persistent storage for the counter variables; just check them when you wake up.

    It's a mistake to try to use many timers in a situation like this. Far better to use one timer and a couple of counters to accomplish the goal.

    The one caution I'd give is that the time required to actually do the GC is a variable amount, depending on how many things need to be GCed. It might be appropriate to have the phase_2 and phase_3 cleanups launched as separate processes from the daemon, and use renice to adjust their system resource priority. In any event, you want to have the daemon waking up more often than you need it to to make sure that you do not overrun your interval.
Re: Multialarm loop
by Perl Mouse (Chaplain) on Oct 10, 2005 at 15:46 UTC
    Your description is conflicting. You say that phase_1 objects need to be collected as soon as possible, yet phase_1 can be around 30 seconds.

    Waiting 30 seconds before garbage collecting something is nowadays a really long time.

    Timers are a way to go, either using SIGALRM or with an event loop (select, and I think POE can do the hard stuff for you) if you're willing to wait at least 30 seconds.

    But if objects need to be destroyed either as soon as possible, or "when the system can", you need more than timers. You also need notification an object is available for garbage collection (and hence, likely coorperation from the other program), and for your phase_2 objects, some system monitoring - to be able to spot whatever "when the system can" means.

    For phase_3 objects, it seems that garbage collecting them once every 8 hours is enough (30 * 10 * 100 seconds is 8h20m). I'd use cron for that.

    Perl --((8:>*
Re: Multialarm loop
by BrowserUk (Patriarch) on Oct 10, 2005 at 16:15 UTC

    One process, with 3 threads each with a loop and it's own sleep.

    Adjust the sleeps (possibly dynamically) to meet your urgancy requirement for each loop.

    Proiritize the process accordingly.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://498816]
Approved by castaway
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (3)
As of 2024-03-19 03:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found