Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Configuration in threaded app

by menth0l (Monk)
on Aug 23, 2011 at 12:48 UTC ( #921884=perlquestion: print w/replies, xml ) Need Help??
menth0l has asked for the wisdom of the Perl Monks concerning the following question:

I hava a multithreaded application with large, nested xml configuration file. I look for 'clean' way of reconfiguring whole app so that all worker threads would use 'fresh' config simultaneously.

I see several ways:
1) create one shared singleton config class through which i would access my values, i.e.:
use MyConfig; # this instance is common for all threads MyConfig::instance()->getVal(...);
2) use non-shared singleton object per each thread and add custom signal handler that would reload configuration data. When main thread would found that config was chaged it would send this signal to all workers. But! for N workers this process would execute N times just to fetch the same content (then parse and validate it)...

Is there any other way? And which is the best?

Replies are listed 'Best First'.
Re: Configuration in threaded app
by Corion (Pope) on Aug 23, 2011 at 12:57 UTC

    My advice is to not mix signals and threads, anywhere.

    Personally, I would structure the worker threads to fetch messages from a central job queue, configuring them at startup. When reconfiguration becomes necessary, I would purge the current job queue (and maybe put the still pending jobs in the fresh job queue) and tell all current worker threads to quit. Then I'd fire up new worker threads with a fresh configuration and a fresh job queue.

      What abount approach #1?

      BTW: What's wrong with threads & signals?

        Why would you want to keep on accessing a shared structure? This will essentially create a central locked data structure that will make all threads block when one thread reads from the configuration.

        Signal handling vastly varies between the thread implementations, from catastrophic failures to behaving like processes. I would avoid signals instead of finding out what behaviour is prevalent with your Perl and your OS and just implement in-band messages through Thread::Queue. There rarely is a reason to tell a thread "Stop whatever you're doing" where you can't just kill the thread and configure a fresh thread.

Re: Configuration in threaded app
by BrowserUk (Pope) on Aug 23, 2011 at 13:32 UTC

    I'd use a global shared hash rather than a "singleton object" which is just a fancy name for a global anyway, and gets you into a painful world of shared methods.

    Simple and effective, I'd do it like this:

    #! perl -slw use strict; use threads; use threads::shared; use Data::Dump qw[ pp ]; use Time::HiRes qw[ time sleep ]; my $sem :shared; sub tprint { lock $sem; print @_; } my %config :shared; sub loadConfig{ open my $fh, '<', 'junk.config' or die $!; lock %config; $config{ $_->[0] } = $_->[1] while @{ $_ = [ split ':', <$fh> ] }; tprint pp \%config; tprint "Config refreshed at ", scalar time; }; $SIG{'INT'} = \&loadConfig; sub worker { my $tid = threads->tid; my $telltale = $config{ telltale }; while( sleep 0.01 ) { if( $config{ telltale } ne $telltale ) { tprint "[$tid] saw change at ", scalar time(); $telltale = $config{ telltale }; } } } loadConfig(); my @workers = map threads->create( \&worker ), 1 .. 8; while( threads->list > 1 ) { $_->join for threads->list( threads::joinable ); sleep 0.1; } __END__ C:\test> { # tied threads::shared::tie bill => "leprechaun\n", fred => 123 , telltale => 1 , } Config refreshed at 1314106074.357 { # tied threads::shared::tie bill => "leprechaun\n", fred => 123 , telltale => 2 , } [3] saw change at 1314106084.6591 [4] saw change at 1314106084.66004 [5] saw change at 1314106084.66113 [6] saw change at 1314106084.66309 [1] saw change at 1314106084.66507 [7] saw change at 1314106084.66507 [2] saw change at 1314106084.66612 [8] saw change at 1314106084.66705 Config refreshed at 1314106084.66944 Terminating on signal SIGBREAK(21)

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Configuration in threaded app
by zentara (Archbishop) on Aug 23, 2011 at 13:21 UTC
    Couldn't you just use shared variables? Put the XML is one shared scalar string, and have another shared variable which is constantly tested in the threads, which would tell it to reload the XML from the shared scalar or hash?

    A signal might work to trigger the reload, but as Corion pointed out, signals and threads are quite tricky to get to work right, see Threads, bash, and networking for example; whearas constantly checking a shared variable is quite reliable.

    Another option is to use the fact that you can share filehandles with threads thru the fileno. See FileHandles and threads. You may be able to place your XML in the main thread's defined filehandle, and have the threads re-read it on some command... either thru shared var or a signal of some sort.

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
Re: Configuration in threaded app
by sundialsvc4 (Abbot) on Aug 23, 2011 at 14:41 UTC

    Another simple way to do it is to maintain a global variable which is incremented each time the configuration changes.   Each time the threads wake up, they notice if the value of that variable has changed since the last time they looked at it.   If so, they replace their configuration data before handling the next request.

    Use a “write once, read many” semaphore arrangement so that the thread which wants to replace the configuration data can’t do so if anyone is reading it at the time ... but you don’t care how many threads might be reading it at the time.

    By the way, you can use the same locking semantics to maintain just one configuration-data structure, at the slight expense of lock-management overhead.   Threads use a semaphore to maintain read-stability of the data structure while they are referring to it, but without blocking any other readers.   They release the lock before going to sleep.   Replacing the configuration data simply consists of preparing the new data, obtaining a write-lock on the semaphore, and replacing the old with the new before releasing the write-lock.

    Of the two approaches, I prefer the latter one because it maintains only one copy of the data structure.   The incremental overhead of the locking semantics should be negligible.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://921884]
Approved by Corion
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2018-04-24 09:50 GMT
Find Nodes?
    Voting Booth?