Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Modperl2 + mpm_event + seemingly forgotten global vars

by spellila (Initiate)
on Mar 28, 2017 at 19:07 UTC ( #1186280=perlquestion: print w/replies, xml ) Need Help??

spellila has asked for the wisdom of the Perl Monks concerning the following question:

Greeting everyone, i get stucked using modperl2 on apache 2.4 (mpm_event) and ubuntu (upgraded on daily base form ubuntu-repo). I try to keep some filehandles and datastructures simply on a "our" variable. This works fine until i generate some network load for this server (1000 persistent connections each wants to get the same stuff). Than it looks like the "our" is forgeting its value. This is happening much faster if you are using more threads per child. Here a simple example:

package example::PerlResponseHandler; use strict; use warnings; use APR::OS (); use Apache2::Const -compile => qw(OK); our $obj; sub handler { my $r = shift; my $tid = APR::OS::current_thread_id(); if(!$obj) { $obj = {one => 1}; warn "[pid $$ tid $tid] New object generated"; } $r->content_type('text/html'); $r->print('mod_perl rules!' . $obj->{one}); return Apache2::Const::OK; } 1;

Using this package as responsehandler in your apache2.conf and the `ab`-tool targeting this handler, you will see at your error.log that the our var is getting setted. After some time you will see that this variable is reseted on and on.

Using 5000 MaxRequestWorkers splitted by threads 250 on each child i could see that only 18 threads was able to hold this variable without any interaction after it was initialized (over days!).

Even if i used only 3 threads per child and 1500 possible servers, the error was reproduced after the 264 thread was spawned. 263 threads was keeping their states over days.

The server have enough ressources over and no limits being reached.

I want to achive that all my threads holding the states of their our vars.

After some sleepless days i decided to ask the perlmonk community. Hopefully somebody can help further.

Replies are listed 'Best First'.
Re: Modperl2 + mpm_event + seemingly forgotten global vars
by shmem (Chancellor) on Mar 28, 2017 at 20:15 UTC

    Hmm...

    if(!$obj) { $obj = {one => 1}; ... $r->print('mod_perl rules!' . $obj->{1});

    You probably meant

    $r->print('mod_perl rules!' . $obj->{one});

    or vice versa.

    Is that hashref $obj being used elsewhere in your code?

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
      Yeah, thanks. Was my fault, i corrected it.
Re: Modperl2 + mpm_event + seemingly forgotten global vars
by Marshall (Canon) on Mar 28, 2017 at 23:15 UTC
    You say "I try to keep some filehandles and datastructures simply on a "our" variable."

    When using threads that modify shared data which is "global the the entire process", there needs to be some kind of locking mechanism. A thread has to acquire an "exclusive write lock" versus its other "thread peers" in order to update that commonly shared information. I don't see any locking code that would adjudicate write/read conflicts amongst its peer threads. It is possible that one thread might read a variable during the writing of that variable by another thread.

    I don't know much about the server software that you are using, but I'm sure that some other Monks do. However, unlocked shared variable sounds like a likely suspect to me.

      I agree, but only under following condition:

      - the threads are sharing their vars

      - the threads are changing their vars

      I don't see any condition matching at this case. The global var is shared only to the scope it was created. This means to thread it self. Thats ok for my case. On this way i can initate handles and keep them per thread. Each thread having its own handles should be no problem, do it?

      So every thread will create its own "our" variable. And after the creation it should stay there. This works fine for low numbers of connections. But under pressure it gets reinitilized on and on.

Re: Modperl2 + mpm_event + seemingly forgotten global vars
by Krambambuli (Curate) on Mar 29, 2017 at 17:29 UTC
    If what you get here in the monastery doesn't help, try visiting and reposting to the modperl mailing list.

    Even if it is a bit hard to find, the modperl list is active and can be very helpful.
Re: Modperl2 + mpm_event + seemingly forgotten global vars
by Anonymous Monk on Mar 29, 2017 at 11:32 UTC
    How many filehandles is it? There are limits on how many filehandles you can have open on some OS not sure about ubuntu

      In my case there are exactly zero filehandles. But every handle will face this same problem. At the end them are only references. And if a thread can't remember the value of this variable, it will reinitilize it.

      There as many handles as threads. There should be no problem. I'm also able to open as many (and more) handles in a single script, without modperl. Here is my `ulimit -a`:

      core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 64000 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1048576 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited

      I would also await an error message for something like that. But there is nothing.

      This global vars should loose its value only under the condition the thread ends. A java developer pointed me to the garbagge colector, but at the moment my knowhow at this area is not at that level i would want it. (Never was thinking about garbagge collection while using perl)

      Let's assume them are ending, why is there a little rest of threads allowed to keep living and holding their vars? (I mean it is a computer. I was thinking everything in it has to be deterministic, so someone has to delete this vars)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1186280]
Approved by 1nickt
Front-paged by 1nickt
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (5)
As of 2023-02-03 19:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer not to run the latest version of Perl because:







    Results (29 votes). Check out past polls.

    Notices?