Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Win32, fork and XS globals

by syphilis (Canon)
on Oct 02, 2011 at 04:49 UTC ( #929097=perlquestion: print w/ replies, xml ) Need Help??
syphilis has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

The demo:
use warnings; #use Inline C => Config => # BUILD_NOISY => 1; use Inline C => <<'EOC'; static int x = 5; void set_x(int z) { x = z; } void get_x(char * id) { printf("%s %d\n", id, x); } EOC get_x('1st call:'); if($pid = fork()) { waitpid($pid,0); } else { set_x(10); get_x('2nd call:'); exit(0); } get_x('3rd call:');
On linux that script fulfils my expectations and outputs:
1st call: 5 2nd call: 10 3rd call: 5
But on Win32, I get:
1st call: 5 2nd call: 10 3rd call: 10
The basic requirement is that the script forks; the child then alters the value of an XS global and exits; that global then still retains its original value.

What options are there that will bring Windows into line with Linux ?
Is there anything that can be done within the C code ?
Perhaps something regarding the way (when/how) the C component is loaded ?

It seems that system() produces a genuine fork, and (as desired) any setting of the global inside a system call is lost when the system call exits.
This is not so straightforward to demo with the Inline::C script, but in the real world scenario I'm dealing with an XS module - and using system instead of fork is one alternative.

But I'm keen to hear what, if any, other options exist - especially any that leave the fork() in place.

Interestingly enough, there's no such problem with *perl* globals. The following script works fine on both Win32 and linux:
use warnings; $g = 5; print "1st call: $g\n"; if($pid = fork()) { waitpid($pid,0); } else { $g = 10; print "2nd call: $g\n"; exit(0); } print "3rd call: $g\n";
It yields (as expected and desired):
1st call: 5 2nd call: 10 3rd call: 5
Cheers,
Rob

Comment on Win32, fork and XS globals
Select or Download Code
Re: Win32, fork and XS globals (cygwin)
by tye (Cardinal) on Oct 02, 2011 at 05:00 UTC
    What options are there that will bring Windows into line with Linux ?

    Use cygwin Perl.

    Win32 Perl's fork creates a thread and then makes copies of the Perl interpreter. Copying the Perl interpreter does not make copies of non-Perl global data such as C globals or things loaded from *.dll files.

    If you want to make a copy, then you need to use something tied to the Perl interpreter instance. You could use a Perl package variable (global), for instance.

    - tye        

Re: Win32, fork and XS globals
by BrowserUk (Pope) on Oct 02, 2011 at 05:15 UTC

    But I'm keen to hear what, if any, other options exist - especially any that leave the fork() in place.

    Interestingly enough, there's no such problem with *perl* globals

    As you know, on Windows, forking is emulated using a thread. Perl globals are duplicated in the pseudo-fork because Perl knows about them. C globals aren't because Perl doesn't know about them.

    The solution may be to declare a CLONE() special subroutine in your XS module (or the PM that fronts it) as described in Perlmod.

    The basic mechanism is that whenever a thread is spawned (including a pseudo-fork), the CLONE() sub in your namespace will be called and you get the opportunity to make thread specific copies of your internal state. I can't give further details as I've never done it, but there should be some samples around (maybe in the DBD::* namespace?).


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      That perlmod link is not terribly useful. For instructions on creating per-interpreter variables, see "Safely Storing Static Data in XS" in perlxs. It even includes an example.

      The mechanism allows each interpreter in a process to have their own copies of variables. This is useful not only when threads are involved, but in all instances when a process has multiple Perl interpreters (e.g. mod_perl?).

        It even includes an example.

        Personally, I found that example almost completely useless. An example of "Safely Storing Static Data" that doesn't appear to declare any static data is pretty worthless.

        And having half a dozen macros all named SOM_BG_MNGLSS_ABBREV only differentiated by a single even more meaningless, wimpy prefix character a or d or p or maybe an almost invisible _ prefix or suffix, is stupid, bordering on the criminal.

        Which is why I didn't mention it.

        Of course, if you'd offered this information as a reply to the OP rather throwing it at me, you wouldn't have got this response.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        Be buggered ... this actually seems to work:
        use warnings; use Inline C => Config => NAME => 'expt', BOOT => " { MY_CXT_INIT; MY_CXT.count = 5; } ", BUILD_NOISY => 1, ; use Inline C => <<'EOC'; #define MY_CXT_KEY "expt::_guts" XS_VERSION typedef struct { int count; } my_cxt_t; START_MY_CXT void set_x(int z) { dMY_CXT; MY_CXT.count = z; } void get_x(char * id) { dMY_CXT; printf("%s %d\n", id, MY_CXT.count); } void CLONE(SV * x, ...) { MY_CXT_CLONE; } EOC get_x('1st call:'); if($pid = fork()) { waitpid($pid,0); } else { set_x(10); get_x('2nd call:'); exit(0); } get_x('3rd call:');
        It outputs:
        1st call: 5 2nd call: 10 3rd call: 5
        Must investigate further.

        Cheers,
        Rob
      As you know, on Windows, forking is emulated using a thread

      I had read that, though I'm not really "up to speed" when it comes to "forking" or "threads".
      Having spent my spare (sparse) moments over the last week or so trying to learn a little about this, I now find myself thinking "Wouldn't it make more sense to have emulated forking by using system() ?"

      I'm thinking that way because it seems to me that system('perl -e "..."') comes closer to the *nix fork() than anything else.

      Needless to say ... I stand to be corrected :-)

      Cheers,
      Rob

        When using system() there remains the small problem of copying the state of the old process over into the new process. Especially copying over allocated external resources, like sockets and open file handles.

        Of course, not copying has its problems too, if one part of the fork closes/deallocates a resource while the other part wants to keep it open. But I think the other direction is more common, that only one of the two forked instances will continue to access a resource. From that direction, using OS threads to emulate fork() makes sense.

        For a very large proportion of uses of fork -- essentially all those using the 'fork & exec' idiom -- system, especially system 1, ... is a far more effective 'emulation'. The spawned process can (by default does) inherit many of the system resources -- open file, pipe and socket handles and the like -- that forked processes inherit under *nix.

        But, as Corion correctly points out, there are limitations that prevent it from being a transparent solution to porting forking programs to Windows.

        It is possible to provide a more realistic and efficient fork emulation on Windows -- one that uses real processes and even COW memory. Creating the new process and giving it COW access to the parents memory segments is relatively trivial. The difficult part is fixing up the perl internals within the spawned copy.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://929097]
Approved by BrowserUk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2014-08-22 08:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (150 votes), past polls