Re: Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windowsby BrowserUk (Pope)
|on Oct 06, 2010 at 08:54 UTC||Need Help??|
In a nutshell, when you ask Win32::Internet to do an async request, it calls you back to tell you when the request completes. That callback is called on a C thread created by the system for the purpose. Perl (obviously) has no knowledge of that thread, so it has never been initialised with an interpreter context. Hence Perl_get_context() cannot return the "context" that would be required in order to allow you to use an interpreter within that thread.
In very simplistic terms--because those are the only terms I know this in--the perl interpreter needs to retain some information--CWD %ENV, @INC etc--in order to function. Pre-threaded days, this information was stored in C static storage. One interpreter, one copy, no problem.
When threading was added, some of this previously process global state needed to be replicated on a per-thread/interpreter basis. Obviously one lump of C static storage wasn't going to cut it, so they invented "contexts". Essentially, this is a dynamically allocated piece of storage--1 per interpreter/thread--and clone the initial threads variables into it. It is the pointer to this per-thread storage that is returned by Perl_get_context().
Because the thread started by the asynchronous inet request is not started by perl, it doesn't have a "context", hence you get the null return from Perl_get_context(); and everything goes tits-up when you attempt to use the perlapi without one.
What your patch does is to bypass the initial problem by providing a "context" (PerlInterpreter *) via Perl_set_context(). But as you have never initialised that pointer to point to a PerlInterpreter structure, you're living on borrowed time.
If the callback sub attempted to do anything that modified the interpreters state--which means just about anything more than it currently does--then the perlapi would attempt to indirect through that pointer and obviously crash.
You might consider initialising it properly using perl_alloc() and perl_construct() (See perlembed for details), but you'd then have to ensure than you clean up afterward--perl_destruct() & perl_free(). And doing that for every callback would get very expensive.
You might consider initialising the perlInterpreter once, in new(), if the ASYNC flag is set, and clean it up in the DESTROY method.
Another--and I think better--option might be to not use perl in the callback at all. That is, have a pure C callback function that you pass to SetCallback() and have that store the relevant information in a C array, ("indexed" by the context value supplied on the OpenRequest()). Thereby avoiding the need to get a context in the callback.
You retain the OpenRequest() context value and use it to retrieve the information from the C array when QueryInfo() is called.
There is much more to this, much of which I don't fully understand, but I'll stop there before this post rivals yours for length :)
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.