Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windows

by sundialsvc4 (Monsignor)
on Oct 06, 2010 at 13:17 UTC ( #863785=note: print w/ replies, xml ) Need Help??


in reply to Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windows

In situations similar-but-different to this, I also have found that the best thing to use for “a callback” is a very short stub routine ... written in baggage-free “C.”   All that the callback routine does, really, is to store relevant status information and send some kind of a signal.   High-level languages such as Perl usually carry a lot of baggage around with them ... but a callback does not really need to have one.

Mutual-exclusion, of course, is quite a necessity ... but in the callback routine you use the same low-level OS primitives that you know the high-level language will also respect and use.   (You aren’t using the language’s constructs, but you make certain that the two pieces of code will interlock as they should.)

I have also learned, again quite painfully, that whenever an API provides for a “handle” (as they, of course, always do...), that handle ought to be a random number that can be safely looked-up in a hash owned by you.   In other words, it should never be “a pointer.”   Aside from the fact that we are now routinely dealing with 64-bit addresses (and 32-bit legacy integer handles), a pointer can’t be verified ... so it is a “memory stomp” waiting to happen.   Random handles, on the other hand, can be looked-up and thereby verified... and if they are no longer present in the hash structure (for whatever reason), we have “a reportable bug-condition” (probably), “but not a stomp.”

The callback logic, then, might go something like this:

  1. Grab a mutual-exclusion object (using a low-level OS call).
  2. Look up the handle that is provided.   If it cannot be found, release the mutex and exit silently.
  3. Store the information into the data-structure thus located.   This data structure should be a buffer that the high-level language side already carved out for you.   Check it for bogosity...
  4. Signal an event that the main logic might be waiting for.   Unlock the mutex and exit.


Comment on Re: Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windows
Re^2: Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windows
by patcat88 (Deacon) on Oct 07, 2010 at 05:32 UTC
    My idea of stress testing the algorithm is removing the sleep from while not done loop in the sample script, and replacing the sleep with usleep(1);.

    I've added mutexes guarding the getting of the callback status from the global PM hash, and the calling of the Perl method that sets the status into the global PM hash from XS.

    I consider stability, FOR NOW, to be 10 runs without crashes or perl panics or strange errors on the console.
    void WINAPI CALLBACK PerlCallback(HINTERNET h, DWORD context, DWORD mystatus, LPVOID mystatusinfo, DWORD mystatuslength) { DWORD myret; HINTERNET myhandle; // Let's try with perl_call_method // ...to clarify: // a C routine // called from Perl // callbacks this C routine // that callbacks a Perl routine. // ;) // if(mystatus!=status) { SV **sp; HANDLE hMutex; char mutexname [sizeof("Win32InternetPerlMutex") + sizeof(DW +ORD)*8]; DWORD waitres; //char [23 + sizeof(DWORD)*8] mutexname; //void * myptr; //printf("PerlCallback: entering\n"); printf("PerlCallback: got handle=%d context=%d mystatus=%d\n", +h,context,mystatus); //printf("PerlCallback: myWin32InternetPerlInterpreterPtr=%p\n +",myWin32InternetPerlInterpreterPtr); //comment below out to trigger original crash/bug in Win32::In +ternet and recompile PERL_SET_CONTEXT(myWin32InternetPerlInterpreterPtr); //printf("PerlCallback: ThreadId=%d\n",GetCurrentThreadId()); //printf("PerlCallback: GetLastError() =%d\n",GetLastError()); //myptr = PERL_GET_CONTEXT; //printf("PerlCallback: GetLastError() =%d\n",GetLastError()); //printf("PerlCallback: pointer from Perl_get_context() =%p\n" +,myptr); //can't use dSP because we are after variable declarations sp = PL_stack_sp; ENTER; SAVETMPS; PUSHMARK(sp); // XPUSHs(sv_2mortal(newSVpv("Win32::Internet\0",0))); XPUSHs(sv_2mortal(newSViv(context))); XPUSHs(sv_2mortal(newSViv(mystatus))); switch(mystatus) { case INTERNET_STATUS_HANDLE_CREATED: myhandle=(HINTERNET) *(LPHINTERNET)mystatusinfo; XPUSHs(sv_2mortal(newSViv((DWORD) myhandle))); break; case INTERNET_STATUS_RESPONSE_RECEIVED: case INTERNET_STATUS_REQUEST_SENT: myret=(DWORD) *(LPDWORD)mystatusinfo; // printf("PerlCallback: received/sent(%d) %d bytes\n",mys +tatus,myret); XPUSHs(sv_2mortal(newSViv(myret))); break; default: XPUSHs(sv_2mortal(newSViv(0))); break; } PUTBACK; // printf("PerlCallback: calling callback with context=%d, sta +tus=%d\n",context,mystatus); sprintf(mutexname ,"Win32InternetPerlMutex%u", GetCurrentPro +cessId()); //printf("PerlCallback: mutexname=%s\n",mutexname); hMutex = OpenMutex(MUTEX_ALL_ACCESS, FALSE, mutexname); if (hMutex == NULL) printf("PerlCallback: OpenMutex error: % +d\n", GetLastError() ); else printf("PerlCallback: OpenMutex successfully opened the + mutex.\n"); waitres = WaitForSingleObject(hMutex,500000); printf("PerlCallback: wait result =%d\n",waitres); perl_call_pv("Win32::Internet::callback", G_DISCARD); printf("PerlCallback: release mutex res=%d\n",ReleaseMutex(h +Mutex)); CloseHandle(hMutex); FREETMPS; LEAVE; // } // status=mystatus; // return; }
    was never stable until usleep(100000);
    void WINAPI CALLBACK PerlCallback(HINTERNET h, DWORD context, DWORD mystatus, LPVOID mystatusinfo, DWORD mystatuslength) { DWORD myret; HINTERNET myhandle; // Let's try with perl_call_method // ...to clarify: // a C routine // called from Perl // callbacks this C routine // that callbacks a Perl routine. // ;) // if(mystatus!=status) { SV **sp; HANDLE hMutex; char mutexname [sizeof("Win32InternetPerlMutex") + sizeof(DW +ORD)*8]; DWORD waitres; //char [23 + sizeof(DWORD)*8] mutexname; //void * myptr; //printf("PerlCallback: entering\n"); sprintf(mutexname ,"Win32InternetPerlMutex%u", GetCurrentPro +cessId()); //printf("PerlCallback: mutexname=%s\n",mutexname); hMutex = OpenMutex(MUTEX_ALL_ACCESS, FALSE, mutexname); if (hMutex == NULL) printf("PerlCallback: OpenMutex error: % +d\n", GetLastError() ); else printf("PerlCallback: OpenMutex successfully opened the + mutex.\n"); waitres = WaitForSingleObject(hMutex,500000); printf("PerlCallback: wait result =%d\n",waitres); printf("PerlCallback: got handle=%d context=%d mystatus=%d\n", +h,context,mystatus); //printf("PerlCallback: myWin32InternetPerlInterpreterPtr=%p\n +",myWin32InternetPerlInterpreterPtr); //comment below out to trigger original crash/bug in Win32::In +ternet and recompile PERL_SET_CONTEXT(myWin32InternetPerlInterpreterPtr); //printf("PerlCallback: ThreadId=%d\n",GetCurrentThreadId()); //printf("PerlCallback: GetLastError() =%d\n",GetLastError()); //myptr = PERL_GET_CONTEXT; //printf("PerlCallback: GetLastError() =%d\n",GetLastError()); //printf("PerlCallback: pointer from Perl_get_context() =%p\n" +,myptr); //can't use dSP because we are after variable declarations sp = PL_stack_sp; ENTER; SAVETMPS; PUSHMARK(sp); // XPUSHs(sv_2mortal(newSVpv("Win32::Internet\0",0))); XPUSHs(sv_2mortal(newSViv(context))); XPUSHs(sv_2mortal(newSViv(mystatus))); switch(mystatus) { case INTERNET_STATUS_HANDLE_CREATED: myhandle=(HINTERNET) *(LPHINTERNET)mystatusinfo; XPUSHs(sv_2mortal(newSViv((DWORD) myhandle))); break; case INTERNET_STATUS_RESPONSE_RECEIVED: case INTERNET_STATUS_REQUEST_SENT: myret=(DWORD) *(LPDWORD)mystatusinfo; // printf("PerlCallback: received/sent(%d) %d bytes\n",mys +tatus,myret); XPUSHs(sv_2mortal(newSViv(myret))); break; default: XPUSHs(sv_2mortal(newSViv(0))); break; } PUTBACK; // printf("PerlCallback: calling callback with context=%d, sta +tus=%d\n",context,mystatus); perl_call_pv("Win32::Internet::callback", G_DISCARD); FREETMPS; LEAVE; printf("PerlCallback: release mutex res=%d\n",ReleaseMutex(h +Mutex)); CloseHandle(hMutex); // } // status=mystatus; // return; }
    Only 1 out of 5 runs weren't stable at usleep(10000); in the getstatus loop in the test script.

    I'm guessing that some of what BrowserUk said in Win32::Internet crash, XS, callbacks, perl stack & context, windows api, interpreter thread safety, 1 perl, many C threads by windows, in suggesting starting a new perl instance every time the C callback was fired by Windows, rather than using the main perl instance, is the "safe" way to do it.

    I'm guessing a single perl instance isn't thread safe. From my amateur looking at the XS of [threads-shared], the "shared" variables have their own perl instance that lives with them, and the "context" is switched around before using perl functions. Neither the parent/main thread or child thread perl interpreters ever get to operate on the shared variables, I think. Mutexes guard the shared interpreter from being commanded by 2 or more threads at the same time I think.

    Saving a copy of the perl instance pointer to use from another thread would only work if I can put the other, main thread to sleep, without question. I wonder if http://msdn.microsoft.com/en-us/library/ms686345%28VS.85%29.aspx would work? or is it completely unacceptable to use that in anything remotely approaching production code?

      I'm looking forward to seeing sundialsvc4's detailed and insightful responses to your questions.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://863785]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (9)
As of 2014-07-24 20:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (167 votes), past polls