Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

"Free to wrong pool" error.

by davido (Archbishop)
on Oct 09, 2011 at 21:53 UTC ( #930504=perlquestion: print w/ replies, xml ) Need Help??
davido has asked for the wisdom of the Perl Monks concerning the following question:

A student I know asked me to help him find a bug in a C++ program that was using a brute force approach to finding all primes between 1 and n. The problem wasn't hard to find, and not very interesting. But with a few spare minutes on a Sunday I rewrote his C++ program using "C'ish Perl", and in so doing seem to have stumbled into a real bug.

Here's an example of the initial Perl code, which is almost an exact literal translation of the C++ code, and which DOES NOT exhibit the bad behavior:

use strict; use warnings; use v5.12; use constant TOP => 1000000; say "1 is prime."; say "2 is prime."; my $found = 2; # We already found 1 and 2. for( my $i = 3; $i < TOP; $i += 2 ) { my $qualifies = 1; for( my $j = $i - 2; $j > 1; $j -= 2 ) { if( $i % $j == 0 ){ $qualifies = 0; last; } } if( $qualifies ) { say "$i is prime."; $found++; } } say "Found $found primes between 1 and ", TOP, ".\n";

The C++ version:

#include <iostream> using namespace std; // First 501 primes are found from 1 to 3571 const int SEARCH_TO = 1000000; const int SEARCH_START = 3; /* * You can double check by comparing with the link below: * http://en.wikipedia.org/wiki/List_of_prime_numbers * to verify accuracy. */ int main() { cout << "1 is a prime number." << endl; cout << "2 is a prime number." << endl; int found = 2; int i; for ( i = SEARCH_START; i < SEARCH_TO; i+=2 ) { int qualifies = 1; for ( int j = i-2 ; j > 1; j-=2 ){ if ( i % j == 0 ) { qualifies = 0; break; } } if ( qualifies ) { cout << i << " is a prime number." <<endl; found++; } } cout << "Found " << found << " primes in range " << 1 << " to " << SEARCH_TO << ".\n"; return 0; }

...and that worked fine; no bad behavior. Then I decided to look at it without a flag, by labeling the outer loop and using "next LABEL" to short circuit to it. Here is an example:

use strict; use warnings; use v5.12; use constant TOP => 1000000; say "1 is prime."; say "2 is prime."; my $found = 2; # We already found 1 and 2. OUTER: for( my $i = 3; $i < TOP; $i += 2 ) { for( my $j = $i - 2; $j > 1; $j -= 2 ) { ( not $i % $j ) && next OUTER; } say "$i is prime."; $found++; } say "Found $found primes between 1 and ", TOP, ".\n";

After watching numbers scroll past me for a few moments I hit CTRL-C (windows console), and was surprised to get an error message from the "this shouldn't happen" category:

..... (long list of primes omitted ).... 59753 is prime. 59771 is prime. 59779 is prime. 59791 is prime. 59797 is prime. 59809 is prime. <-------I hit CTRL-C after this iteration. Out of memory! Use of uninitialized value $j in modulus (%) at primes.pl line 17. Use of uninitialized value $i in modulus (%) at primes.pl line 17. Illegal modulus zero at primes.pl line 17. Free to wrong pool 2f4d50 not 1000 during global destruction.

I ran it a few other times. Sometimes there was no problem. Sometimes the output from the end of the run had a bunch of "line noise" after the last prime. And a couple times it said, "Panic..." with a similar error message.

This post isn't about how to efficiently generate primes, or how to write such code elegantly. I'll be the first to admit this is a C-ish style approach, and the algorithm used is dumb brute force.

The question is, does anyone know why there's an error occurring here? The errors generated really are in the category of errors that we shouldn't see. Since I can't produce it on 100% of the runs, I don't know how to wrap it all in a test that would demonstrate the bug reliably. If I could at least generate it 100% of the time (and without requiring the intervention of "CTRL-C" keyboard input) I might be able to send in a meaningful bug report.

Update: A couple of responses (as well as my own testing on Linux) have led me to believe this problem is not going to manifest itself on Linux. While I haven't heard from any Mac users, I suspect this is a problem only under a Windows environment. Whether or not it's limited to Strawberry Perl, or only to version 5.12.3, I don't know yet.


Dave

Comment on "Free to wrong pool" error.
Select or Download Code
Re: "Free to wrong pool" error.
by choroba (Abbot) on Oct 09, 2011 at 21:59 UTC
    I can't reproduce it, running perl 5.12.1 on Linux (openSUSE 11.3).

      I did notice its frequency is greater on runs of over 18000 iterations (Update: On further testing I found that theory not to hold up. The rest of the thread stands.</end update>), and even then it's about 30% of the time when I terminate by keyboard intervention.

      Hmm, what other variables..... It's Windows Vista SP2. Perl version is 5.12.3, Strawberry Perl.

      Here's another example run, where I terminated the loop earlier.

      6043 is prime. 6047 is prime. 6053 is prime. Out of memory! panic: leave_scope inconsistency at C:\Users\Dave\Desktop\primes.pl li +ne 15. panic: leave_scope inconsistency at C:\Users\Dave\Desktop\primes.pl li +ne 15. panic: leave_scope inconsistency at C:\Users\Dave\Desktop\primes.pl li +ne 15. panic: leave_scope inconsistency at C:\Users\Dave\Desktop\primes.pl li +ne 15.

      And here's another example:

      4409 is prime. 4421 is prime. 4423 is prime. Out of memory! Use of uninitialized value $i in concatenation (.) or string at C:\Use +rs\Dave\Desktop\primes.pl line 18. is prime. 2 is prime. Label not found for "next OUTER" at C:\Users\Dave\Desktop\primes.pl li +ne 16.

      ...and one more...

      6271 is prime. 6277 is prime. Out of memory! panic: leave_scope inconsistency at primes.pl line 15.

      Dave

Re: "Free to wrong pool" error. (hints)
by tye (Cardinal) on Oct 09, 2011 at 23:54 UTC

    "Out of memory!" makes no sense for that code. It seems clear to me that Perl's Ctrl-C handler (for Windows) is having a very hard time. "Free to wrong pool 2f4d50 not 1000 during global destructio≠n" makes no sense since you aren't using threads. But that bit of hex translates to "/MP" so something is overwriting memory that it shouldn't be.

    - tye        

      Thanks for the tips, tye. I had a feeling that the CTRL-C wasn't being handled gracefully. As I fiddle, any time my code retains the last OUTER;, where it jumps from an inner loop to the outer loop, I get the behavior. So maybe the error messages that mention scope are on to something.

      I do wish I could reproduce it in a way where a test could be written that would demonstrate it reliably.


      Dave

Re: "Free to wrong pool" error.
by kennethk (Monsignor) on Oct 10, 2011 at 01:05 UTC
    I tested the posted code with
    C:\>Perl-5.12\bin\perl.exe -v This is perl 5, version 12, subversion 4 (v5.12.4) built for MSWin32-x +86-multi-t hread (with 9 registered patches, see perl -V for more detail) Copyright 1987-2010, Larry Wall Binary build 1205 [294981] provided by ActiveState http://www.ActiveSt +ate.com Built Jun 20 2011 18:35:25
    following the directions above and got multiple error codes, including:
    • Out of memory! panic: leave_scope inconsistency at primes.pl line 13.
    • Can't coerce F&#9829; ns with other barewords, keywords, and subroutine names. Constant names must begin with a letter or underscore. Names beginning with a double underscore are reserved. Som +e poor choices for names will generate warnings, if warnings are enabled + at compiUNKNOWN to string in leave at primes.pl line 14. Use of uninitialized value in concatenation (.) or string at primes.pl + line 16.
    • Can't coerce UNKNOWN to string in rds, keywords, and subroutine names. Constant names must begin with a letter or underscore. Names beginning with a double underscore are reserved. Som +e poor choices for names will generUNKNOWN ings, if warnings are enabled + at compile time. =head2 List constsubtract at primes.pl line 13.
    • Terminating on signal SIGINT(2)
    • Can't coerce UNKNOWN to string in £~Ť&#9786;gt at primes.pl line 13. Use of uninitialized value in numeric gt (>) at primes.pl line 13.
    • Can't coerce UNKNOWN to string in t&#9786;rds, keywords, and subroutine names. Constant names must begin with a letter or underscore. Names beginning with a double underscore are reserved. Som +e poor choices for namUNKNOWN KNOWN te warnings, if warnings are enabled + at compile time. 2677 is prime. =head2 List constants Constants may be lists of munstack at primes.pl line 13. panic: leave_scope inconsistency at †††† eir last value in scalar context as one might expect. They currently return the number scalar context as one might expect. They currently return the number Label not found for "next OUTER" at primes.pl line 14. Label not found for "next OUTER" at primes.pl line 14. of values, but B<this may change in the future>. Do not use constants with multiple values in scalar context. B<NOTE:> This implies that the expression defining the value of a constant is evaluated in list context. This primes.pl line 13.
    I don't know if this information is terribly diagnostically useful.
      I got a access violation null ptr in safesysrealloc in util.c at line 199.

      Callstack, non-perl thread
      perl512.dll!Perl_safesysrealloc(void * where=0x018612ac, unsigned +int size=44) Line 199 + 0x9 C perl512.dll!Perl_sv_grow(interpreter * my_perl=0x00353d7c, sv * c +onst sv=0x003578ec, unsigned int newlen=28) Line 1555 + 0xd C perl512.dll!Perl_sv_catpvn_flags(interpreter * my_perl=0x00353d7c +, sv * const dsv=0x003578ec, const char * sstr=0x2821454c, const unsi +gned int slen=25, const long flags=2) Line 4780 + 0x2c C perl512.dll!Perl_sv_vcatpvfn(interpreter * my_perl=0x00353d7c, sv + * const sv=0x003578ec, const char * const pat=0x2821454c, const unsi +gned int patlen=32, char * * const args=0x003bfeec, sv * * const svar +gs=0x00000000, const long svmax=0, char * const maybe_tainted=0x00000 +000) Line 9504 + 0x1a C perl512.dll!Perl_sv_vsetpvfn(interpreter * my_perl=0x00353d7c, sv + * const sv=0x003578ec, const char * const pat=0x2821454c, const unsi +gned int patlen=32, char * * const args=0x003bfeec, sv * * const svar +gs=0x00000000, const long svmax=0, char * const maybe_tainted=0x00000 +000) Line 9259 + 0x25 C perl512.dll!Perl_vmess(interpreter * my_perl=0x00353d7c, const ch +ar * pat=0x2821454c, char * * args=0x003bfeec) Line 1197 + 0x28 C perl512.dll!Perl_vwarn(interpreter * my_perl=0x00353d7c, const ch +ar * pat=0x2821454c, char * * args=0x003bfeec) Line 1455 + 0x11 C perl512.dll!Perl_warn(interpreter * my_perl=0x00353d7c, const cha +r * pat=0x2821454c, ...) Line 1495 + 0x11 C perl512.dll!sig_terminate(interpreter * my_perl=0x00353d7c, int s +ig=2) Line 2112 + 0x1d C perl512.dll!win32_ctrlhandler(unsigned long dwCtrlType=0) Line 4 +689 + 0xb C kernel32.dll!_CtrlRoutine@4() + 0x19c kernel32.dll!_BaseThreadStart@8() + 0x37
      main perl thread
      ntdll.dll!_DbgPrint() ntdll.dll!_RtlDebugFreeHeap@12() + 0x97 ntdll.dll!_RtlFreeHeapSlowly@12() + 0x246cf ntdll.dll!_RtlFreeHeap@12() + 0x17646 msvcrt.dll!_free() + 0xc3 > perl512.dll!VMem::Free(void * pMem=0x01832384) Line 229 + 0xa + C++ perl512.dll!CPerlHost::Free(void * ptr=0x01832384) Line 68 + 0x2 +2 C++ perl512.dll!PerlMemFree(IPerlMem * piPerl=0x0024577c, void * ptr= +0x01832384) Line 311 C++ perl512.dll!Perl_safesysfree(void * where=0x01832384) Line 262 + + 0x1a C perl512.dll!Perl_sv_clear(interpreter * my_perl=0x00353f4c, sv * +const sv=0x00357afc) Line 5865 + 0xc C perl512.dll!Perl_sv_free2(interpreter * my_perl=0x00353f4c, sv * +const sv=0x00357afc) Line 5985 + 0xd C perl512.dll!Perl_sv_free(interpreter * my_perl=0x00353f4c, sv * c +onst sv=0x00357afc) Line 5962 + 0xd C perl512.dll!Perl_free_tmps(interpreter * my_perl=0x00353f4c) Lin +e 167 + 0xd C perl512.dll!Perl_pp_unstack(interpreter * my_perl=0x00353f4c) Li +ne 218 + 0x17 C perl512.dll!Perl_runops_debug(interpreter * my_perl=0x00353f4c) +Line 2049 + 0xd C perl512.dll!S_run_body(interpreter * my_perl=0x00353f4c, long old +scope=1) Line 2308 + 0xd C perl512.dll!perl_run(interpreter * my_perl=0x00353f4c) Line 2233 + + 0xd C perl512.dll!RunPerl(int argc=2, char * * argv=0x00243ec8, char * +* env=0x00244f20) Line 270 + 0x9 C++ perl.exe!main(int argc=2, char * * argv=0x00243ec8, char * * env= +0x00242c58) Line 23 + 0x12 C perl.exe!mainCRTStartup() Line 398 + 0xe C kernel32.dll!_BaseProcessStart@4() + 0x23
      C:\Documents and Settings\Owner\Desktop\w32i>perl -v This is perl 5, version 12, subversion 2 (v5.12.2) built for MSWin32-x +86-multi- hread Copyright 1987-2010, Larry Wall Perl may be copied only under the terms of either the Artistic License + or the GNU General Public License, which may be found in the Perl 5 source ki +t. Complete documentation for Perl, including FAQ lists, should be found +on this system using "man perl" or "perldoc perl". If you have access to + the Internet, point your browser at http://www.perl.org/, the Perl Home Pa +ge. C:\Documents and Settings\Owner\Desktop\w32i>perl -V Summary of my perl5 (revision 5 version 12 subversion 2) configuration +: Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=und +ef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -Od -MD -Zi -DDEBUGGING -DWIN32 + -D_CONSO E -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLIC +IT_SYS -D SE_PERLIO', optimize='-Od -MD -Zi -DDEBUGGING', cppflags='-DWIN32' ccversion='13.10.3077', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64 +', lseeks ze=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -libpath:"c:\pe +rl512\lib CORE" -machine:x86' libpth="c:\Program Files\Microsoft Visual Studio .NET 2003\VC7\lib +" libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib + comdlg3 .lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uui +d.lib ws2 32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32 +.lib msvc t.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool +.lib com lg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib + uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comc +tl32.lib svcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl512.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -lib +path:"c:\ erl512\lib\CORE" -machine:x86' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PERL_TRACK_MEMPOOL PL_OP_SLAB +_ALLOC USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF Built under MSWin32 Compiled at Mar 23 2011 08:10:43 %ENV: @INC: C:/perl512/site/lib C:/perl512/lib .
      4012K ram in task manager

      Its a race condition with perl's malloc system and multiple threads using 1, non locked interpreter at the same time. Ill keep playing around. Theres probably something in RT about making Ctrl-C into a deferred signal/safe signal rather than current way. Maybe a Perl SIGINT handler will stabilize up perl?

      This doesn't crash since the ctrl-c is now a deferred signal, before it was instant signal. see win32/win32.c#l3989 in perl.git and win32/win32.c#l1990 in perl.git<- this one decides whether to deal with it "now" or put it on the defer queue and then asynccheck() in runops will pick it up at a "safe" time. Im sure if I search RT ill find other complaining about this. Problem is the signal is raised in a different thread than perl. yeah, maybe ill keep updating this post Yes, its a windows only problem. Unix signals appear (correct if im wrong, im a windows person), as your signal handler being randomly assembly level jumped to in your thread. On windows, it seems, your signal handler is run in a new thread created just to run your signal handler (read http://msdn.microsoft.com/en-us/library/ms682541%28VS.85%29.aspx ). Its just the more windows way to do things. The idea of set/lngjmps and interrupts/signals is akin to truck bombs in windows programing. Although every perl croak/die is implemented as a perl stack unwind (not C ++ stack unwind), and then a longjmp (crude C stack memory freeing, all heap/mallocs and all C++ are leaked forever if you use longjump aka Croak/die). croaking through a 3rd party C library is begging to crash, main perl->XS->3rd party library->C callback in XS-perl language->XS->croak means disaster, perl and typical XS code never use heap pointers that aren't attached to a SV * somehow. But 3rd party librarys obviously use malloc and have no idea what a sv * is, so longjumping through them is stupid. Longjumping through C++ is running at cops with knife insane.
      #!/usr/bin/perl -w use strict; use warnings; use v5.12; use constant TOP => 1000000; say "1 is prime."; say "2 is prime."; $SIG{INT} = sub { print "ctrl c hit\n"; exit(); }; my $found = 2; # We already found 1 and 2. OUTER: for( my $i = 3; $i < TOP; $i += 2 ) { for( my $j = $i - 2; $j > 1; $j -= 2 ) { ( not $i % $j ) && next OUTER; } say "$i is prime."; $found++; } say "Found $found primes between 1 and ", TOP, ".\n";
      Lets run your script through deparse. Always interesting what the perl interp thinks it is.
      C:\Documents and Settings\Owner\Desktop>perl -MO=Deparse n8.pl BEGIN { $^W = 1; } sub BEGIN { use warnings; use strict 'refs'; require v5.12; } use constant ('TOP', 1000000); use warnings; use strict 'refs'; BEGIN { $^H{'feature_unicode'} = q(1); $^H{'feature_say'} = q(1); $^H{'feature_state'} = q(1); $^H{'feature_switch'} = q(1); } say '1 is prime.'; say '2 is prime.'; my $found = 2; OUTER: for (my $i = 3; $i < 1000000; $i += 2) { for (my $j = $i - 2; $j > 1; $j -= 2) { next OUTER unless $i % $j; } say "$i is prime."; ++$found; } say "Found $found primes between 1 and ", 1000000, ".\n"; n8.pl syntax OK C:\Documents and Settings\Owner\Desktop>
      Its a bug that perl's malloc wrapper is not multithread safe, yet on win32 it is called from a non-perl thread. File a bug report. If you add,
      $SIG{INT} = sub { print "ctrl c hit\n"; exit(); };
      the bug goes away because you created a "safe signal" rather than shutting down the perl interpreter from another thread.

      I also "fixed" (its not a fix) the problem with the following change
      #!/usr/bin/perl -w use strict; use warnings; use Time::HiRes 'usleep'; use v5.12; use constant TOP => 1000000; say "1 is prime."; say "2 is prime."; #$SIG{INT} = #sub { # print "ctrl c hit\n"; # exit(); #} ; my $i; my $j; my $found = 2; # We already found 1 and 2. OUTER: for( $i = 3; $i < TOP; $i += 2 ) { for( $j = $i - 2; $j > 1; $j -= 2 ) { if(( not $i % $j )) { usleep(100); next OUTER; } } say "$i is prime."; $found++; } say "Found $found primes between 1 and ", TOP, ".\n";
      The usleep turns it from 99% crash/console warning noise to 99% success. Success is
      3557 is prime. 3559 is prime. Terminating on signal SIGINT(2) C:\Documents and Settings\Owner\Desktop>
Re: "Free to wrong pool" error.
by Khen1950fx (Canon) on Oct 10, 2011 at 08:01 UTC
    I tried it on my old FC6, 2Ghz processor, 512mb ram. It ran ok until it reached 175000 when I got a "temperature above threshold" warning. To get around that, I rewrote it like this, trying to keep to "brute force":
    #!/usr/master/bin/perl -l use Modern::Perl; use Parallel::Runner; use Data::Dumper::Concise; use Math::Prime::XS qw(mod_primes); $|=1; my $runner = Parallel::Runner->new(1); $runner->run( sub { my @all_primes = mod_primes(); print Dumper @all_primes; }); $runner->finish;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://930504]
Approved by Perlbotics
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2014-08-29 10:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (277 votes), past polls