Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Hard to Debug windows memory error

by jandrew (Chaplain)
on Dec 31, 2015 at 18:56 UTC ( #1151609=perlquestion: print w/replies, xml ) Need Help??
jandrew has asked for the wisdom of the Perl Monks concerning the following question:

I'm experiencing a windows error Exit code: 3221226356 while running a script using a module that I am trying to write. This appears to be a heap corruption problem on the windows side. When I run the script step by step in the perl debugger the script doesn't fail.

I'm running Strawberry perl 5.22.1 64 bit in windows 7. I upgraded to 5.22.1 from 5.22 just in case this was the problem and I still get the same issue. I'm wondering if there are suggestions about how to debug this problem.

Update: I'm getting Exit code: 255 mixed in with the initial error now

Replies are listed 'Best First'.
Re: Hard to Debug windows memory error
by fishmonger (Chaplain) on Dec 31, 2015 at 20:11 UTC

    This may or may not help, but you could try using Devel::Trace or Devel::DumpTrace and send its output to a log file to see what line(s) of code were being executed when it fails.

      Hi Jandrew :)

      1) I see StrawberryPerl64, have you tried with StrawberryPerl 32 bits?
      I say this because I installed version 32 bits on Windows Server 2012 64 bits for threads problem or stability.
      2) Did you profile your code?
      (if there is memory leaks, over usage of RAM...).

      With Devel::NYTProf in console:
      perl -d:nytprof<br> ./nytprofhmtl --open #to generate the html output and open the index f +ile)

        hotchiwawa Thank you for your suggestions.

        At this point I would rather try and get StrawberryPerl64 to work or at least get some kind of coherent/small failing test case rather than/before I use a 32 bit StrawberryPerl

        From a general memory standpoint I am not really budging the needle (~72M of RAM before dying) but I'm doing my checking in the task manager so there may or may not be a different result in perl. However based on my research this error code can occur (among other reasons) in windows when an attempt to access a memory location is made after the contents have been deleted. Currently I'm following up on fishmongers suggestions as well as looking into Devel::cst and Devel::SizeMe.

        Update: it looks like Devel::cst and Devel::SizeMe are both Linux specific modules requiring execinfo.h Devel::cst in this case seemed very promising.

      fishmonger++ perl -d:Trace 01-libxml.t errored out after the line from the primary module


      which I believe is a Moose addon. This seems like the biggest smoking gun.

      I got no joy from Devel::Dumptrace. The first time I ran it in verbose mode it overran the memory on my 16G Ram machine. The second time I ran it in quiet mode and I couldn't tell if it locked up or was just grinding away with no output but after about a half an hour I quit.

      Based on the last executed line from Devl::Trace as $self->_DESTROY I'm suspecting some trickiness that I tried where two instances are holding an attribute with a reference to each other. In fairness the simple case of this doesn't fail but I suspect that there probably is a really large "don't go there!" sign somewhere in the Moose documentation that I missed. I think I have some possible workarounds to try but I have reached the limit of root causing this based on available tools from the Monastery so I'm headed back to the architecture drawing board to poke at alternatives. It would have been nice if the tricky cross referencing had worked though.

      OK I'll try that. Update to follow.

Re: Hard to Debug windows memory error
by GotToBTru (Prior) on Dec 31, 2015 at 19:58 UTC

    Can you use $SIG{__DIE__} to add diagnostic code? At least to identify where the program is failing.

    But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Romans 5:8 (NASB)

      I should have mentioned that I tried that too and the script errors out before the $SIG{__DIE__} handler can print.

      Update: Windows opens a popup prior to the final error saying  Perl interpreter stopped working

      The actual error message comes up in a pop-up window not as part of the script.

Re: Hard to Debug windows memory error
by ysth (Canon) on Dec 31, 2015 at 19:25 UTC

    Do you know whether your script is actually finishing? Or even fully getting started? Try cutting out bits of the script to see whether that has any effect and to see if you can come up with a minimal complete example that still fails that you can show us.

    A math joke: r = | |csc(θ)|+|sec(θ)| |-| |csc(θ)|-|sec(θ)| |

      The very last line of my script is a print statement that does print under several tested environments.

      I suspect the problem is in the perl garbage collection. The difficulty is the module I'm writing using the script is is complex and multi-layered. I was hoping to get some debugging advice/techniques that I could use so that I could narrow it down to a simple script that reliably fails.

        Try putting some print statements in END blocks: BEGIN, UNITCHECK, CHECK, INIT and END in perlmod

        That might help you debug the unwind. You can also create some objects with controlled scope and put diagnostics in their Destructors.

        #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

Re: Hard to Debug windows memory error
by stevieb (Abbot) on Dec 31, 2015 at 20:25 UTC

    Can you make your code publicly available? Perhaps put it on github so those of us who have access to multiple platforms can test it out...

      The code that is failing is a substantial re-write of the guts of Spreadsheet::XLSX::Reader::LibXML the failing branch is iss04 and the current test I'm working on is 01-libxml.t. I appreciate any help but I'm not sure anyone needs to dive too far into that branch. It's still a hot mess.

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Jan 06, 2016 at 19:08 UTC

    Status Update 1: I have tested the fail on Strawberry perl 5.18, 5.20, and 5.22 on the failing windows machine and they all failed. Weirdly this is the only perl script failing in this way on that machine. I tried the failing script on a different windows machine and it passes. After working on reduction of complexity in the background modules the failure still occurs but has migrated to an earlier point in the script. (Although still failing at the $self->_DESTROY command when traced in the debugger) I suspect some weird corner case interaction between Strawberry perl and windows but since another windows box passes I'm now pursuing this as a pure windows problem. I just wish I was able to narrow it down enough to write a bug case for submission somewhere.

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 05, 2016 at 20:06 UTC

    Status Update 3: In a potentially unrelated issue, while working on the code base that originally prompted the question I ran into another garbage collection issue on a different windows machine. I'm still in the process of getting the test suit to fully pass after the fix but I thought it useful to post what I found since it is relevant to at least one thread in this overall node. For those in the know it turns out that perl does two-stage garbage collection and for objects that have circular references (which mine do in spades) the object destruction is deferred to the exit phase not just when the variable where the object is stored goes out of scope. There is a warning about this in perldoc for perl 5.8 but the perldocs for 5.22 Destruction are silent on this issue luckily my google foo was strong when I found this. Additionally, The very underrated but handy Perl Cookbook has a specific recipe (13.13) to fix this as well! I have included two sets of example code that represent information gleaned from both sources. The first download is an object module that is self referential but still cleans up as soon as it goes out of scope ( and the second is a script which uses the module for demonstration and then shows some de-referencing magic for simple recursively referenced objects. I have chosen to use the Cookbook style solution in my case.

    Update: removed link to the pirated copy of the Perl Cookbook with apologies

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 08, 2016 at 20:25 UTC

    Status Update 4: I have filed a ticket for Strawberry Perl based on the failing results of depends.exe however these same fails are present in a machine that passes this test so I'm now not sure that this is the root cause of my originally posted problem. I suspect that the issue lies in core perl itself at this point. I remain stuck on how to boil this issue down small enough to submit a case that will resolve the problem.

    Update: to clarify the failure appears to be the way core perl does garbage collection on libxml2 objects. (murkier) I was unable to find obvious open bugs in libxml2 or perl on this issue.

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Jan 25, 2016 at 03:10 UTC

    Status Update 2: For anybody wandering by this thread. After a full hardware test on the laptop and a complete re-image of the OS I still get the failure. Yes it continues to be hardware dependent but I don't have any way to writing a test case to submit either for bug fixing in Strawberry perl or to the laptop vendor.

        Thank you for the suggestions. It got me further down the road than before. The googleverse seems to be mixed on the missing dynamic dll's and the concrete suggestion of reinstalling visual studios 2013 - 2015 didn't fix the problem. I also can't seem to find any useful suggestions on resolving the kernelbase.dll not returning 1. I keep poking at this when I have time but clearly my attention has wandered.

        Update: I stripped the log down since it exceeds perlmonks post size. If something is relevant that was removed let me know and I can send it to you.

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on Mar 15, 2016 at 05:52 UTC

    Status Update 5: At this point I have tried everything I can think. I have found some interesting things but none of my discoveries have been conclusive and to make matters worse I have started getting some similar fails for my Travis-CI testing in linux. In the end I think I may need to persue a non XML::LibXML solution. I hate to roll a pure perl alternative but I'm not even beginning to excersize libxml2 to it's capabilites and I can't seem to get a clean test.

Re: Hard to Debug windows memory error
by jandrew (Chaplain) on May 11, 2016 at 00:19 UTC

    Status Update: 6 The package without XML::LibXML has been released as Spreadsheet::Reader::ExcelXML and is available for use. It does appear to work where the old one did not including a confirmed pass for two prior failing machines.

      Hi. I've briefly reviewed this entire thread, but I did not expand every readme, nor follow every link;

      Did you ever post (minimal) code that demonstrates this issue?

      I ask, because vaguely remember reading your OP from 5 months ago and thinking that when you got around to posting code that I could run, that demonstrated the problem, this would probably be quite simple to tie down to a cause. From my review, you never have?

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
        BrowserUk++ Thank you for offering. The original intent of my first post was indeed to look for techniques I could use to simplify the case. I did follow several suggestions provided by other monks but I remain unsuccessful. I kept working on the problem for awhile but eventually was only able to infer an issue not demonstrate it. From a practical standpoint the solution implemented in the new package doesn't fail so I feel somewhat vindicated in my inference but at best it remains just that - a guess. The obsessive compulsive person in me hates that I just walked away from the simplification task but I feel like I tried all avenues.

        If you want to review the complex case here is a link to the Travis-CI failing output and here is a link to the package code that fails or I have also posted to CPAN with a deprecation warning

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1151609]
Approved by stevieb
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (3)
As of 2018-01-20 01:46 GMT
Find Nodes?
    Voting Booth?
    How did you see in the new year?

    Results (226 votes). Check out past polls.