Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Perl interpreter stops when writing logfile - why?

by Ratazong (Monsignor)
on Dec 09, 2014 at 13:32 UTC ( #1109723=perlquestion: print w/replies, xml ) Need Help??
Ratazong has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks!

I have a nested hash data-structure with about 80000 entries, which I want to dump to a logfile for debugging reasons. I use the following line:

print( Data::Dumper->Dumpxs( [ \ %WPs ], [ qw{ *WPs } ] ));
The output is directed to the logfile by the Windows-7 redirection
D:\tools\perl\bin\perl -Id:/tools/perl/lib -w report.xlsx > l +og.txt
Unfortunately, I get a pop-up-error-window stating that the perl-interpreter is no longer working and needs to be stopped. Looking at the logfile, this error occurs somewhere "in the middle" of the line above - the hash has not been fully dumped. I am using ActiveState Perl 5.8.2 under Windows 7.

Do you have any idea why this could happen and how to prevent it? Fortunately, it is only a minor annoyance, as the script finished processing, and the error occurs afterwards - nevertheless it would be fine if there would be an easy explanation (is there a limit of characters when redirecting script-output?) and a workaround.

Thanks in advance! Rata

Replies are listed 'Best First'.
Re: Perl interpreter stops when writing logfile - why?
by Corion (Pope) on Dec 09, 2014 at 13:44 UTC

    Constructing the string for the dump of a nested hash structure with about 80_000 entries might be too large for your (presumably) 32-bit Perl. In theory, Perl should quit with Out of memory, but maybe it doesn't here. Have you considered using Storable or something else to get the data out? Alternatively, don't dump the nested hash structure as one big thing but dump it in loops:

    my %struct= (80_000 other hashes); for my $key (sort keys %struct) { print {LOGFILE} Dumper $struct{ $key }; };
Re: Perl interpreter stops when writing logfile - why?
by LanX (Bishop) on Dec 09, 2014 at 13:37 UTC
    I can only guess this might be an OS related "feature" to check if an app is still "responsive" by sending signals or monitoring resources.

    Perl (to be precise the dumper XS code) might be too busy writing and is not replying.

    Cheers Rolf

    (addicted to the Perl Programming Language and ☆☆☆☆ :)

      Dear LanX, Corion and Anonymous Monk!

      Thanks for your suggestions! Here are some results of my tests:

      • It is depending on the perl-version ... perl 5.14.2 works without problems
      • It really seems to depend on the size of the dump - the result is the same independently if I write directly to the logfile or if I redirect
      • Even with dumping the data in a loop (with redirection), the same error occured (there might be a problem that the perl-script writes the data faster than it could be processed ...)
      Summary: I "solved" the problem by reducing the amount of debugging-info, and by using a newer perl-version when all data is really needed. Your comments helped me reaching this solution :-) .

      Thanks a lot! Rata

        > Even with dumping the data in a loop 

        Maybe try to sleep or usleep between iterations ... just to nail down the original problem. :)

        Cheers Rolf

        (addicted to the Perl Programming Language and ☆☆☆☆ :)

Re: Perl interpreter stops when writing logfile - why?
by Anonymous Monk on Dec 09, 2014 at 13:42 UTC

    If LanX is right, then a workaround might be to write the file from the Perl script instead of using redirection.

    Also, that's a pretty old version of Perl, why not update?

      On second thought, the suggested workaround actually doesn't relate to LanX's post - it's a workaround for the possibility of something strange going on with redirection, not dumping.

      Adding an option to your script to write to an output file instead of STDOUT might still be worth it, since then you have more control over it and don't have to rely on the shell for that.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1109723]
Approved by toolic
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (7)
As of 2018-04-25 20:56 GMT
Find Nodes?
    Voting Booth?