http://www.perlmonks.org?node_id=717906

xxqtony has asked for the wisdom of the Perl Monks concerning the following question:

Hi, hopefully someone can help me out.

Short version: my scripts is a GUI wrapper which directly/indirectly calls other applications, it runs ok with small datasets, but dies when I test it with large datasets.

Long form: I'm using Perl/Tk to design a GUI, in the scripts a subroutine A calls another Perl scripts call_B.pl, which in turn calls a third party application C. I have lines like this:

my $call = system("call_B.pl -a option1 -b option2 -c option3");

my $res = $path . "$result_file_generated_by_application_C";

The lines work no problem when my datasets are small (1 Gigs), and with small datasets, it takes about 2 or 3 minutes for C to complete; however the line of 'my $res' complains uninitilized $result_file_generated_by_application_C when my testing datasets are big (10 Gigs); with big datasets, it usually takes 10-30 minutes for C to complete. In the failing case, I found no result_file_generated_by_application_C file generated. In another word, it looks like my scripts keeps going forward even the calling of call_B.pl (and then calling application C) does not finish.

I wonder if the problem is because my scripts calls B, which is submitted and returned, even C is still running. However, if my guess is right, then it should have problem too even when my datasets are small. Since no code available for application C, I think maybe I should try to incorporate call_B.pl codes into my scripts, which I hate to do because of legal issue.

Anyone knows what might cause the problem when my datasets are big? I don't think it's memory issue, because it can run in non-GUI version no problem.
  • Comment on scripts die when calling other program, but only certian circumstances

Replies are listed 'Best First'.
Re: scripts die when calling other program, but only certian circumstances
by Perlbotics (Archbishop) on Oct 18, 2008 at 09:10 UTC

    Can you elaborate on die? Do you mean, your GUI window disappears, or does the GUI freeze ... ?

    You could check:

    • the return value $call: was a signal caught?
    • the string given to system(): does it contain characters which are interpreted by the shell - maybe in the 'C'-case only? Alternatively, try system with a list argument.
    • for any output your chain of calls send to STDOUT and STDERR
    • experimentally, if the problem is in your GUI by running e.g. system("sleep 2000") instead of call_B.pl
    • memory consumption again

    Because of ...is submitted and returned, even C is still running..., maybe an unescaped & sneaked into the system call? That could also explain the uninitialised $res, because the result is not yet ready for long running tasks when the GUI checks for the result-files existence. For short running tasks, some (maybe not all) results are processed and the result file is already opened/created, giving the impression the GUI work fine for small tasks. Well, just some speculations...

Re: scripts die when calling other program, but only certian circumstances
by zentara (Archbishop) on Oct 18, 2008 at 12:44 UTC
    If I understand your question correctly, you probably need to put your long db stuff into a separate thread. Calling a long system command in a Tk program will interfere with the eventloop, and make it unresponsive while system is running.

    As a quick test, can you fork off the commands(instead of system), and write the results to a file? If forked, does Tk stay alive?


    I'm not really a human, but I play one on earth Remember How Lucky You Are
Re: scripts die when calling other program, but only certian circumstances
by Anonymous Monk on Oct 18, 2008 at 06:21 UTC
    when a program die's, there's an error message