Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

How would you use perl debugging tools to pinpoint the problem here?

by tphyahoo (Vicar)
on Nov 10, 2005 at 15:03 UTC ( #507392=perlquestion: print w/replies, xml ) Need Help??

tphyahoo has asked for the wisdom of the Perl Monks concerning the following question:

Here's an artificially simple program that will crash your computer. (It crashed mine anyways!)
use strict; use warnings; sub_1(); sub_2(); sub sub_1 { my $hash = {}; $hash->{$_}=1 for 1 .. 10; } sub sub_2 { my $hash = {}; $hash->{$_}=1 for 1 .. 10000000; }
UPDATE: I guess this just isn't possible without using old fasioned elbow grease. In this case obviously you can see the problem by eyeballing it, but in the wilderness these problems tend to be better hidden. Could the masters of perl -d or Devel::* or whatever tool share how they would trick out the fact that

-- sub_2 is the problem
-- $hash in sub_2 is the more specific problem

Much obliged!

UPDATE: Well, I guess it's just not possible.

UPDATE 2: No, you can't have a pony (scroll to the bottom for a picture of the pony :(

  • Comment on How would you use perl debugging tools to pinpoint the problem here?
  • Download Code

Replies are listed 'Best First'.
Re: How would you use perl debugging tools to pinpoint the problem here?
by BrowserUk (Pope) on Nov 10, 2005 at 15:35 UTC

    First I would comment out half the top-level calls in the main program. In this case, commenting out sub_2(); prevents the "crash".

    Having isolated (using a binary chop) which routine is causing the problem, I'd go into it, and assuming the problem didn't leap out at me as here, I'd comment out half of the body of the sub to find which half the problem lay in. That very quickly allows you to zero in on the line that is the problem.

    There are other ways, and some of them are definitely more applicable in other languages, but using a binary chop of comments works very well and quickly in Perl (and most dynamic langauges).


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: How would you use perl debugging tools to pinpoint the problem here?
by rinceWind (Monsignor) on Nov 10, 2005 at 15:34 UTC
    $hash->{$_}=1 for 1 .. 10000000;

    What you have here is known as a memory sponge. You don't need a debugger to see what is happening. You are asking the perl process to soak up (pretty well) as much RAM as you have on your system. Whether this will:

    • Terminate with a message about exceeded resources (ulimit)

    • Cause the machine to thrash and swap with its page file

    • Lock up and require a reboot

    depends on which operating system you are running.

    Hope this helps

    --

    Oh Lord, won’t you burn me a Knoppix CD ?
    My friends all rate Windows, I must disagree.
    Your powers of persuasion will set them all free,
    So oh Lord, won’t you burn me a Knoppix CD ?
    (Missquoting Janis Joplin)

Re: How would you use perl debugging tools to pinpoint the problem here?
by tilly (Archbishop) on Nov 10, 2005 at 15:58 UTC
    With a bit of proactive work you don't have to worry about crashing your computer.
    use BSD::Resource; my $limit = 100*1024*1024; # 100 MB setrlimit(RLIMIT_AS, $limit, $limit);
    If you're using Apache, you can do this by making sure that the setrlimit utility is in your PATH, and setting the environment variable APACHE_ADDRESS_SPACE_MB to whatever you want before starting Apache.

    This doesn't stop you from having to debug the problem, but this makes the bug be less of a development problem for you.

      my $limit = 100*1024*1024; # 100 MB
      Also,
      my $limit = 100<<20; # 100 MB
      which I find to be better readable, but as usual YMMV.
Re: How would you use perl debugging tools to pinpoint the problem here?
by blazar (Canon) on Nov 10, 2005 at 16:33 UTC
    The program can be even simpler:
    #!/usr/bin/perl use strict; use warnings; my %hash; $hash{$_}=1 for 1 .. 10_000_000; __END__

    Your use of a hashref makes it slower because perl has to continuosly dereference it. But there's no need of any particular debugging tool to understand that the problem hast to do with your attempt to fill a hash with a huge number of keys.

    The fact is that perl runs quick out of memory and starts swapping to disk. At least this is what I can infer from the music my case just started playing...

    The fact that you're filling it an item at a time doesn't help either, but the keys are so many that even preassigning to keys won't help:

    #!/usr/bin/perl use strict; use warnings; use constant KEYS => 10_000_000; my %hash; keys %hash=KEYS; $hash{$_}=1 for 1 .. KEYS; __END__

    You should really consider a db based approach or a tied hash solution instead.

    Also, I'm not really sure, and I may be utterly wrong, but I suspect that due to the nature of the keys, i.e. numbers, they may behave poorly to the hashing function.

    Incidentally, were they not that many, I'd initialize it like thus instead:

    #!/usr/bin/perl use strict; use warnings; use constant KEYS => 10_000_000; my %hash; keys %hash=KEYS; @hash{1..KEYS}=(1) x KEYS; __END__

    (To be sure: this has the same problems as the above!)

Re: How would you use perl debugging tools to pinpoint the problem here?
by ikegami (Pope) on Nov 10, 2005 at 15:14 UTC
    I'm curious... How do you define crash in this instance?
      Well I'm on windows, and everything just gets slower and slower until it completely freezes up. ctrl-alt-delete kill to task manager and then kill process works to get things moving again.
        It's not crashing at all. A hash with 10 million elements is huge! Allocating a hash of that size -- at least 100MB, I bet -- takes time, and if the OS starts using virtual memory, it'll take forever.
Re: How would you use perl debugging tools to pinpoint the problem here?
by VSarkiss (Monsignor) on Nov 10, 2005 at 16:09 UTC
      Personally I don't say any reason why it should be caught with perl -c. It's not a syntax error after all. To be precise it's not an error at all. Depending on the hardware, the OS and the OS configuration, it may run just smooth...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://507392]
Approved by Limbic~Region
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2020-11-25 21:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?