Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)

by kcott (Abbot)
on Mar 01, 2013 at 02:37 UTC ( #1021141=note: print w/ replies, xml ) Need Help??


in reply to Why does Perl get slower when building a larger hash? (Not due to the memory swapping)

G'day chialingh,

Welcome to the monastery.

The first thing I'd suggest doing is adding use warnings; (warnings) after use strict; and fix up problems that are reported. There may be others, but a couple that grabbed my attention were close IN; (a filehandle that is never opened) and my %hash1; ... my %hash1 ... (declared twice in same scope - ditto for %hash2).

One place where I believe you're doing unnecessary processing is:

my %file_genes = %$file_gene_ref; ... %hash1 = %{$file_genes{$m1}}; %hash2 = %{$file_genes{$m2}}; ... $intersection{$_} = $hash1{$_} if exists $hash2{$_}; ...

Rather than performing all those hash dereferences, I believe you could simply write something like:

$intersection{$_} = $file_gene_ref->{$m1}{$_} if exists $file_gene_ref->{$m2}{$_};

You'd need to handle other references to %hash1 and %hash2 in a similar fashion. At the very least, I don't believe you need to create the intermediate hash %file_genes: you could just create %hash1 and %hash2 directly, like this:

%hash1 = %{$file_gene_ref->{$m1}}; %hash2 = %{$file_gene_ref->{$m2}};

If that doesn't help, you'll need to profile your code. Devel::NYTProf may be a good place to start; other modules listed under CPAN - Development Support might be useful.

-- Ken


Comment on Re: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)
Select or Download Code
Re^2: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)
by chialingh (Initiate) on Mar 03, 2013 at 20:44 UTC

    Hi Ken,

    Thank you so much! Your solution works! :D

    I'm very curious about the reason behind this. Is it due to the memory usage?

    Chialingh

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1021141]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (13)
As of 2014-09-22 12:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (191 votes), past polls