Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)

by kcott (Abbot)
on Mar 01, 2013 at 02:37 UTC ( #1021141=note: print w/ replies, xml ) Need Help??


in reply to Why does Perl get slower when building a larger hash? (Not due to the memory swapping)

G'day chialingh,

Welcome to the monastery.

The first thing I'd suggest doing is adding use warnings; (warnings) after use strict; and fix up problems that are reported. There may be others, but a couple that grabbed my attention were close IN; (a filehandle that is never opened) and my %hash1; ... my %hash1 ... (declared twice in same scope - ditto for %hash2).

One place where I believe you're doing unnecessary processing is:

my %file_genes = %$file_gene_ref; ... %hash1 = %{$file_genes{$m1}}; %hash2 = %{$file_genes{$m2}}; ... $intersection{$_} = $hash1{$_} if exists $hash2{$_}; ...

Rather than performing all those hash dereferences, I believe you could simply write something like:

$intersection{$_} = $file_gene_ref->{$m1}{$_} if exists $file_gene_ref->{$m2}{$_};

You'd need to handle other references to %hash1 and %hash2 in a similar fashion. At the very least, I don't believe you need to create the intermediate hash %file_genes: you could just create %hash1 and %hash2 directly, like this:

%hash1 = %{$file_gene_ref->{$m1}}; %hash2 = %{$file_gene_ref->{$m2}};

If that doesn't help, you'll need to profile your code. Devel::NYTProf may be a good place to start; other modules listed under CPAN - Development Support might be useful.

-- Ken


Comment on Re: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)
Select or Download Code
Replies are listed 'Best First'.
Re^2: Why does Perl get slower when building a larger hash? (Not due to the memory swapping)
by chialingh (Initiate) on Mar 03, 2013 at 20:44 UTC

    Hi Ken,

    Thank you so much! Your solution works! :D

    I'm very curious about the reason behind this. Is it due to the memory usage?

    Chialingh

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1021141]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (10)
As of 2015-07-08 02:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (93 votes), past polls