http://www.perlmonks.org?node_id=1017540


in reply to Re^2: Optimization Help on Perl Hash Traversal (eval use)
in thread Optimization Help on Perl Hash Traversal (eval use)

Your 3 snippets can be easily converted to (far more readable) subroutines thus:

'sub {
    my( $call_ids, $call_id, $index ) = @_;
    ceil( 
        ( $call_ids->{$call_id}->{$index}->{duration_milliseconds} / 1
+000. ) / 6
    ) * 6 / 60;
}'

'sub {
    my( $call_ids, $call_id, $index ) = @_;
    if($call_ids->{$call_id}->{$index}->{ release_code } == 503 ){
        1;
    } 
    else{
        0;
    }
}'

'sub {
    my( $call_ids, $call_id, $index ) = @_;
    
    my $route      = $call_ids->{$call_id}{$index}{ route };
    my $src_state  = $call_ids->{$call_id}{$index}{ o_state }; 
    my $dest_state = $call_ids->{$call_id}{$index}{ t_state }; 
    my $juris_indicator = 'f';
    
    if( !$src_state ){
        $juris_indicator = 'c';
    }
    elsif( $src_state eq $dest_state ){
        $juris_indicator = 'a';
    }else {
        $juris_indicator = 'b';
    };
    $route =~ /^[a|b|c](1[2-9][0-9]{2}[2-9][0-9]{2})/;
    $corrected_route = $juris_indicator .  $1;
    $corrected_route = $route if $route =~ /loop|none|lnp_error|no_jur
+is_digits/;
    $corrected_route = $route if !$corrected_route;
    
    join ',', 
        $call_ids->{$call_id}{$index}{ day },
        $call_ids->{$call_id}{$index}{ day_chunk },
        $call_ids->{$call_id}{$index}{ o_trunk },
        $call_ids->{$call_id}{$index}{ t_trunk },
        $call_ids->{$call_id}{$index}{ route },
        $corrected_route;
}'

[download]

Once you have loaded them into your $agg_snippets hash, those text snippets can be replaced by instantiated subroutines in one pass using eval like this:

$agg_snippets{ $_ }{snippet} = eval $agg_snippets{ $_ }{snippet} for k
+eys %{ $agg_snippets };

[download]

Then later, when you are processing the %$call_ids hash, you can invoke them like this:

 for my $this_call_id ( sort keys %$call_ids ) {
     $count++;
     next if !$this_call_id;
     my $route_attempts = 0;
     for my $this_index ( sort keys %{ $call_ids->{ $this_call_id } } 
+) {
         $route_attempts++;
         foreach $aggregate_name ( keys %{ $agg_snippets } ){

             my $this_group_data = eval $grouping_data_eval;

             $summary_data->{$this_group_data}{$aggregate_name} = 0 if
+ !$summary_data->{$this_group_data}->{$aggregate_name};
             $summary_data->{$this_group_data}{$aggregate_name} += 
                 $agg_snippets->{$aggregate_name}{snippet}->( $call_id
+s, $this_call_id, $this_index );  
#                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                         
         }
     }
 }

[download]

That still leaves the eval of the $grouping_data_eval, which you give no information for, but it should also be possible to eliminate that eval by instantiating it into a subroutine once near the top of the code.

The overall effect should be to substantially speed up the processing. (BTW: Note how much clearer things are with: a) a little formatting; b) the omission of unnecessary punctuation; c) a little extra horizontal whitespace.)

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^4: Optimization Help on Perl Hash Traversal (eval use)
by mwb613 (Beadle) on Jun 04, 2014 at 00:39 UTC

    Thanks again for your suggestions. Unfortunately, due to the "on demand" nature of my job I often have to drop projects until I have time to dig into them again.

    Anyways, your suggestions and examples, along with BrowserUK's suggestion to store the code in YAML config files helped greatly speed up and organize the project when I got back to it last week

    Cheers!

[reply]

In Section Seekers of Perl Wisdom