As Efficient as Possible

[mr_p]

Hello Monks,

I want to make this function efficient as possible. Can someone please let me know what I need to change or add to make it so.

This function takes two arrays and diffs them, and updates $my_bonly with if element does not exist in @a_ref and it does in @b_ref. Also, combines the intersection of @a_ref and @b_ref with @my_bonly.

sub deDuppedArray
{
    my ($a_ref, $b_ref, $my_bonly, $my_isec_plus_bonly) = @_; # refere
+nce
    my (%Aseen, %Bseen, %count, @isec, @diff, @union, @aonly) = ();
    @Aseen{@$a_ref} = (); # lookup table
    @Bseen{@$b_ref} = (); # lookup table
    foreach my $e (@$a_ref, @$b_ref) { $count{$e}++ } # put all items 
+in hash table
    foreach my $e (keys %count)
    { # interate over each key of hash table
        push(@union, $e); # keys of hash table = union
        if ($count{$e} == 2)
        {
            push @isec, $e; # seen more than once = intersection
        } else
        {
            push(@aonly, $e) unless exists $Bseen{$e}; # seen once + f
+rom A = Aonly
            push(@$my_bonly, $e) unless exists $Aseen{$e}; # seen once
+ + from A = Aonly
        }
    }
    @$my_isec_plus_bonly = ( @isec, @$my_bonly);
}

previously working on thread: http://www.perlmonks.org/?node_id=832494

I don't believe this is not efficient, but I want everyones opinion on how to make it more faster like. Whether if I can do what I'm doing faster using different method.

[BrowserUk]

sub intersectPlus {
    my( $a, $b, $c, $d ) = @_;
    my %a; undef @a{ @$a };
    my %b; undef @b{ @$b };
    push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b;
    push @$d, @$c;
    return;
}

[mr_p]

Woooowww..This has to be fast.

can you please explain what this statement does?

push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b;

and

and how can u have Hash %a and array ref @$a?

[BrowserUk]

Woooowww..This has to be fast.

Don't guess. Test!

Firstly that it does the same as yours does.

And when you are sure it does, then time both with the same data.

can you please explain what this statement does? push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b;

1. It iterates over the unique values in the B array,
2. and if that value exists in the A array, it exists in both, so it pushes it to the D (intersection) array.
3. otherwise, it only exists in the B array, so it pushes it to the (B only) C array.

and how can u have Hash %a and array ref @$a?

There is no conflict. You can have $a, @a, & %a all in the same scope.

But, rename them to whatever suites your personal preference.

[crashtest]

Premature optimization is the root of all evil.
~ Donald Knuth

You fail to specify how to optimize. For execution time? Memory usage? Lines of code?

In my experience, Knuth's quote is not an exaggeration. I've done stupid stuff in the name of unneeded optimization more times than I can count. Are you having specific performance problems? You seem to indicate not. If it works, leave it alone.

[BrowserUk]

Knuth also said:

The improvement in speed from Exampe 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today's software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by pennywise-and-pound-foolish programmers, who can't debug or maintain their "optimized" programs.

In established engineering disciplines a 12 % improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering.

Of course I wouldn't bother making such optimizations on a one-shot job, but when it's a question of preparing quality programs, I don't want to restrict myself to tools that deny me such efficiencies.

[mr_p]

That is why I want to understand what the code is doing before I put it in my code base. but, I still need my code to be as fast as possible.

[mr_p]

Yes, I should have used the word optimize.

No, I am not having any problems. But the Data is time sensitive data.

[ww]

As a courtesy to the Monks, you should have included a link to your previous thread, Referencing in Function not working correctly..

Note also, this is an "I want a pony" question. We have no obligation to give you a pony.

Explain why you believe this to be other than "as efficient as possible" and support that belief with test results, benchmarks against comparable functions or some other data that shows you've put some effort into this beyond begging for a pony.

[mr_p]

I have applied my smartness as much as I could to this function.

So, I wanted the perl geniuses to look at it and let me know If I was doing something that is costing me time and If I can change it to enhance performance of the function.

basically, what I'm saying is I can not go further in this function to make it any better. I applied all my knowledge.

[moritz]

If you want good answers, please provide a short benchmark code, including samples of input data that are typical for your application.

One way to speed up your program is to get rid of @union, you just write to it but never read from it.