Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Performance, Abstraction and HOP

by hv (Parson)
on Aug 31, 2005 at 22:17 UTC ( #488254=note: print w/ replies, xml ) Need Help??


in reply to Performance, Abstraction and HOP

I have a different viewpoint: abstraction can be an aid to performance.

Here's an example: the perl interpreter is written in C, yet for many string matching tasks, perl's regular expression engine is faster than hand-crafted C code. That sounds like a contradiction, but for most tasks the speed of string matching is not worth spending several man-years of development time on. Every time you use the regexp engine, you are leveraging the many man-years of development that have gone into making it fast.

Regular expressions (and the irregular extensions available in perl) are an example of an abstraction that gain you performance by letting you use someone else's work. The better the abstraction, the more widely applicable it is, the more benefit stands to be gained.

And by using abstractions that hide the gory details of highly optimised code, it frees up my precious development time to find performance wins in other parts of the code, or to write a Sudoku solver.

Hugo


Comment on Re: Performance, Abstraction and HOP
Re^2: Performance, Abstraction and HOP
by BrowserUk (Pope) on Aug 31, 2005 at 22:49 UTC

    I completely agree with you, with one caveat. The abstraction has to be well designed and well optimised. Perl's regex engine is a great example of abstraction done well.

    A counter example is trees & tries. These are immensely useful structures for many purposes, and there are quite a few flavours of both on CPAN. But, for the most part, they are almost useless for anything but experimentation and the most trivial of applications. They are, mostly, based upon using hashes to construct the trees, with the result that the are slow, clumsy and hugely memory hungry.

    Basing my opinion only upon the little I have read here and elsewhere, not yet having succeeded in laying my hands on a copy of HOP, the main problem with the style of coding it explores is that Perl isn't sufficiently tuned for it. To see the problem, take a look at this HOP-like implementation and constrast it (performance-wise) with procedural implementation of solutions to the same problem.

    That's not to say there isn't a lot to be learnt from the concepts explored in HOP, just that the costs of sub calls, coderefs and recursion in Perl 5 do not lend themselves to this type of programming where performance is a consideration.

    And not all Perl apps are web applications, DB or IO bound, or otherwise "interactive" where performance can be measured in response times.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

      I think that we are on the same page. I am from the Java world, and in most of cases, Perl's level of abstraction is well below the level I can accept. Abstraction is not a problem to me in general, but a flag is raised when you see the performance of some HOP-like programs. Doesn't matter how important one think performance is, or what the definition of performance is, when it becomes too slow and not practical any more, that's where you start to worry.

      I completely agree that Perl is not tuned for certain things.

      A side note, I believe that for Perl to survive the war of languages, the real winning point is "application areas". For example, Perl was once considered the language for CGI, and that's when Perl had its most glorious time. This position is no longer there for Perl, and I see Perl's hope in the future as to re-establish its position in certain application areas, not anything else. When people pick languages, I don't think they will pick Perl because of HOP.

      Did Java reach its position today because of the nice OO idea, yes but not mainly. The main reason was that it was perceived as the networking language for application areas like "internet" and "intranet" (Perception), and the strong killing will of Microsoft's rivals (Politics and Money) ;-)

      A counter example is trees & tries. These are immensely useful structures for many purposes, and there are quite a few flavours of both on CPAN. But, for the most part, they are almost useless for anything but experimentation and the most trivial of applications. They are, mostly, based upon using hashes to construct the trees, with the result that the are slow, clumsy and hugely memory hungry.
      I'm guessing that you mean that tries are implemented with hashes. I can't image trees implemented with arrays being that much more memory hungary than arrays alone. Sure, operations on trees will be slower than similar operations on arrays, but that's mostly comparing perl vs. C.
      #!/usr/bin/perl -w use strict; use Data::Dumper; my $MAX = 10000; my $tree = make_tree($MAX/2,undef,undef); for(my $x=1;$x<$MAX;$x++) { $tree = insert(int(rand($MAX)), $tree); } print "sum = ", sum_tree($tree), "\n"; #print Dumper $tree; sub sum_tree { my $tree = shift; return 0 if not defined($tree); return node($tree) + sum_tree(right($tree)) + sum_tree(left($tree) +); } sub insert { (my $elem, my $tree) = @_; if(not defined($tree)){ return make_tree($elem, undef, undef); } my $curr = node($tree); if( $elem == $curr) { return $tree; } elsif($elem < $curr) { return make_tree($curr, insert($elem,left($tree)), right($tree)); } elsif($elem > $curr) { return make_tree($curr, left($tree), insert($elem,right($tree))); } } sub make_tree { [$_[0], $_[1], $_[2]] } sub node { $_[0]->[0] } sub left { $_[0]->[1] } sub right { $_[0]->[2] }
        I can't image trees implemented with arrays being that much more memory hungary than arrays alone.

        Don't imagine--measure :)

        P:\test>junk Array[ 1..10000]: 200056 Sum @array = 50005000 Tree[ 1..10000 ]: 1120016 Sum tree = 50005000 Rate tree array tree 17.1/s -- -82% array 95.1/s 455% --

        I think that 6x bigger and 5x slower pretty much makes my point.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://488254]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (19)
As of 2014-08-01 13:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (20 votes), past polls