Beefy Boxes and Bandwidth Generously Provided by pair Networks kudra
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

The Great Computer Language Shootout

by yperl (Novice)
on Sep 21, 2004 at 16:07 UTC ( #392696=perlquestion: print w/ replies, xml ) Need Help??
yperl has asked for the wisdom of the Perl Monks concerning the following question:

Hi All!

Each time I visit this site,

http://shootout.alioth.debian.org/

I'll be angry.

Perl is all the time at the end of the score list. Even for regular expressions, its badly scored.

Could we try to change the algorithms to get our prefered language on the top??

Regards,

Younès

Comment on The Great Computer Language Shootout
Re: The Great Computer Language Shootout
by davido (Archbishop) on Sep 21, 2004 at 16:13 UTC

    Can't place much stock in a website that when I try to download the shootout results, the PHP script that is supposed to handle the shootout download errors out:

    #! /usr/bin/php4 -f Warning: Cannot add header information - headers already sent by (outp +ut started at /usr/share/gforge/www/tarballs.php:3) in /usr/share/gfo +rge/www/include/exit.php on line 43 Software/Group Skill People .... Error Error creatingGroup object

    People can devise tests that favor any agenda. Your most significant test should be your own productivity.


    Dave

Re: The Great Computer Language Shootout
by hardburn (Abbot) on Sep 21, 2004 at 16:15 UTC

    I have reaffirmed a long-standing and strongly held view: Language comparisons are rarely meaningful and even less often fair. A good comparison of major programming languages requires more effort than most people are willing to spend, experience in a wide range of application areas, a rigid maintenance of a detached and impartial point of view, and a sense of fairness. -- Bjarne Stroustrup, The Design and Evolution of C++

    "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

Re: The Great Computer Language Shootout
by herveus (Parson) on Sep 21, 2004 at 16:28 UTC
    Howdy!

    I just looked. Perl ranked about one third of the way down the list, right around Java, Python, etc. when CPU was the only consideration.

    Making lines of code and memory of equal weight, Perl ranked ninth or so...diddle the weights yourself and see what comes out. The books are there to be cooked.

    Don't worry. Be happy.

    yours,
    Michael
Re: The Great Computer Language Shootout
by ikegami (Pope) on Sep 21, 2004 at 16:34 UTC

    Check the programs above Perl. They are either:

    1. compiled to native code,
    2. overly complex and unreadable, or
    3. use a simple regexp language or don't use regexp at all.

    Given that -- without even counting the other benefits of Perl not illustrated -- I'm quite happy with the results.

Re: The Great Computer Language Shootout
by tilly (Archbishop) on Sep 21, 2004 at 16:39 UTC
    Nobody wants Perl to be slow. But concrete suggestions are more useful for making it fast than generalized complaints. Furthermore on some items you'll find that there are difficult trade-offs and speed (or speed on the item that you may care about) is of lower priority than something else that you complain about.

    To take the specific item that you complain about, I happen to know that Perl's regular expression engine could easily be sped up by removing a sanity check for pathological regular expressions. The result would be to speed up a lot of programs by an unnoticable amount, at the cost of making some pathological ones will surprise by taking a few billion years to finish. That change might make Perl look good on a benchmark, but would result in more bug reports. Do you really want that change?

    Furthermore other areas of slowness are due to unavoidable design considerations. For instance Perl is a highly dynamic interpreted language. That is just never going to be as fast as a static compiled language. Which matters more to you, performance or programming convenience? If it is raw performance, then you're probably using the wrong language.

    However I have good news for you. The Parrot project is creating a new version of Perl, and is very concerned with performance considerations. If you want to be of assistance, you could try implementing the shootout test suite in Parrot byte-code, submit that to the project, and identify specific performance issues that you uncover.

      However I have good news for you. The Parrot project is creating a new version of Perl, and is very concerned with performance considerations. If you want to be of assistance, you could try implementing the shootout test suite in Parrot byte-code, submit that to the project, and identify specific performance issues that you uncover.
      While I'm all up for performance shakeouts, one thing to keep in mind is that this version of the shootout needs Debian-installable packages of the languages in question to add them to the test. (So if someone wants to build a debian installer target for the Makefile... :)

        And if you do make a Parrot .deb for the shootout, don't forget to put in the optimizations. Since Parrot is still heavily under development, optimizations are turned off in the builds by default to help with debugging.

        "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

Re: The Great Computer Language Shootout
by FoxtrotUniform (Prior) on Sep 21, 2004 at 19:18 UTC

    I just recalculated the rankings with CPU weight 0.1, memory weight 0.01, and LOC weight 1.

    Perl came out sixth. (OCaml came out first and fourth, Pike second, Haskell third (and, er, last), and Ruby fifth.)

    My theory here is that a programmer with a good grasp of a given language writes lines of code at a fairly constant rate. (It's often argued that bug count is linear in lines of code, so I'm going out on a limb and assuming that speed of hacking is similar. My own experience confirms this.)

    Why is programmer speed important? It's really cheap and easy to upgrade your memory; costs about two hundred bucks for a gig of good RAM, and takes maybe five minutes to install it. It's relatively cheap and easy to upgrade your processor -- maybe seven hundred dollars for a nice Athlon 64 and quality motherboard to go with it. It's really hard to upgrade your programmer. If you need raw development speed, you're better off with a language that you can program quickly in than one that takes maximal advantage of hardware that you can readily upgrade anyway.

    This isn't always the case; for some applications (games, for instance) code speed is critical. However, for most business tasks -- by which I mean the kind of one-off or in-house programs that a lot of us write -- nobody's going to notice a five-second performance difference, but everyone's going to care about a five-week delivery delta.

    --
    F o x t r o t U n i f o r m
    Found a typo in this node? /msg me
    % man 3 strfry

Re: The Great Computer Language Shootout
by CountZero (Chancellor) on Sep 21, 2004 at 19:52 UTC
    As they say, statistical figures will confess to anything under torture.

    Seriously, do they take into account the time to compile the program as well? Or the time to write and debug the code?

    If you include a module (or header file, or whatever, ...) does it count as 1 line or as as many lines as this module, ... contains?

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: The Great Computer Language Shootout
by TedPride (Priest) on Sep 21, 2004 at 20:26 UTC
    I bet if we wrote everything in machine language, it would run really fast - but the programming time would be horrendous. PERL beats other languages not because of its speed - though it is quite fast - but because of its power and ease of use. There are very few things it can't do, and you can always use C or C++ modules for the parts that require more speed. I'd personally either ignore the benchmarks (nobody who uses PERL really cares), or go out and write better code to improve the language. Complaining about the situation isn't going to do much.
      s/PERL/Perl/ plzkthx.
Re: The Great Computer Language Shootout
by jdporter (Canon) on Sep 21, 2004 at 21:14 UTC
Re: The Great Computer Language Shootout
by Your Mother (Canon) on Sep 22, 2004 at 07:05 UTC

    Others could poke better holes but here's my stab, assuming I'm using Benchmark correctly which I'm never sure about. Their random number generator perl versus perl's vanilla rand(). Off by between 200% and 1,500%.

    use Benchmark qw(cmpthese); use strict; my $theirs = q# use constant IM => 139968; use constant IA => 3877; use constant IC => 29573; my $LAST = 42; sub gen_random { ($_[0] * ($LAST = ($LAST * IA + IC) % IM)) / IM } my $N = ($ARGV[0] || 100) - 1; gen_random(100.0) while ($N--); sprintf "%.9f\n", gen_random(100.0); #; my $native = q{rand(100) for 1 .. 100}; cmpthese(1_000_000, { theirs => $theirs, native => $native, }); __END__ Rate theirs native theirs 4414/s -- -68% native 13961/s 216% --

    And if you turn it down to their default of a single run/generation:

    Rate theirs native theirs 121507/s -- -93% native 1851852/s 1424% --

    (Update: I know they're doing "the same algorithm" for each; just seems wrong to feed a cat dog food.)

Re: The Great Computer Language Shootout
by Beechbone (Pilgrim) on Sep 22, 2004 at 11:31 UTC
    (all discussion about benchmarking and comparing of languages silently ignored *g*)

    ...but some of the Perl snippets are really bad. I took the time to speed up the 'hash2' by about 15%:

    use strict; my $n = $ARGV[0] || 1; my %hash1 = (); for (my $i = 0; $i <= 9999; $i++) { $hash1{"foo_$i"} = $i; } my %hash2 = (); for (my $i = 1; $i <= $n; $i++) { foreach (keys %hash1) { $hash2{$_} += $hash1{$_}; } } print "$hash1{foo_1} $hash1{foo_9999} $hash2{foo_1} $hash2{foo_9999}\n +";

    Search, Ask, Know
Re: The Great Computer Language Shootout
by perlfan (Deacon) on Sep 22, 2004 at 14:29 UTC
    I do not understand why this guy insists on all code implementations being the same. It would be a better comparison if he allowed users to contribute optimized code instead of forcing all code into the same square hole. This is just dumb - if he wants to test a language, allow it to utilize its strengths, not some preconceived notion of what the code *should* look like.
Re: The Great Computer Language Shootout
by gmpassos (Priest) on Sep 23, 2004 at 01:59 UTC
    Well, I don't know when you saw the rank, but Perl is in the 8th place:
    # Language Implementation Score 1 C gcc 154.6171 2 Ocaml ocaml 124.4804 3 C++ g++ 105.6944 4 Eiffel se 98.7194 5 Ocaml ocamlb 93.0624 6 SML mlton 91.8259 7 Clean clean 90.5943 8 Perl perl 79.6822 9 Haskell ghc 77.4361 10 Pike pike 77.1981 11 Oberon-2 oberon2 77.0360 12 Lua lua 75.2653 13 Scheme bigloo 75.1351 14 S-Lang slang 73.4023 15 Python python 73.0627 16 Ruby ruby 72.6933 17 Lisp newlisp 69.7620 18 Common_Lisp cmucl 65.5113 19 Forth gforth 65.2669 20 Awk mawk 64.9447 21 Java gcj 64.3719 22 Python psyco 63.3687 23 Nice nice 63.3655 24 Scheme chicken 63.1898 25 Awk gawk 63.1771 26 SML smlnj 62.8468 27 Java java 62.3177 28 Common_Lisp sbcl 61.1720 29 Tcl tcl 60.9442 30 Erlang erlang 60.8645 31 Java kaffe 60.4781 32 Erlang hipe 58.9117 33 Scheme guile 57.9570 34 Java sablevm 57.4272 35 Smalltalk gst 56.6310 36 Java gij 56.3162 37 Lisp rep 51.1753 38 Dylan gwydion 48.7750 39 Icon icon 47.1262 40 Ada gnat 41.8272 41 Scheme mzscheme 39.3508 42 PHP php 39.3355 43 Mercury mercury 38.9003 44 Bash bash 38.3828 45 Scheme mzc 36.9159 46 Emacs_Lisp xemacs 35.3363 47 Mozart/Oz oz 33.0343 48 C# csharp 31.2426 49 Scheme stalin 29.2409 50 ElastiC elastic 14.8057 51 Objective_C objc 12.5165 52 Prolog gprolog 4.0209 53 Ciao_Prolog ciao 2.3282 54 Haskell curry 0.8528
    Table generated at http://shootout.alioth.debian.org/craps.php?xcpu=1&xmem=1&xloc=1 (with 1 score multiplier for CPU, Mem and Lines):

    Formated with

    my @data = qw( C gcc 154.6171 2 Ocaml ocaml 124.4804 2 C++ g++ 105.6944 2 Eiffel se 98.7194 5 Ocaml ocamlb 93.0624 2 SML mlton 91.8259 2 Clean clean 90.5943 5 Perl perl 79.6822 2 Haskell ghc 77.4361 4 Pike pike 77.1981 2 Oberon-2 oberon2 77.0360 5 Lua lua 75.2653 3 Scheme bigloo 75.1351 3 S-Lang slang 73.4023 5 Python python 73.0627 2 Ruby ruby 72.6933 2 Lisp newlisp 69.7620 6 Common_Lisp cmucl 65.5113 2 Forth gforth 65.2669 4 Awk mawk 64.9447 8 Java gcj 64.3719 3 Python psyco 63.3687 3 Nice nice 63.3655 2 Scheme chicken 63.1898 6 Awk gawk 63.1771 8 SML smlnj 62.8468 2 Java java 62.3177 3 Common_Lisp sbcl 61.1720 2 Tcl tcl 60.9442 5 Erlang erlang 60.8645 0 Java kaffe 60.4781 3 Erlang hipe 58.9117 0 Scheme guile 57.9570 2 Java sablevm 57.4272 3 Smalltalk gst 56.6310 4 Java gij 56.3162 3 Lisp rep 51.1753 6 Dylan gwydion 48.7750 5 Icon icon 47.1262 9 Ada gnat 41.8272 9 Scheme mzscheme 39.3508 7 PHP php 39.3355 12 Mercury mercury 38.9003 9 Bash bash 38.3828 14 Scheme mzc 36.9159 7 Emacs_Lisp xemacs 35.3363 11 Mozart/Oz oz 33.0343 8 C# csharp 31.2426 2 Scheme stalin 29.2409 16 ElastiC elastic 14.8057 18 Objective_C objc 12.5165 23 Prolog gprolog 4.0209 23 Ciao_Prolog ciao 2.3282 24 Haskell curry 0.8528 26 ); sub FORMATING { $^A = ""; formline(shift,@_); return $^A; } print " # Language Implementation Score\n" ; my $format = qq`@>> @<<<<<<<<<<<< @<<<<<<<<<<<< @>>>>>>>`; my $c ; for(my $i = 0 ; $i <@data ; $i +=4) { ++$c ; my $form = &FORMATING($format , $c , @data[$i..($i+2)] ) ; print "$form\n" ; }

    Graciliano M. P.
    "Creativity is the expression of the liberty".

Re: The Great Computer Language Shootout
by QM (Vicar) on Sep 23, 2004 at 18:59 UTC
    I didn't see any explicit mention of this idea, but it's obvious in the other replies...

    Why not start your own site which adds the criteria for Development Time, possibly broken down into Coding Time and Debugging Time? This should be weighted something like 10% or 1% of the CPU weighting (insert arguments for and against here).

    And the various points about using native idioms to solve the same problem lead me to think that a Native Idiom score would be telling as well. Sure, by forcing all entries to avoid native idiom advantages, it shows what the performance of a generic, doesn't-fit-into-a-native-idiom algorithm would do. But it also avoids showcasing what programming areas each language excels at (performance wise). Add in the native idiom measure, and development time becomes apparent and interesting too.

    -QM
    --
    Quantum Mechanics: The dreams stuff is made of

Re: The Great Computer Language Shootout
by FoxtrotUniform (Prior) on Oct 10, 2004 at 20:23 UTC

    On the same subject, here's an older article by Peter Norvig that might interest you.

    --
    Yours in pedantry,
    F o x t r o t U n i f o r m

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://392696]
Approved by Old_Gray_Bear
Front-paged by grinder
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (11)
As of 2014-04-17 11:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (446 votes), past polls