Re: Benchmark Arena Proposal

[salvadors]

A good place to practice your optimising is Doug Bagley's Great Code Shootout, where he compares a variety of different scripting languages doing different tasks.

It's quite fun.

Tony

[Albannach]

A very interesting link indeed! While Perl might not look to good in many individual tests, the scorecard page on that site allows assignment of weights to the benchmarks and under the default weights Perl does fairly well (I'll have to go back when I have more active brain cells and refine the weights a bit).

Now that other languages have been raised in this thread, I'd like to add the point that (for me at least) one major benefit of Perl is greatly reduced development time - in the dark ages I did all my text processing in Fortran or C (funny, they don't include Fortran on that site ;-). Now I imagine this would be very hard to do, but has anyone ever attempted to compare the development times? Sure it would be hard to avoid just comparing the coding proficiency of people, and stats on LoC per day I've seen have usually been pretty indefensible. How about errors per day? Errors per line? Define a line or error! One might get around all these silly measures by simply comparing total time to meet an end-product specification, especially since many fields use (or are moving to) an end-product focus. Of course that ignores maintainability...

All the same, I imagine someone (in some hairy-titled ACM journal perhaps) has looked at this question, and since it's an issue that Perl responds to well, it might be worth further investigation, vis-a-vis Perl advocacy.

In any case, while I do find Perl vs. Perl benchmarks most amusing, all a benchmark will tell you is how fast a given platform can run the benchmark!

--
I'd like to be able to assign to an luser

[salvadors]

In any case, while I do find Perl vs. Perl benchmarks most amusing, all a benchmark will tell you is how fast a given platform can run the benchmark!

I disagree. A good benchmark should *compare* 2 or more things, not to see how fast either runs, but to see how they run relative to each other.

In this case Doug compares Perl to lots of other scripting languages - all on the same platform, and where possible, under the same conditions. The results are valid for seeing a lot more than how fast (in raw speed) something is.

Tony

[Albannach]

Oh, a good benchmark? ;-) OK, I'll admit I was being a bit flippant there. I'll agree with you if you add that the test must use real world data and real conditions as much as possible, but I just don't know how often that level of effort is made.

In this case Doug compares Perl to lots of other scripting languages - all on the same platform, and where possible, under the same conditions. The results are valid for seeing a lot more than how fast (in raw speed) something is.

I did read the page and know what he's comparing, but I still don't see what broader insights can be gained from these statistics. I'm not saying they are not interesting, but I sure wouldn't pick a language based on these results alone. Can you elaborate a bit on what you think the results show?

If you compare languages or platforms using the more traditional benchmarks which are often highly idealized and optimized little exercises that have little to do with a real world use, you'll be able to say which ones run the benchmark the fastest, but not which one is better for a particular application (unless you have thoroughly profiled that application and know the appropriate weights to insert in the scorecard page or equivalent).

--
I'd like to be able to assign to an luser

[sierrathedog04]

Thanks for the link. I checked out how Perl did, and was somewhat shocked at what I saw.

Perl ROCKS at text processing. It beat all other languages, including gcc, when it came to regular expression matching, and was near the top in spell checking, word concatenation, etc.

On everything else Perl was, well, in the toilet. Indeed, the only language it consistently beat was tcl. When it came to method invocation, perl was the worst language tested in terms of CPU time.

One point to consider is that these tests probably included the time for perl to first compile the test script and then run it. In some environments (mod_perl?) I believe that Perl will cache a compiled version of whatever it is doing, so performance would improve.

Even so, what I am seeing is that for tasks in which the primary processing load is something other than text processing Perl may not be the most efficient execution language.

That does not mean we should not use it. After all, assembler is the most efficient language that there is, but few recommend developing in it. However, I now believe that a competent programmer should excel in more than just perl. Like Larry. He knows Perl backwards and forwards, but he also knows C, and that is what he chose to write Perl in.

[thaigrrl]

I believe that Larry wrote Perl in C due to Perl's widespread use in the Unix environment, considering Unix was also written in C. He pretty much compiled various Unix language uses, such as awk, c... and created Perl. I would hope that he knows Perl backwards and forwards considering he did write it. Although, I must admit, I prefer coding in Perl because it's just so damn easy :) - thus probably why scripting in Perl is so popular. Of course scripting could be written in C if one were ever so inclined to do so, but as you stated above: "Perl ROCKS at text processing."!!! (exclamation points added by me!) And mod_perl is all that you say it is - faster - because it's a preloaded program. :)

[sierrathedog04]

In a recent interview at LWN Larry Wall provided an explanation for why one might use Perl to script a task even when Perl is not the optimal language for that task. Larry wrote:

People don't want to have to use Perl plus other things. If there is a job that really ought to be written in C++ or Java or Ruby or Python or something like that, but they like Perl, and Perl may not be the best tool yet for it, but they would like it to be.

So rather than learning a different language, they just want to extend Perl toward what is better for that. So I think it it's still laziness."

When Larry speaks of laziness, however, he means it as praise. The programmer is avoiding unnecessary effort, in this case the unnecessary effort of learning a new language.