http://www.perlmonks.org?node_id=952765

kikuchiyo has asked for the wisdom of the Perl Monks concerning the following question:

When the first version of Rakudo Star came out, I was eager to try it, but I was, erm, somewhat underwhelmed by the fact that its performance was not quite up to what some people might have expected:

$ time ./perl6 -e 'my $s; for 1..10000 {$s+=1/$_**2};say $s' 1.64483407184806 real 0m38.387s user 0m38.154s sys 0m0.232s $ time perl -E 'my $s; for (1..10000) {$s+=1/$_**2};say $s' 1.64483407184806 real 0m0.009s user 0m0.008s sys 0m0.004s

One year later, I've tried the same "benchmark" with Rakudo Star 2011.07:

time ./perl6 -e 'my $s; for 1..10000 {$s+=1/$_**2};say $s' 1.64483407184806 real 0m9.588s user 0m9.513s sys 0m0.068s

Well, it's an improvement.

Now, the latest version (2012.01) came out, supposedly with great improvements, so I thought I'd try it.

To my great consternation the same script didn't complete within three minutes. I tried to lower the loop range:

time ./perl6 -e 'my $s; for 1..1000 {$s+=1/$_**2};say $s' NaN real 0m2.164s user 0m2.080s sys 0m0.080s

Note that the result is incorrect (it is a number actually).

I've tried other ranges:

n of iterresulttime
2001.639946546853190.799
400NaN0.937
600NaN1.195
800NaN1.592
1000NaN2.179
1200NaN2.972

A very pronounced O(N2) trend can be observed here. Is this a known problem? Because it is a problem: there is nothing in the algorithm that would imply such a quadratic behavior, and indeed, with Rakudo 2011.07, elapsed time grows linearly with the number of iterations.

As to the NaN,

./perl6 -e 'my $s; for 1..358 {$s+=1/$_**2};say $s' 1.64214466837792 ./perl6 -e 'my $s; for 1..359 {$s+=1/$_**2};say $s' NaN

To paraphrase a certain presentation about weird programming languages that's become popular lately,

WAT

Replies are listed 'Best First'.
Re: Perl 6 and performance
by moritz (Cardinal) on Feb 09, 2012 at 17:26 UTC

    You've indeed hit a regression in Rakudo here. Let me explain it.

    1/$_**2 produces a Rat object in Perl 6 if $_ is an integer (it is here). That is, it's a rational number that stores the numerator and denominator as integers. Thus $s is als a Rat, and the way that $s is built makes the denominator grow very fast.

    Since newest Rakudo has bigint support, the integers can now grow larger than 64bit, and the growing size of the integers causes the O(n²) slowness you observed.

    The specification says that once the denominator exceeds 2 ** 64 128, the arithmetic ops are supposed to fall back to floating point numbers to avoid such cases of pathological performance. Rakudo hasn't implemented that part yet, hence the regression.

    If you use floating point numbers to begin with, you'll get:

    $ time ./perl6 -e 'my $s; for 1..10000 {$s+=1e0/$_**2};say $s' 1.64483407184807 real 0m1.659s user 0m1.360s sys 0m0.200s

    niecza correctly implements the fallback to floating point values, and yields the correct result (in 3s on a different machine of mine; it's usually slow to compile stuff, but faster at run time).

    I've already started a branch to fix the Rat arithmetic issues (branch 'Rational' on github), and I'll try to make it work before the next Star release; I can't promise it though.

    P.S. I'm sad to see that perlmonks is slowly becoming more like reddit, where problems are met with rancorousness instead of being discussed on a technical level.

    UPDATE (2012-02-13): I've fixed Rakudo to fall back to Num if the denominator exceeds 2**64, so now you get:

    $ time ./perl6 -e 'my $s; for 1..1000 {$s+=1/$_**2};say $s' 1.64393456668156 real 0m1.341s user 0m1.240s sys 0m0.096s

    You can get this fix by using the latest development revision from git, or wait for the next release. Thanks again for your feedback.

    Second update: when running up to 10_000 it takes 2.8s on my machine, so while still a big step away from perl 5 speed, it's faster than your previous measurements.

        P.S. I'm sad to see that perlmonks is slowly becoming more like reddit, where problems are met with rancorousness instead of being discussed on a technical level.

      I think the peevishness is a result of hype exhaustion on the topic of Perl6. I'm thrilled that a barrel-full of very bright people are working on the next generation of Perl, but I'll happily wait till it's cooked, and I'll continue to happily use the excellent Perl5 in the meantime. It's worked very well for me for the last 15 years, and it's still very much alive.

      And, as the kids say, "It's all good."

      Alex / talexb / Toronto

      "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

        My point of view is that Perl6 is not so much a successor to Perl5 but rather an alternative to it. To that end it might have been preferable not to call it Perl6 but maybe something like PerlVM that would indicate that it takes a radically different approach to "classic" Perl.

        If/when Perl6 does reach release-level maturity it probably won't supplant Perl5. The two can quite happily co-exist as different implementations which share (most of) a common syntax. At that point we can all choose the right tool for the job at hand.

        Just my tuppence.

        Update: It's now October 2019 and "Perl6" is now "Raku" so that, belatedly but thankfully, solves the naming problem. The syntax is probably more divergent from Perl than I anticipated but the two languages (and communities) can now co-exist and provide two different tools for every programmer.

      Thank you for the detailed answer. It indeed makes the cause of the problem clear.

      For me, it's more natural to write a floating point constant one as 1.0, however, that doesn't work. And this won't even run:

      time ./perl6 -e 'my $s; for 1..10000 {$s+=1./$_**2};say $s' ===SORRY!=== Unable to parse blockoid, couldn't find final '}' at line 2

      I've also tried to explicitly declare the loop variable as a floating point:

      ./perl6 -e 'my $s;my Num $i;for 1..10000 -> $i {$s+=1/$i**2};say $s'

      This has no effect, as can be seen from the following:

      ./perl6 -e 'my Num $s;my Num $i;for 1..10000 -> $i {$s+=1/$i**2};say $ +s' Type check failed in assignment to '$s'; expected 'Num' but got 'Rat' in method reify at src/gen/CORE.setting:4471 in method reify at src/gen/CORE.setting:4376 in method reify at src/gen/CORE.setting:4376 in method gimme at src/gen/CORE.setting:4740 in method eager at src/gen/CORE.setting:4715 in method eager at src/gen/CORE.setting:1028 in sub eager at src/gen/CORE.setting:5000

      Does this mean that even though I declare $i as a Num, it's still an integer when used as a loop variable? Is it because the .. operator supplies integers?

      P.S. I'm sad to see that perlmonks is slowly becoming more like reddit, where problems are met with rancorousness instead of being discussed on a technical level.

      I admit that I posed the original question in a slightly more trollish way than necessary, and I apologize for the inconvenience I might have created. I didn't intend to denigrate the effort of the Perl 6 developers - an effort which is nothing short of heroic.

      However, I do think that the problems with the current state of Perl 6 go way beyond the technical level, and pretending that these problems can be treated at the mere technical level is an attitude that does more harm than good on the long term.

        To clarify things, 1. is not a valid number in Perl 6. 1.0 is a Rat, 1e0 or 1.0e0 are Num.

        I've also tried to explicitly declare the loop variable as a floating point:

        There are two issues with it. The first is that you don't declare the loop variable to be a Num, but an outer variable of the same name. In -> $i { ... } the part between the arrow and the block is a signature, which declares a new variable. So it's like writing

        my Num $i; sub ($i) { # new $i here }

        A way to fix that is writing -> Num $i { }, but then you get a type check failure because 1..1000 indeed produces Ints, not Nums. So either you coerce to Num somewhere (for example $i.Num ** 2), or you write the range as 1e0 .. 1e3.

        I admit that I posed the original question in a slightly more trollish way than necessary, and I apologize for the inconvenience I might have created. I didn't intend to denigrate the effort of the Perl 6 developers - an effort which is nothing short of heroic.

        My side remark was more about _foo's trolling, not about your original question, which I think is 100% valid, and a good bug report at that. Sorry if that came out wrong on my part. User feedback like yours is what we need, and one of the reasons for making the star releases in the first place.

Re: Perl 6 and performance
by Old_Gray_Bear (Bishop) on Feb 09, 2012 at 16:53 UTC
    The Three Stages of Program Development:
    • First, make it run
    • Second, make it run right
    • Finally, make it run fast
    Perl6 is still in the second stage.

    ----
    I Go Back to Sleep, Now.

    OGB

      The stages of Perl 6 development

      a. First, make a small part of it run.

      b. Then make that small part run on a hundred VM's

      c. Then re factor all those 100's of parts every 2 years.

Re: Perl 6 and performance
by raiph (Deacon) on May 27, 2015 at 04:46 UTC
    »»» This post is about alpha status Perl 6, not rock solid Perl 5 «««

    I just encountered this post (thanks hippo) and thought an update would be helpful.

    The OP suggests that running this "benchmark" with the initial Rakudo Star release from 2010 was about 4000x slower than Perl 5.

    Now, 5 years later, a current Rakudo looks like it's something like 50x slower than a current Perl 5 for this particular "benchmark". (The bogus behaviors -- NaNs and quadratic slow down -- have of course been fixed.)

    raiph@hack:~$ time perl6 -e 'my $s; for 1..10000 {$s+=1/$_**2};say $s' 1.64483407184807 real 0m0.355s user 0m0.316s sys 0m0.036s raiph@hack:~$ time perl -E 'my $s; for (1..10000) {$s+=1/$_**2};say $s +' 1.64483407184807 real 0m0.007s user 0m0.004s sys 0m0.000s

      And here's an update comparing "Christmas Perl6", aka v6.c, aka rakudo star 2016.01.1 to Strawberry Perl 5.16.3, using a bat script on Windows 7:
      echo %time% - perl5 call c:\strawberry_perl\portableshell.bat -E "my $s; for ( 1..1000000 +) { $s+=1/$_**2 }; say $s" echo !time! echo %time% - perl6 call perl6 -e "my $s; for ( 1..1000000 ) { $s+=1/$_**2 }; say $s" echo %time%
      Note I'm using a million iterations to make the times more accurate to compare. The output was:
      14:03:07.14 - perl5 1.64493306684877 14:03:07.64 14:03:07.65 - perl6 1.64493306684877 14:03:19.05
      So, perl5 took .5 sec and perl6 took 11.4 secs, making perl6 almost 23 times slower than perl5. I really hope this is improved on in subsequent releases of perl6. John
        So, perl5 took .5 sec and perl6 took 11.4 secs, making perl6 almost 23 times slower than perl5. I really hope this is improved on in subsequent releases of perl6. John

        My considered assessment is that in order to be able to support the needs of MOP, and installable parsers, the levels of indirection required within the opcode tree are such that Perl6 will never be able to achieve the performance of Perl5; much less (uncompiled) Java and similar.

        From my brief appraisals of the performance it currently achieves, and an even more brief exploration of the source code, I think it quite unlikely that the Perl6 syntax will ever be able to get within an order of magnitude of Perl5 performance as a purely interpreted language. I think it would require a full optimising compiler to achieve any real breakthrough in its performance.

        It is possible that a JIT compiler could fold out some of the indirections, but I suspect that it would require serious machine level skills on each targeted platform for that to be achieved.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.
        A few things to note:

        In Perl 6, that math is going to default to Rationals, meaning you're keeping around a numerator and denominator and doing a bit more math than you might otherwise be expecting than p5's floats.

        The point of the Christmas release was to stabilize the spec. Work in the upcoming year on Rakudo Perl 6 is going to target optimizations in both JIT and code generation.

        While I don’t find this kind of benchmark too exciting I will add that the difference is greater on OS X; well over a hundred times faster for 5. It’s easy, for me anyway, to forget how well tuned 5 is…

        # This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin- +2level moo@cow[22]~>time perl -E 'my $s; for ( 1..1_000_000 ) { $s+=1/$_**2 } +; say $s' 1.64493306684877 0.111u 0.000s 0:00.11 100.0% 0+0k 0+2io 0pf+0w # Both the last two Rakudo builds of 6.c produced same results moo@cow[30]~>time perl6 -e 'my $s; for ( 1..1_000_000 ) { $s+=1/$_**2 +}; say $s' 1.64493306684877 16.198u 0.300s 0:16.55 99.6% 0+0k 0+0io 0pf+0w
Re: Perl 6 and performance
by Anonymous Monk on Feb 11, 2012 at 03:46 UTC
    Every "programming language that has been around for a while" has to cope with irrelevance ... and Perl-6 is at that point now. The language reached its true zenith with Perl-5, and a group of well-meaning people tried to produce something better while at the same trying to bootstrap on top of what territory Perl-5 had already mastered. But, what they have managed to come up with is ... ADD 1 TO PERL GIVING PERL. It truly has nothing to do with "Perl," in the same way that "object-oriented COBOL" has nothing to do with COBOL. And, during the many years that Perl-6 has foundered in the birthing stage, the river has moved on. Programming languages, even good ones, aren't that hard to make anymore. They don't attract the following that they once did. They do not have the "single target" that they used to. Even if Perl-6 does make it to a deliverable, it will not attract the interest that its erstwhile predecessor has earned.
A reply falls below the community's threshold of quality. You may see it by logging in.