Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re^3: Perl 5 Optimizing Compiler, Part 9: & The Low-Magic Perl Commandments

by dave_the_m (Prior)
on Nov 09, 2013 at 12:56 UTC ( #1061816=note: print w/replies, xml ) Need Help??

in reply to Re^2: Perl 5 Optimizing Compiler, Part 9: & The Low-Magic Perl Commandments
in thread Perl 5 Optimizing Compiler, Part 9: & The Low-Magic Perl Commandments

Several of the things I listed are not in your commandments. For example you say "don't tie", but you don't say "don't use variables supplied by other people, which may or may not be tied", such as %Config. You say "don't use lvalue subs", but you don't warn against "substr($x,1,2) = $y".

Maybe you want to actually look at the things I wrote and perhaps even download some RPerl code?

I had a quick look, but frankly my strength was sapped by your previous incomprehensible Part 8 post.

The thing I don't get (perhaps it's explained clearly somewhere, in which case I missed it), but what is the *point* of RPerl? Is it for making trivial blocks of perl code to run faster (but as soon as you want to do anything at all useful, you have to turn it off)?

It seems to be named in honour of RPython, but that has a very specific, limited use case, and this seems unrelated.

Also, I think the contribution of magic to making perl run slower is greatly overstated. The overhead that magic adds is a one-bit test with a conditional function call per variable access. There's still a big other bunch of overheads. For example polymorphic types. When you do something like

$a + $b

perl has to retrieve the the SVs from the current pad, check their 'get magic' flags and if set call mg_get() on them, then check whether overloading is enabled and if so call the overloaded add method, or failing that, try and convert their values into integers or floats, possibly doing a string to integer conversion, or stringifying a ref into "HASH(0x1234)" using the referent's address as an integer if it's a ref, or warning if the value is undef. It then does a whole bunch of stuff related to avoiding overflow or loss of precision where possible, such as upgrading from int to float or vice versa.

Of all that, skipping the magic part is just skipping a 1-bit flag test - everything else still needs doing.

(In fact, the actual check in pp_add() or's the flags of the two SVs at the top of stack together, and calls a function if the combined bits indicate magic or a ref: the latter indicating possible overload).


  • Comment on Re^3: Perl 5 Optimizing Compiler, Part 9: & The Low-Magic Perl Commandments
  • Download Code

Replies are listed 'Best First'.
Re^4: Perl 5 Optimizing Compiler, Part 9: & The Low-Magic Perl Commandments
by Will_the_Chill (Pilgrim) on Nov 09, 2013 at 13:30 UTC

    You win another brownie point! I did not strongly enough ban non-variable non-slice lvalues, the commandments have been updated yet again thanks to your sharp eye!

    LMPC #20. Thou Shalt Use Scalars, Arrays, Hashes, Filehandles, & Mundane Lvalues (Variables & Slices)

    LMPC #21. Thou Shalt Not Use Typeglobs, Code References, Weak References, Or Magic Lvalues (Builtins, Non-Variables, Non-Slices, etc)

    I must disagree with your comment about not-using-other-people's-maybe-tied-vars, commandment #3 says not to use non-PBP code, which includes code-with-tied-vars written by either you _or_ somebody else.

    LMPC #3. Thou Shalt Not Use ... Non-PBP Code ...

    My previous post (Perl 5 Optimizing Compiler, Part 8: The Book Of RPerl) was made over 6 weeks ago, surely that's been enough time for your strength to return? ;-)

    I think your questions about RPerl are generally answered in The RPerl FAQ, clearly linked at the very top of the Part 9 original post. Still, to answer your question specifically, the point of RPerl is to create an optimizing Perl 5 compiler. To achieve this, RPerl will initially remove all magic from Perl 5 and create a "restricted" (like RPython) subset of the Perl 5 language which can be compiled directly to C/C++ code, which is itself 100% compatible with, and equivalent to, the original Perl 5 code. This is implemented in RPerl by generating specially-crafted C/C++ code that can be fed through Inline::C(PP) and tie back into Perl 5 via XS. Once we have RPerl working with low-magic Perl 5 code, we can start selectively adding back in the magic components 1-by-1, retaining the ability to turn off all magic at any time to keep the pure speed boost of low-magic compiled code. As I've stated already, you can mix compiled low-magic code with normal non-compiled high-magic code.

    To quote myself from my most immediately previous response to you, "Note, my use of the term 'magic' refers to both the specific so-named magic bits attached to Perl data structures, as well as Perl's wacky/weird/complex operations in general." You're right in pointing out that the magic bits attached to the Perl data structures are only 1 part of the "magic" that needs to be turned off to achieve massive performance benefits. This is why there are more commandments under the "Operations" section than the "Data" section.

    ~ Will
      (As it happens I did read the FAQ)

      Since "magic" has a specific meaning within perl, can I suggest you use some other term?

      To expand upon the RPython issue: RPython is not intended for writing general code; its intended specifically only for writing interpreters; within that limited use case, it makes sense to have a language similar to python, but highly restricted (in particular, static typing).

      In the RPerl case, you seem to be targetting it towards general code, where you mark limited sections of your code as targets for optimisation. The problem seems to me that your restrictions are so severe (no regexes!!!!) that virtually no real-life code is amenable to any speed-ups.



        I appreciate your suggestion of not expanding the already-specific meaning of the term "magic" in relation to Perl. Do you have any suggestions? Maybe something magic-like, such as "mystical" or "charmed" or "miracle"?

        I am very interested in expanding upon the RPython issue. :) Like RPython, RPerl can definitely be used to write interpreters. In fact, I see that as a possible huge win in the future. Like RPython, RPerl (initially, at least) also requires static typing.

        To a degree you are correct, I am targeting RPerl toward general-use code in the long-term. In the short-term, RPerl will most immediately be useful for hot code, such as a computational kernel which is already separated from other code and can safely be re-written into "low-magic" (barring better terminology) form.

        I think the deeper issue is the Perl community's (perhaps unhealthy) love affair with high-magic code, and even medium-magic code. This is echoed in your statement that "Perl without its internal 'magic' implementation is essentially no longer Perl." However, have no fear, lover of magic! As I've stated repeatedly:

        1. We can mix low-magic code with high-magic code.


        2. We can add back in all the high-magic components after we've got RPerl v1.0 working with the low-magic components.

        Specifically, regular expressions will likely be the VERY FIRST THING added back into RPerl after v1.0 is done. I know how much everybody loves their regexes.

        ~ Will

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1061816]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (5)
As of 2018-03-20 22:03 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (259 votes). Check out past polls.