Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Re: Perl 5 Optimizing Compiler, Part 4: LLVM Backend?

by dave_the_m (Prior)
on Aug 28, 2012 at 18:58 UTC ( #990307=note: print w/replies, xml ) Need Help??

in reply to Perl 5 Optimizing Compiler, Part 4: LLVM Backend?

Having closely followed these threads, plus based on some private correspondence, my gut reaction is that there is no system discussed so far which meets all of the following criteria:

  • handles all (or practically all) existing Perl syntax and semantics;
  • doesn't require modification of existing Perl source (e.g. requiring you to add type declarations);
  • provides an order-of-magnitude speed-up for general Perl code; (e.g. 5X rather than 10%);
  • doesn't require a huge coding effort (of a similar magnitude to implementing parrot and perl6)

I'm not saying at this point that all of the proposals definitely fail these criteria; rather that no-one has yet explained any proposal to me in such a way that I have a real "Ooh I see how that might work!" epiphany; and my suspicion is that no-one ever will.


  • Comment on Re: Perl 5 Optimizing Compiler, Part 4: LLVM Backend?

Replies are listed 'Best First'.
Re^2: Perl 5 Optimizing Compiler, Part 4: LLVM Backend?
by chromatic (Archbishop) on Aug 28, 2012 at 19:00 UTC

    Full agreement. None of these are possible in a year without a small army of the right coders, and even then a year's a short time to produce something as reliable as Perl 5.

      dave_the_m & chromatic,

      I think BrowserUK's 3-phase idea (pasted below) and Yuval's comments are both in the same vein. I think this general Perl5-to-LLVM concept is valid, and although it might very well take more than a year to fully implement, I believe we could have some kind of initial demo ready by YAPC::NA 2013 in Austin.

      dave_the_m, discounting only the amount of work needed to fully implement Perl5-to-LLVM, would you please tell me which part of BrowserUK's 3-phase idea is not producing the "I see how that might work!" epiphany in your mind?

      ~ Will

      BrowserUK wrote:

      1. There is the bit of perl that parses the source code and builds the AST from the Perl programs source. This would need to be left pretty much as is. The whole perl defines Perl thing. This would need to be compiled (back to) C and then linked into an executable.

      2. Then there is the bit that converts the AST into byte code. I believe that this would need to be separated out and converted to produce IF.

      3. Then there is the bit of perl that ordinarily runs the bytecode. Not just the runloop, but all the code the runloop dispatches to. I think that should be compiled to IF and built into an IF library (.bc) It would be "combined" with the IF from stage 2, at runtime, and given to the JIT to convert to machine code.

        Perl 5 doesn't run slower than C because it lacks a JIT. Perl 5 runs slower than C because it's designed to be more flexible than C and that flexibility is not free. That flexibility includes but is not limited to:

        • reference counting
        • automatic destruction at scope leave
        • flexible typing and coercion
        • eval
        • dynamic dispatch
        • method dispatch
        • overloading
        • autoloading
        • importing
        • pragmas
        • monkeypatching
        • metaprogramming
        • closures
        • symbolic references
        • exceptions
        • dynamic data structures

        Add to that the facts that LLVM was not designed for dynamic languages and that Perl 5 was not designed for static optimizations. Using LLVM from Perl 5 without changing the internals of the Perl 5 VM probably will speed up code that's already written in C but certainly without addressing most of the reasons why Perl 5 code can run slower than C.

        discounting only the amount of work needed to fully implement Perl5-to-LLVM, would you please tell me which part of BrowserUK's 3-phase idea is not producing the "I see how that might work!" epiphany in your mind?
        Well, since the current perl interpreter has neither an AST, nor bytecode, I don't really understand the proposal.

        But my basic issue in this case is how do you get the 5X speedup rather than the 10%? I understand in great detail exactly what C-level code the perl interpreter executes while running an ops loop (and the ops within it). No-one has yet explained to me in a way I can understand how all the necessary stuff to carry out those actions will somehow go faster.

        For example, to do a subroutine call in perl, you need to do a whole bunch of stuff like

        • set up @_ and copy the stack items to it;
        • set up a context stack entry so that return etc know what to do;
        • create a new scope stack level;
        • bump the reference count of the live CV so that it can't be freed in mid-execution
        • make the appropriate pad the current one;
        • etc.
        All (or at least the vast majority) of this has still to be done, unless the perl interpreter is radically redesigned somehow. At the moment all this is done by a single, tuned C function. I don't see LLVM making it significantly faster.


        You know, you can cut out the italics and use <blockquote></blockquote> much easier to read

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://990307]
[Corion]: choroba: Currently two or three that my program handles (WWW::Mechanize:: Chrome), but there might be more that become interesting
[Corion]: But I don't expect more than 100 to be active at the same time, so I'm not really sure if there is a not-too-fancy data structure that is maintained with few lines of code where the performance is better than the linear scan ;)
[Corion]: But I should do a mock-up program so that others can see what I'm talking about ;)
[robby_dobby]: Corion: I hope you know all too well that passing around "fancy" datastructures is a recipe for disaster :-)
[robby_dobby]: As in, it's-too-fancy- that-it-will-be- messy-to-handle
[choroba]: bit vectors as keys?
[robby_dobby]: Hmm, I keep falling asleep at my desk, while maintaining an active appearance. Am I getting old?
[robby_dobby]: Every time I fall asleep, there's a small guy in the dreams, shouting "Whoo!" and it jolts me awake. :/
[Lady_Aleena]: robby_dobby, at least you aren't driving. I seem to always be driving somewhere in my dreams and end up at a weird house.

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (10)
As of 2017-05-29 08:01 GMT
Find Nodes?
    Voting Booth?