Re^7: Perl 5 Optimizing Compiler, Part 4: LLVM Backend?

[BrowserUk]

With that said, compiling all of Perl 5 with a compiler that knows how to do link time optimization does offer a benefit, even if you can use LTO only on the core itself. This won't be an order of magnitude improvement. If you get 5% performance improvement, be happy.

If all you do is use LLVM's C front-end to compile the complete existing code base to IF, optimise, and then back to C to compile, I think you are probably in the ball park, if a little pessimistic. I'd have said low double digit percentage improvements.

But, if you break out the runtime components from the compile-time components -- ie. everything before the (say*) ${^GLOBALPHASE} = INIT -- and compile the before to C and link it. But then convert the (for want of a better term) "bytecode" to IF and pass it to the LLVM JIT engine, what then?

And what if the IF generated could be saved and the reloaded thus avoiding the perl compilation phase for second and subsequent runs (until edits)?

And how about combining the IF generated from the bytecode with the IF form of the core and linking it to build standalone executables?

Is any of this possible? There is only one way to find out.

(*)I appreciate that you would probably need to intercept, repeatedly, at the ${^GLOBALPHASE} = 'CHECL' or 'UNITCHECK' stages in reality.

[chromatic]

My instinct (after not exploring this in much detail) is that you will get some improvements, if you don't blow past the memory limits of LLVM. I believe the 16 MB limit is gone, which will help, but you're still talking about deserializing plenty of bytecode for LLVM to process.

However Reini and I disagree about some things, I think we both agree that improving the typefulness of Perl code to narrow down the dynamic possibilities offer more potential improvements for memory use and optimization. That is probably also compatible with LLVM, but I still think to get real advantages out of LLVM, you have to port the VM to generate IR internally rather than compile the C components to IR with clang.

[BrowserUk]

It is when you get into the details, that casual discussion starts to get difficult. There are so many phases, it gets very hard to make it clear what we are talking about at any given point. (It is hard to keep them distinct in your (my!) own mind.)

I *think* we are talking about (roughly) the same things here.

1. There is the bit of perl that parses the source code and builds the AST from the Perl programs source.

   This would need to be left pretty much as is. The whole perl defines Perl thing. This would need to be compiled (back to) C and then linked into an executable.

2. Then there is the bit that converts the AST into byte code.

   I believe that this would need to be separated out and converted to produce IF.

3. Then there is the bit of perl that ordinarily runs the bytecode.

   Not just the runloop, but all the code the runloop dispatches to.

   I think that should be compiled to IF and built into an IF library (.bc)

   It would be ""combined"" with the IF from stage 2, at runtime, and given to the JIT to convert to machine code.

All total speculation of course. And possibly completely impractical or impossible. But we won't know ...