Re^3: Perl 5 Optimizing Compiler, Part 5: A Vague Outline Emerges

[chromatic]

I understand that strategy, and it's probably easier to start to see results than most other approaches, but keep in mind a few drawbacks:

• You have to compile every part of the system to LLVM bitcode, including the Perl core, any modules you use, and every XS component. This is cacheable, but you still have to do it.
• Before running a program, you have to link all of the bitcode together into a single image. This image will be large.
• Before running a program, you really want to run as many of LLVM's optimization stages as possible. This takes time and memory.
• You may be able to emit native code which represents only that program and execute that. Essentially you're replicating what PAR does, with all of the caveats about program analysis and requirement discoverability.

I expect the resulting binaries to be huge. I expect the link and optimization steps to take a long time. I expect you'll have to keep the Perl 5 interpreter around anyway because there are things you just can't do without either rewriting the language (to get around the BEGIN/import symbol declaration dance, for example. I don't know if you have to include any LLVM runtime components.

I can imagine that you can optimize some programs with this approach, but I don't know that the intrinsic overhead in the Perl 5 VM you get is more than 10%. Maybe link-time optimization can cut out another 10%. Part of that is the inherent flexibility of Perl 5's design, and part of that is that LLVM is at heart a really good compiler for languages that act like C++.

[BrowserUk]

I'm not sure there is a direct question in there for me to respond to, so all I'll say is that it is a minimal up-front effort strategy that puts the hooks in place that would allow some real measurements to be taken.

And even that minimal effort needn't be discarded if the LLVM experiments fail, because it leaves an intact perl distribution that works just as efficiently and effectively as it does now, with the only difference being that the compiled perl executable comes in 3 parts rather than the current two. (In windows terms: perl.exe dynalinked to perl5.x.part1&2.dll & perl5.x.part3.dll)

And that split might even benefit the normal distribution in some way.

Once the hooks were in there, the ability to pass the PCG to some other dll, module or process, might allow other investigations to move forward.

/code

[chromatic]

If this approach could lead to a cleaner separation of the parser from the runtime, that might be a worthwhile thing in and of itself, apart from any LLVM success or failure. It's worth considering further.