(And you can get most of the gains on the 'bad' XS code just by using 5.14.0 or later, and using a non-threaded (or non-multiplicity) build.

If the gains are there to be had -- and they are -- with a sufficiently wide field of view, then they can be applied at LTO time. With argument promotion and function re-writing and SSA lifting.

Or, for local subtrees of code, they can be applied at runtime using JIT with a sufficiently deep view.

In other words, non-multiplicity efficiencies selectively applied ot subtrees of code on multiplicity builds running single threaded processing. Or even, subtrees of threaded coded that doesn't access shared data.

Note also that LLVM is very unlikely to be able to able to optimise away any of the get_context() calls in the XS code.

Why? If they can be #defined away, why can they not be optimised away?

I anticipate your answer is: because the compiler cannot see widely enough to know to do it. And that is very definitely true for C compilers doing compilation unit optimisations.

But for a non-C optimiser running whole program analysis and/or SSA-analysis, there is no reason it couldn't lift the get_context up to whatever level it last changed, and make it a Single Static Assignment. And perhaps place it in a register.

If you believe differently, the onus ...

• It would be the work of minutes for you; and (probably) days for me.

  Besides which, I do not feel either of us should feel any "onus", because ... see next bullet point.

  But as you suggested that seeing it might clarify something ...

• All it would prove, is that on one compilation, with one configuration, on one platform something did or did not get optimised.

  The beauty of the LLVM approach to optimisation is that it (can) produce optimisations in a platform and compiler independent manner -- at its IF level -- that then get translated into whatever is the best machine-dependent implementation at compile-time or link-time or runtime.

Well, my speculation, based on 11 years experience of working on the perl core, is that improvements with LLVM will come into the "10% better" category rather than the "5x better" or "perl runs at C speed" categories. Which is where this all started. Don't get me wrong, I'm not opposed to things like LLVM; I just haven't been convinced yet that they can offer game-changing improvements.

I only joined this thread belatedly, to counter the negative energy of "it couldn't work", "it wouldn't work", "it cannot be done", and "it would be a waste of energy to try".

And please ignore any possible negative implications of this, but I did so because I saw -- and see -- people thinking of LLVM as simply another C compiler. And citing "failed" projects where it has been used in exactly that way. All I'm trying to do is to get people to stop thinking in C; read a little of the tutorial information, and consider the possibilities.

Your knowledge and experience is -- and would be -- invaluable to such a project, even if only in an advisory capacity if that's all you have the time and interest for. And all I'm trying to do here is persuade you (and others) to consider the possibilities. If I can speculate with my dearth of knowledge of the internals, what might someone like you see, if you were of a mind to understand that LLVm is nothing like any other "compiler"?

>Another mind's eye speculation for you to consider:

Much of the costly code in PP_* routines is there to deal with the 5 or 10% cases. Imagine if -- manually for now -- each SV was born marked (or better unmarked) as carrying magic or not. Say a single bit in the (unused) least-significant 3 or 4 bits of the SVs address, was used to flag when an SV had magic attached to it.

And then imagine a whole duplicate set of pp_* routines that never even considered the possibility of magic, and only called other functions that similarly never considered magic.

And then in the runloop, a one-time inspection was made and the appropriate chain of pp_no_mg_* or normal pp_* routines was dispatched to.

Do you see any gains to be had from running the 90% or 95% (?) of code that doesn't use magic, through a set of opcodes that not only don't test for magic, but also do not carry all the extra variables and conditional branch code that can confuse and overpower the C optimiser and/or the CPU branch predictor?

Please read http://llvm.org/pubs/2004-09-22-LCPCLLVMTutorial.pdf, and see that LLVM was able to do stuff not unlike the above 8 years ago. And things have moved on. A lot!

(There is a fairly detailed walk through of a moderately complex example starting at slide 28, but you would probably need to read the preceding to understand the nomenclature.)

Again, no offense is meant by this, but maybe you have been so close to the Perl sources for so long, that you view them the way a C compiler does. And so cannot imagine the possibilities of looking at them not as C source, or even C compiler generated machine code, but rather with Explicit SSA dataflow; and an explicit control-flow graph; and explicit language-independent type information; and (most importantly in the case of Perl), explicitly typed pointer arithmetic; in conjunction with an infinite SSA register set and loads/stores with typed pointers.

I can see that I do not have the gravitas to convince you. So, please, put aside my speculations and read the .pdf, and allow it to engender your own speculations about the possibilities. Because your speculations would be so much more insightful and therefore useful than mine.

And, if you read it, it will engender your speculations.