What's the perl5's future?

[xiaoyafeng]

Actually I'm shocked when I've read this blog written in 4 months ago.

I'm shocked not only because many codes I've written is used with AnyEvent but also because I'm afraid it's a sign which perl5 world is beginning mess.

We love perl because it's expressive, because it's cpan and more importantly, because it's consistent. you can install a same perl in many OS and retrieve the same. But let's think, because perl5 has not a offical spec, what if there are many incompatible perl in the world like cperl stableperl and Rperl? Do you really want to stick with perl?

I'm not a member of p5, maybe this question need that guy to answer: does the present policy on perl5 development make perl5 good or bad?

Or maybe it's time to emigrate to perl6? Any ideas? I hope monks could point me to a brighter future.

[dave_the_m]

I have been a perl5 maintainer for about 14 years, so I can speak of this with some knowledge.

The blog entry you quote is by Marc Lehmann, a deeply unpleasant individual (at least judging by his on-line persona). He writes XS modules that make heavy use of perl core internals, often undocumented internals. When we make a change to the internals (e.g. to fix a bug or improve performance) it will sometimes break his code. Rather than working with us to resolve the issue (sometimes a module might need fixing, sometimes we might need to back out the change), he just lashes out, accusing us of having no respect for backwards compatibility, usually in highly inflammatory terms.

One irony is that these breakages are usually spotted first by us and reported to Marc. We run a system whereby the current development branch of perl is continually run against thousands of CPAN modules, and if any break, it is reported to the p5p list, along with a bisect of which commit broke it. We then look carefully, decide whether perl needs fixing or the module, and proceed accordingly. This is because we care about backwards compatibility so much.

As far as I'm aware, there are people currently working to make Marc's modules work under 5.22. Whether Marc will accept that work, I don't know.

AFAIKT, stableperl isn't intended to be an ongoing fork of perl; its simply a snapshot of 5.22 created by Marc with whatever commits broke his code removed.

cperl is a recent fork of perl by Reini Urban who is incapable of working with others, so is writing a perl where he can do whatever he likes, unfettered by the difficulties of having to reach consensus.

Rperl is a module run within perl that allows you to write chunks of code in a toy subset of perl that go much faster; so it isn't a fork or rival to perl.

Dave.

[xiaoyafeng]

Thanks, I'm often out of p5p circle, but I think the root reason to cause this is because age of perl5. What's the next of perl5? just bug-fix, optimization? do the p5p guys consider add new features to perl5?

[Discipulus]

Oh they added a lot of things! you can profit the read of a related thread: Should I come back to Perl? where you can find very good perpectives on the matter.

L*

[rurban]

> cperl is a recent fork of perl by Reini Urban who is incapable of working with others, so is writing a perl where he can do whatever he likes, unfettered by the difficulties of having to reach consensus.

To the contrary, Reini Urban is not incapable to work with others. He showed that effectively where ever he worked with. It's rather the other way round, and this is what I was critizising. p5p is traditionally incapable to work with others for over 20 years, and everybody who has a bit of an idea how to improve perl5 already left the perl community. There's no one left, and nobody new is coming up. A shark tank of very important wannabes.

That's why perl5 has no future at all with p5p, the porters, acting as maintainers of the language. p5p cannot take technical criticism, cannot lead decision making, and is technically not able to make any proper decisions. That's obvious to everybody outside the p5p echo chamber. Other languages have proper feature management and eventually are successful.

p5p was not able to come up with any language feature. Many of them were spec'ed and outlined 15 years ago with perl6. Many of the new interesting bits, like given/when, which were not implemented by Larry, were implemented by outsiders, not by p5p. And everything p5p did was just horribly wrong and broken, until they had to disable those "experiments". E.g. just recently they disabled lexical topics, instead of simply fixing it. given/when and smartmatch is still broken, because of the sheer nonsense p5p did. They implemented signatures in the most horrible way, even if there exist many good implementations outside core. cperl signatures have 2x more features and are 2x as fast. perl5 signatures are the most prominent exposure of p5p incapabilities, but the lack to be able to maintain the language properly is even more worry some.

In one year cperl already implemented 20 new features from perl6 and has much better security and performance records. cperl has a goal, perl5 has none. And cperl doesn't have to deal with the problematic people inside p5p. (dave_m is not problematic btw, he just doesn't see that much light)

Only with cperl features and bugfixes are discussed publicly and smoked before being merged to master. In perl5 many changes were made just because one committer thought he liked it, and thereby broke everything around him. And of course those mistakes were never reverted.

cperl is effective continuation of the perl5 implementation efforts from the day when Larry left p5p, perl5 is the stagnation thereof. bug reports are ignored, feature and implementation discussions are hampered by p5p's incapabilities. Decisions are based on friendship and personal connections, but not based on any technical merits. The top 5 implementors are effectively dead wood who produce only nonsense and do more harm than good. This is going on for 20 years, and the code base is an unmaintainable battlefield. And in total violation of their new CoC guidelines p5p is constantly violating those, calling their users and critics assholes without any visible outcry. And the one who is critizising those abuses is called the abuser.

At least they can analyze bugs and sweet talk their achievements.

[marioroy]

Update: For the example script, there is no noticeable difference when comparing Perl v5.20+ against cperl v5.22+. I had compared Perl installed on the OS; Perl 5.18.2 (Mac OS X 10.11.6) and Perl 5.16.3 (CentOS 7.2) against cperl 5.22c. Thank you dave_the_m for catching this.

Thank you for cperl, rurban. I entered a demonstration running 25% to 33% faster with cperl depending on the platform.

Re^3: use threads for dir tree walking really hurts

Furthermore, the optional modules for MCE and MCE::Shared work without a glitch. Unless PDL is present, MCE and MCE::Shared will use Sereal 3.008 or later for serialization if available on the system. Storable is used otherwise. The IO::FDPass module is beneficial for passing file descriptors to the shared manager process. This applies to MCE::Shared::Condvar, MCE::Shared::Handle, and MCE::Shared::Queue.

1. Canary-Stability-2011 (required by IO::FDPass)
2. IO-FDPass-1.1 (optional, used automatically by MCE::Shared)
3. Sereal-Decoder-3.015 (optional, used by MCE and MCE::Shared)
4. Sereal-Encoder-3.015 (optional, ditto)
5. Sereal-3.015 (optional, ditto)
6. MCE-1.805
7. MCE-Shared-1.804 (add-on module for MCE)

The modules listed above work flawlessly with perl11/cperl Github branches maint-5.22c and maint-5.24c. Threads is not required to run MCE or MCE::Shared. I built cperl with threads support and that works too.

./Configure -Dprefix=/opt/cperl-5.22c -sder -Dusethreads -Dusecperl -A
+ccflags=-msse4.2

./Configure -Dprefix=/opt/cperl-5.24c -sder -Dusethreads -Dusecperl -A
+ccflags=-msse4.2

Thank you for making Perl run faster.

Warm regards, Mario.

[dave_the_m]

To the contrary, Reini Urban is not incapable to work with others

That would explain why hordes of programmers have flocked to contribute to cperl. Oh wait, no they haven't.

Dave.

[Tux]

personally I think perl5 is heading in the right direction. The fact that some developers use undocumented or unsupported (hidden) features start complaining when they get fixed or reaped is by no means a measure of quality.

Personally I have blocked *all* modules written by the author of that blog for several reasons. One of them is that he does not follow the "rules" for writing CPAN modules from the start (like requiring a C compiler newer than what perl5 itself requires).

Even if his modules are actually good, his level of support (or better, the absence of it), revokes his right to complain completely.

Your opinion may differ, but you asked for opinions.

perl6 is mostly fun, and you can learn an awful lot from it. For me it is not production ready for just two reasons: 1. speed, 2. usable documentation. Documentation is worked hard on right now, but speed is not quite what I hoped it would be.

[xiaoyafeng]

Thanks for your reply. And I totally agree your opinion.

perl5 is too old, and obviously, it won't be added too many new, modern features in following version. So, you have to heavily rely on some 3rd modules on cpan. AnyEvent is a case, we all know its author is a unpleasant guy, but what if DateTime's author become bad?(sorry just a joke ;)) That's why I think perl6 would be a better choice although it's still on the way yet.

[stevieb]

"perl5 is too old, and obviously, it won't be added too many new, modern features in following version."

New features in: v5.8, v5.10, v5.12, v5.14, v5.16, v5.18, v5.20 and the most recent public release v5.22.

That covers 12+ years of perl development. New features are added during each release cycle.

To further, the p5 maintainers have implemented a newer 'experimental' feature, which allows you to import and/or enable/disable warnings for new features that are currently in 'testing' phase. This allows Perl end-user coders to test new features at will, and provides a life-cycle to test whether anything existing breaks (regression). This is an extremely sane way to introduce new features while allowing interactive community feedback. They also have a very strict deprecation regimen, so that you have at least two full release cycles to swap out code if something is deemed to be removed from core (an accepted experimental feature or a past piece of syntax that breaks new features. Both are uncommon though; usually it's the removal of a CPAN module that has been included in core, but it is being taken back out).

All that adds to the basic fundamental that backward compatibility matters, and these people are really listening to the community, and not disregarding backward compatibility. It's us end users who realistically dictate the future of perl, but only if we speak up using proper channels and sociable conduct. dave_the_m stated above that the maintainers test each build against numerous existing CPAN modules internally, and even reach out to module owners if they spot problems before we do. I think I'll be happy sticking with this setup forever, if possible.

PS. The C programming language is MUCH older than perl is, but it is still widely used...

Update: This is probably a good time to point out that writing unit tests to cover as much of your code base is always a good idea, regardless of how simplistic or minute you think your prod code is. This should be done for all code, but I have a new appreciation for CPAN modules after hearing what dave_the_m said. (Personally, I often write my unit tests as example usage code even before I write the actual code the example will test against).

[AppleFritter]

Age in and on itself isn't an indicator of quality. It's true that development in old projects can slow down[1], but this is by no means necessarily true, and young projects in turn can be lacking, e.g. in maturity, code quality, features and so on. I'd advise against judging projects by their age.

Perl 5, for what it's worth is undergoing active and exciting development; just check the perldeltas for the latest versions.

I also feel I should take up the cudgels for CPAN. You talk about CPAN as if it's a bad thing, but the opposite is true; CPAN is an extremely valuable part of the Perl ecosystem. CPAN allows you to (fairly) easily share the fruits of your labor, and benefit from others' in turn; as such CPAN is a large part of what has made Perl a community, rather than just a language.

Sometimes there's several competing modules on CPAN, but that too is a good thing. After all, isn't TIMTOWTDI (There Is More Than One Way To Do It) one of Perl's mottos? Competition is good, and the best solution will eventually win out. In fact universally-useful, well-maintained CPAN modules can be bundled into the Perl code, and are.

And if no module wins out, and if different people keep on using different modules, then those modules obviously fill different niches. There's room for more than one module for any given task on CPAN anyway, and people being able to scratch their particular itches and getting their jobs done with a minimum of fuss is what I'd call a success.

1. Whether that is a sign of stagnation, or simply conversion on the "Golden Code", is another question as well. TeX (i.e. core TeX itself, not the surrounding ecosystem) is perhaps the best example of the latter, a project that's not seeing much development anymore because it's approaching perfection.

[Old_Gray_Bear]

I have been writing and maintaining Perl code since 1994. The first thing that I worked on was a mail-handling system written in Perl4. I have written production code in both Perl4 and Perl5. I have been playing around with Perl6 for the last couple or three years, just to see what is happening in that space.

Last week I got a note from an old boss at the job I had in 1994. He is now the CIO and he has a contract to upgrade the old mail-handler system into something more modern. They run off the "if it ain't broken, don't fix it" theory; so the last time someone worked on it was when they "converted" it to Perl 5 (specifically 5.6.1) by changing the shebang and fixing the obvious errors.

That mail handler is a non-trivial system (you try parsing email address in Perl4....) that has been in service with out a major re-write for over 20 years. I expect the same kinds of life-spans for properly designed and built Perl5.xx systems. Particularly if the P5Porters continue the work of retro-fitting Perl6 concepts into the Legacy code base.

N.B.: I am actually, seriously, considering coming out of retirement for this. Redoing a Corporate E-Mail system and building it right, this time. (With a proper design, using CPAN modules as much as possible, a real test suite that exercises the entire system, modern hardware with enough RAM, disk and multiple multi-core processors, a real backup and recovery process, plus documentation for everything), is very**3 tempting.

[sundialsvc4]

This kind of life-span for a mainline Perl application is actually very common ... and there are of course a lot of programmers in the market today who were not even born yet (or they were but toddlers) when the systems that they are now being asked to work on were first made.

This, more generally, is the “legacy-code” issue, and this was the essential point of my (much-downvoted, of course) previous post.   It is not that the IT staff of the company in question is in any way clueless.   They know that the system now in-place is working, and that the company simply needs it to continue doing so ... which it will, unless someone decides to re-code the thing in Perl6 Java.   (You never correctly estimate either the cost or the true business risk of a re-write.)

This is also why I repeatedly advocate that you should write your source-code for the future ... because it will have one (even if you don’t ... watch out for bread-trucks next time).   The business value of your work is not tied to the language that you wrote it in, and the task of converting a mission-critical system, to anything else whatsoever, is fraught with costs and risks.   Lots of programmers in the business today simply have no idea of this.   (They are not, dare I say, old enough yet.)

Perl is not the only mainstream language today that is doing this sort of thing, of trying to radically reinvent itself under the same brand-name.   Python, PHP, and Ruby all did it, too.   Every one of them discovered that the “older, badder” versions did not disappear from service, because there was no business case to put existing business software assets at such risk, no matter how “–er” the new language-version might have been.   The contributed-software libraries which accompany all of these languages was the biggest part of the coffin-nail in that decision.

[stevieb]

There are some reasonable assertions here, which Old_Gray_Bear has given a very good example for. (Some of your assertions I disagree with though).

I've been coding Perl starting sometime between 2000-2k2, and I know for fact that some of that code is still in prod, unchanged at past orgs I worked for. Other than my modules that strictly need a certain version because I've decided to use a new feature, I've never had any of my past code break due to upgrades in the perl binary.

I want to point out here though that Perl6 is not an upgrade or a hack/feature-set to Perl5. It is a new language, where some of the benefits are ported to Perl5 so we can benefit. Going to P6 from P5 is akin to going from P5 to Python (or equivalent). We just thankfully get to share some of the same syntax/sigils etc

[stevieb]

perlpolicy is the document that describes the policy for the perl5-porters (those who maintain the perl core). Perl has always had the stance that backward compatibility is top priority, and work very hard to not break existing code.

If some author find their modules breaking on a new release of perl, they can and should always bring it to the attention of the perl5-porters email list.

This is why it is important to test your modules against perl-blead (trunk), so that if one of your modules breaks, you have that full release cycle to bring up your issues, and either argue perl should be changed, or fix your modules before the next official public release of perl is made available.

If an author is using undocumented pieces of perl internals (or is using documented parts in a non-conventional way) which breaks their code, they should bring it up to the maintainers to open a discussion. Whether this person did so or not I don't know. However, outright stating that the maintainers don't care about backward compatibility is both childish and irresponsible.

To complain about it after the fact, and release your own version of perl just for your modules is locking in your module users to using two separate versions of perl (that are incompatible) in their production environments.