http://www.perlmonks.org?node_id=1029441

I guess Perl would be pretty useless then. But that's actually a red herring question, because if CPAN doesn't exist now, people will soon create it or something like it.

The real question to ponder is: if you were to design something like CPAN today (a.k.a. 2013 instead of 1995), how would you design it? Assume things like RubyGems, PyPI, Packagist, npm are already there for us to steal good ideas from. Would you design it completely differently, or do you want nothing to change at all?

We are talking about the whole ecosystem here (PAUSE, the metadata, the clients, the accompanying services like CPANTesters, MetaCPAN, etc).

API instead of filesystem. Looking at the trends, the static pages and filesystem-oriented design are out, replaced by an API-/service-oriented one. A feature will soon be added, one which always remains absent from current CPAN: download/install counters.

More centralized. Servers are now usually powerful enough (in terms of CPU and bandwidth) to host something like a CPAN web application, so being distributed and FTP-/mirror-friendly now perhaps do not need to be a primary design criteria. A caching/proxy/minicpan-like mechanism can always be added later.

Client. The main CPAN client will be something no fuss and configless like cpanm right from the start, as there are no mirrors etc or static index files to choose/download anyway. Features like installing from github/bitbucket by pointing at a URL will be standard.

Metadata. Things that CPAN do right like separating build/test/runtime dependencies (something which most other languages do not bother to do) are certainly still in.

I actually love the Text::Something::Blah or WWW:: namespace style, so that will also stay in (it's a good balance between full reverse DNS style in Java like org.mycompany.Package.SubPackage, and unique+cryptic names in the Ruby/Python/npm world like cucumber/jinja/twig/whatever). One thing that should be popularized is for authors to add tags or categories metadata to its distributions (it's something that's already in the spec but rarely, if ever, used).

Updating module status, like marking it as deprecated, should be doable without creating a new release.

Something like BackPAN should definitely exist.

Other ideas: Live inside a single git repo or as a github project?

It's still early here and I'm still half awake. But I'll add more later.

  • Comment on Would you stay with Perl if there were no CPAN?

Replies are listed 'Best First'.
Re: Would you stay with Perl if there were no CPAN?
by curiousmonk (Beadle) on Apr 19, 2013 at 04:46 UTC

    Though CPAN is Perl's killer app. I would say much of Perl's early success came without CPAN. For most of my daily Perl work, I hardly use CPAN.

    Its just that if you watch Larry Wall's talks on Perl 6, you will see why Perl had such viral adoption. Perl is designed to fill a niche which was totally empty when it came out. And that continues to be the state till today's date.

    When I first learned Perl in 2006(This was actually the time when 'When Perl is dead' screaming was its highest), actually I was pretty late to the party. I was predominantly a Electronics guy and had experience working only with Assembly language and C. When I took up a job at a MegaCorp, I started using C. Then I went through that phase of using C and mashing with a dozen unix text processing utils. A colleague once saw what I was doing on the monitor as asked me to look into Perl. And that was it. I didn't even have a internet connection(I'm serious, the company was not a very programmer friendly company). All I had was the Camel Book. For the next six months I did pretty much all the work with Perl, without even knowing CPAN even existed. And pretty much nearly every body around me did the same.

    Only 6 months later, another colleague of mine who had joined us recently, introduced us to CPAN. It didn't change the reason why we were using Perl. It just added more reasons.

    I'd say for Perl is still indispensable, for nearly anything do with Unix scripting. I will rather use $var =~ s/<find-regex>/<replace-regex>/g on any day than use Python's .compile .match .group and all those tens of API's to do simple tasks what can be achieved in a line of Perl code.

    Need of the hour isn't redesigning the CPAN or the packaging system. Its really to educate users as to how Perl can be helpful to them. Its about writing good books, like the Modern Perl Book and make them famous. To give you an idea about how bad the situation is- A good part of everyday OO programmers in the world today, don't even know something like Moose exists. Even if they were to know, there is nothing like a nice pdf friendly manual to tell them how it can be use full to them. Compare this to something like Python or Java where very elaborate manuals exist on how awesome their stuff is.

    Also advocacy involves going to programming/tech forums and showing solutions in Perl. To give you an example nearly every thread on Hacker news which has a need for a program gets posted with Python snippets. Compare this with like say the early days of Perl and the 90's where the forums/mailing lists were full of people like Tom Christiansen and others showing solutions in Perl. The 90's were also full of people writing real awesome articles and circulating them on how Perl can solve their problems easily.

    Back in the day comp.lang.<insert-some-list-here> existed where people would have golf contest or some sysadmin joined into show how pipe kungfu can be avoided with one line of Perl. These those have been replaced with forums like Stackoverflow.

    Also CPAN is a killer feature, but unfortunately even if you are awesome people get used it and sooner or later it no longer remains awesome. Its like eating an apple everyday, first it feels very good then - slowly it becomes boring. Its called 'law of diminishing utility'. Python was first all about 'Readeability matters' but now they have moved to stuff like Twisted, Django. Those are the new killer apps. The thing is Killer apps themselves don't remain constant.

    Lastly Perl 6 needs to come out soon.

    Languages like Python and Ruby can today run on multiple VM's, they have lesser technical debt and backwards incompatible problems than Perl.

    Beyond all this Perl 6 problem is something like promising kid an Ice Cream and then going back on the promise. It creates a lot of disappointment within the community.

Re: Would you stay with Perl if there were no CPAN?
by tobyink (Canon) on Apr 19, 2013 at 09:08 UTC

    Perl's module ecosystem is probably the only reason Perl 5 still has any success, and the lack of the ability to run any of it is the main thing holding back Perl 6.

    Perl 5 as a language has some nice features: It allows most high-level LISP-ish constructs (except macros, though they can often be faked with stringy eval, Devel::Declare, etc) but with a less insane syntax; its object system is bare-bones (but enough to allow powerful OO systems like Moose to be built on top).

    But Perl 5 has some ugly warts too - ambiguous syntax requiring perl to fall back to heuristics in some cases (such as deciding when "{" begins a block, and when it begins an anonymous hash); two-argument open; the fact that built-in exceptions are thrown as strings rather than objects (requiring string parsing if you want to catch and handle them); awful, awful support for shared data in multi-threaded code; etc.

    There are some other languages that have many or all of Perl 5's good points, but fewer or none of its drawbacks. Perl 6 and, to a lesser extent, Ruby and Scala are examples.

    But they don't have CPAN.

    And so Perl 5 wins.

     

    As to the current structure of CPAN, I don't really think the internal organisation of the archive is of much importance to most end-users. What's important is that people can type a single, simple command to install a distribution and all its dependencies; what's important is that the right module for a particular task is easily findable; what's important is the QA infrastructure around it (RT, CPAN testers, CPAN ratings, etc).

    If you can deliver that, then the actual mechanics of how it's hosted (centralised versus distributed, etc) is irrelevant to most people. If it were hosted on SD cards delivered by carrier pigeon, that would be just fine by me, provided they delivered quickly.

    Some people (I think) if they were designing CPAN today... you'd sign in via a Facebook/Twitter login; you'd have to upload via an Javascript-driven form; you'd be listed on the front page if your upload got enough "likes"...

    package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name

      ... or to put it another way, Esperanto may be a saner language than English, but it ain't got no Shakespeare.

      package Cow { use Moo; has name => (is => 'lazy', default => sub { 'Mooington' }) } say Cow->new->name
      you forgot to mention the lack of function signatures, forcing people to use workarounds like my ($a,$b) = @_ and the need of manual parameter check.

      > the fact that built-in exceptions are thrown as strings rather than objects (requiring string parsing if you want to catch and handle them)

      And I wonder why this was never expanded to return blessed strings providing all meta informations...

      IMHO many "problems" of Perl5 could be easily fixed if changing the code was easier.

      Cheers Rolf

      ( addicted to the Perl Programming Language)

Re: Would you stay with Perl if there were no CPAN?
by Your Mother (Archbishop) on Apr 19, 2013 at 20:45 UTC

    I realize the title wasn’t the overriding meaning of the post but–

    Would you stay with Perl if there were no CPAN?

    I programmed in Perl for about two years before I was even aware the CPAN existed. Perl’s sweet-spot-finding ability to conform to the hacker’s thought process + its ubiquity on *nix is what won me and would keep unless there were something very similar. I adore Perl. Though I use the CPAN all day, every day, that doesn’t really enter into it. Larry doesn’t get enough credit for his design. I can hack Java and PHP et cetera but it’s a joyless chore; without Perl I would be in a completely different line of work.

    Also metacpan really does a good job of tying together a lot of stuff and continues to improve.

Re: Would you stay with Perl if there were no CPAN?
by space_monk (Chaplain) on Apr 19, 2013 at 13:11 UTC

    What never ceases to amaze me is the number of people who start to code without a quick look through CPAN to see which parts of their project have been implemented before.

    If any of my proposed solutions have minor errors, it's because I don't waste my genius on trivial matters. :-P
      > What never ceases to amaze me is the number of people who start to code without a quick look through CPAN to see which parts of their project have been implemented before.

      A friend of mine just told us at a perlmomgers meeting how bad Data::Dump output is and that he invented a better, more readable and terse format for displaying nested data.

      Turned out to be a just kind of YAML ... only incomplete and not parsable!

      sigh...!

      Cheers Rolf

      ( addicted to the Perl Programming Language)

      I imagine the situation to be as bad as (or if not, worse) in languages like Ruby, Python, JavaScript, PHP (Packagist). I mean, looking at http://modulecounts.com the module count for those languages go up like crazy. There's got to be a lot of duplicated efforts, hasn't it?

      How do we improve this?

        > I mean, looking at http://modulecounts.com the module count for those languages go up like crazy.

        Better don't confuse quantity with quality.

        Gem seems to have plenty of orphaned alpha stuff because publishing seems to be very easy and hipsters are encouraged to reinvent the wheel, just with two or three strokes of monkey patching and no testing.

        I mean should we encourage people to upload plenty of empty distributions with cryptic names?

        And PyPi is far from CPAN.

        See http://davidrobins.net/?idx=4305 for a decent comparison!

        He's a nice exception to the usual pythonistas spreading FUD, this community has a high ratio of fundamentalists.(at least I can't avoid meeting them)

        Please don't misunderstand me, it's always good to learn from others and I like a lot about Python & Guido.

        But don't underestimate the benefits of Perl culture and a decent community.

        Cheers Rolf

        ( addicted to the Perl Programming Language)

Re: Would you stay with Perl if there were no CPAN?
by davies (Prior) on Apr 19, 2013 at 17:04 UTC

    For what puts one user off CPAN (the utility rather than the archive), see Re^3: Installing Template::Toolkit on Windows. When I asked a similar question - how would you do CPAN differently - at a London PM social, I was introduced to semantic versioning (http://www.semver.org) as something that would be an improvement.

    Regards,

    John Davies

Re: Would you stay with Perl if there were no CPAN?
by educated_foo (Vicar) on Apr 20, 2013 at 15:12 UTC
    I started using Perl because it was ported to everything and consistent across platforms, unlike shell script. It may have been several years before I was aware of CPAN -- I don't remember, because that was a long time ago.

    I'm not intimately familiar with other languages' packaging systems, but from what I know, they seem worse. For example, Ruby Gems are catastrophically stupid, combining the thing that downloads modules with the thing that locates and loads them on your local machine. This is just begging for complexity and security problems. Perl's simple directory hierarchy for storage, and multiple clients for installation, are a much better solution.

    EDIT: Perspective from a real sysadmin. Basically, "you are not a special flower, so don't require your own special packaging system."

    EDIT 2: I briefly thought "just use an existing CDN rather than a custom set of mirrors," but realized that it's easier to get people to donate mirrors than to hit them up for money to pay a hosting company. Without a corporate champion, that's kind of important.

    Just another Perler interested in Algol Programming.
Re: Would you stay with Perl if there were no CPAN?
by Anonymous Monk on Apr 19, 2013 at 07:46 UTC

    API instead of filesystem

    filesystem is an API -- all that's required for download stats is to publish ftp/rsync logs somewhere the repository -- all that's required for install stats is for users to enable CPAN::Reporter or equivalent

      I love the style of the answers :) (that's sarcasm btw).

        I love the style of the answers :) (that's sarcasm btw).

        And I love incomplete incoherent and muddled thinking (not sarcasm)

Re: Would you stay with Perl if there were no CPAN?
by sundialsvc4 (Abbot) on Apr 19, 2013 at 21:11 UTC

    I frankly suspect that CPAN is “what all the fuss is really about.”   A language is a language is a language ... but a language plus a powerful and extensive library of (mostly...) known-good software is a tool.

    I don’t think it wrong to say that “CPAN will never change, because it should never change.”   There are millions of man-hours of effort represented in that code base, and there are well-known, well-understood methods for working with it which are by now deeply embedded in all sorts of mission critical working procedures.   The rightful resistance to change is tremendous, no matter how “–er” some “improvement” might be.

    Sometimes we overlook just how good we do have it.   For example, a few years ago at A Prominent Place-of Legendary Expertise, I watched as someone decided to endeavor to replace a very important Perl subsystem with Ruby.   Observing that Ruby had “gems” that appeared to be more-or-less equivalent, they (self-)confidently plowed ahead ... into a hedgerow of FIXMEs and TODOs.   The depth and maturity of the Ruby packages, at that time, were nowhere close to being “equivalent to” Perl’s, and they neglected to take that properly into account – or, I suspect, even to consider it – during their initial project planning.   They looked at Ruby at its then state of the art, and saw Perl at its then state of the art, and did not prove their guiding assumptions.   The outcome was painful to watch.   Now, it goes without saying that the Ruby teams are no slouches, and this is n-o-t language-bashing (or team-bashing!), but the maturity and completeness of the CPAN library is also very much “what the fuss is about.”

    In every case, “the contributed library” is a fundamental part of every language system, and its raison d’être.   In some odd way, a programming language is really just a tool that enables you to apply existing, well-tested solution libraries to your problem de jour.   (And, oh yeah, to construct and manipulate data structures.)

Re: Would you stay with Perl if there were no CPAN?
by DrHyde (Prior) on Apr 23, 2013 at 10:51 UTC

    What makes the CPAN so damned great is that 20 years ago, no-one designed it. People just built stuff to satisfy their needs. 20 years ago it would not have been possible to imagine all the things that now hang off the CPAN, so what we ended up with is something incredibly simple and flexible.

    To address your individual points:

    • API instead of filesystem: thankyou, but no. A filesystem is simple, and simple is good. API plus filesystem is a nice idea though, and, just like lots of other services hanging off the CPAN, some people thought it would be useful to them, and wrote it. And so we have metacpan.

      However, if I'd had to fight with a fancy-pants HTTP (presumably) API instead of a nice simple directory hierarchy, I doubt I'd have produced CPANdeps or cpXXXan. Why? Because I did most of the development offline, and having to fight to set up a web server and someone else's application just so I could develop my application is a barrier to entry that I, being quite lazy, would probably not have bothered to overcome in my free time. And in the case of cpXXXan, I would have had to do a lot more work to replicate or subvert the API so that other people could use the service, instead of just spitting out a plain text index file.

    • More centralized: being distributed and mirror-friendly makes it really easy to slice and dice the CPAN, without having to write lots of extra code. It makes it easy to have local copies that you can use offline too.
    • Client: Back In The Day, configuring your CPAN client was a useful thing to do. Now, though, it's pretty much automated and so the normal CPAN client is pretty much the same as cpanm, just with useful output.
    • Metadata: build vs test vs runtime dependencies actually came very late to the CPAN.
    But as for "if CPAN doesn't exist now, people will soon create it or something like it" - really? I'm not aware of anything for any other language that comes close to the CPAN, which seems to indicate that they won't. Even ignoring all the peripheral projects like CPAN-testers, I'm not aware of anything that approaches CPAN + search.cpan.org + CPAN.pm. I wonder how much of that is because people look at the CPAN and try to over-design their clone of it.
Re: Would you stay with Perl if there were no CPAN?
by Anonymous Monk on Apr 19, 2013 at 07:42 UTC

    Test::BDD::Cucumber? :P

Re: Would you stay with Perl if there were no CPAN?
by 5mi11er (Deacon) on May 22, 2013 at 18:52 UTC
    It seems to me that the new languages that have taken the idea of CPAN and attempted to improve upon it have mostly failed. Everyone still compares PEAR, RubyGems, PyPi, etc to CPAN. (Please note I am not saying these repositories have failed - I'm saying they failed to significantly improve the CPAN model)

    If one of them had actually improved greatly on the CPAN model, comparisons to that new model would have taken over.

    -Scott

      If one of them had actually improved greatly on the CPAN model, comparisons to that new model would have take over.

      can't fight fame inertia