Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^2: Losing faith in CPAN - unresponsive module authors (ownership--)

by tye (Cardinal)
on Jun 25, 2008 at 07:51 UTC ( #693902=note: print w/ replies, xml ) Need Help??


in reply to Re: Losing faith in CPAN - unresponsive module authors
in thread Losing faith in CPAN - unresponsive module authors

More CPAN modules need to be hosted on revision-control-system repositories so that it is easier to submit patches and easier to produce releases after patches are applied. Although it may not seem like a lot of work to apply a patch and release a new version, it is certainly more work than it should be and this leads to authors being more likely to be blockers of new releases.

I wish CPAN would ( place more emphasis on / pay more respect to ) progress and less on/to "ownership". Although the shift here would be rather subtle, I still think that it would help a lot to have the default be to allow new versions of modules to be indexed even if they aren't by an "authorized" author, so long as it has been a few months since an "authorized" author released a version.

It shouldn't require manual intervention to allow somebody to release a new version of an abandoned module. If the "owner" objects, then they can become active and prevent such releases from being "official" (and thus prevent them from being fully indexed) by simply releasing every few months.

If Joe Random Contributor can easily not just write a patch that fixes a bug but actually check in a patch and then check out a new snapshot of a module (and reconcile any work-in-progress that has not yet been released to CPAN), then Joe is more likely to feel like producing the full patch that updates the Changes file and adds appropriate unit tests (etc.).

I've certainly refrained from producing patches for modules quite a few times because the black box (hole?) of "you have to submit it to the 'author' and hope they do something with it" often doesn't look promising.

I have quite a few small patches to modules in various states at various places. From time to time I try to produce a new release of a module and usually find that the little details end up soaking up enough time that the release never actually gets finished and on CPAN before I'm pulled away for other tasks. Then, when I return, more little details have amassed (or been misplaced) and the same failure happens again.

So I've started down the road of putting my modules into public git repositories. But, alas, that task is not yet completed either. It would be cool if CPAN (or, more likely, PAUSE) were to acquire such features (or at least to facilitate them), but we are all volunteers, of course...

- tye        


Comment on Re^2: Losing faith in CPAN - unresponsive module authors (ownership--)
Re^3: Losing faith in CPAN - unresponsive module authors (ownership--)
by moritz (Cardinal) on Jun 25, 2008 at 08:38 UTC
    I fully agree with all of your points. (Except that I use svn instead of git, but that really doesn't matter).

    As long as the technical side isn't resolved, it might help to encourage people to hand out co-maintainership freely.

    It might help to have a wiki (or some such) where you can offer upload privileges, and list TODO items. For example I'm happy to help with Unicode handling of pure-perl modules. I'd just scan that list once a week, and see if there's something interesting. And when I work with a module anyway, I can just as well apply a few simple patches that live in the RT.

    Or I could just notice a module I'm familiar with, and accept co-maintainership to to decrease the response times to bug reports and patches.

    My experience with the pugs repository are very encouraging. We hand out commit bits very freely (when somebody say "I found a typo in file $file", the answer is usually "should I fix it, or do you want a commit bit?"), and so far haven't had any problems with vandalism or very poor commits.

    In any case I think it's important to try to lower the entry barrier for contribution to CPAN modules. Maybe somebody wants to write a Meditation on that matter (to increase visibility and to have a better title)?

      (Except that I use svn instead of git, but that really doesn't matter).

      I'd disagree a little bit with that (in the context of abandoned modules and public repositories (and in general because git and hg are just cooler than SVN, but I digress :)).

      If you're using something hosted in SVN you're at the mercy of the repository host, since the project history lives in that one location. If they go walkabout / get hit by a bus / take their ball and go home you lose the project's history unless people have been maintaining their own mirror of the entire repository. Whereas with git (or hg, or any of the new distributed systems) every contributor has the entire history and every repository is just as good as the "master" as soon as you've pulled in any remote changes.

      As for making contributing easy, git has a builtin command that bundles up a revision list into ready-to-email patches. There's also a corresponding command for taking an mbox of patches and conditionally pulling them in for the upstream. Or the prospective committer can just put their repo up and say "Hey, I've done X. Look at branch test-X at git://mybox.example.com/snork and see what you think".

      There's also a security aspect (not the exact word I'm looking for, but ...) in that it's harder to monkey with project history since revision identifiers are cryptographically determined from change contents (in git and hg it's using cryptographically secure hashes; I believe monotone goes even further in that updates are signed using public keys). Linus alludes to this in his talk at google in that he can lose his entire copy of the kernel source, but if the copy he gets from J. Random Schmuck matches his last good HEAD revision id he knows the contents are the same as what he had to that point.

      Did I mention git is just cooler than SVN? :)

      PS: Git. Git git git.

      Git.

      The cake is a lie.
      The cake is a lie.
      The cake is a lie.

        In general I agree that distributed systems are much better than centralized ones.

        <rant>

        The reason that I don't advocate git usage is that I simply can't work with it.

        I tried, a few times already. With different tutorials and with different tasks in mind, and I always failed.

        Last time I tried to bisect a regression in perl. There's that cool git-bisect tool which really works great, but it always reported some errors about files that would be overridden with a merge. It took me quite some time that I could work around those with git-clean (no tutorial on git-bisect mentioned that, and I don't think I was doing something unusual).

        After I found out, I managed to make a bisect, report the offending changeset to p5p, and rgs answered that it didn't seem to be the right one. D'oh.

        I keep hearing good things about git, so I try it out from time to time, but so far it was always disappointing.

        Am I too dumb to work with git? Or do the various tutorials suck? or is there some underlying concept that I have to grok, and then all problems turn into a harmless cloud on the distant horizon?

        Am I the only one who can't seem to get along with git?

        </rant>

        If you're using something hosted in SVN you're at the mercy of the repository host, since the project history lives in that one location. If they go walkabout / get hit by a bus / take their ball and go home you lose the project's history unless people have been maintaining their own mirror of the entire repository. Whereas with git (or hg, or any of the new distributed systems) every contributor has the entire history and every repository is just as good as the "master" as soon as you've pulled in any remote changes.

        So that's why people like git :-)

        meh.
        Do you find Git's long version numbers cumbersome to work with (e.g. compared with the simple numeric version id's used by svn or hg)?
Re^3: Losing faith in CPAN - unresponsive module authors (ownership--)
by dragonchild (Archbishop) on Jun 25, 2008 at 15:08 UTC
    This is the point of http://svn.ali.as. I've put DBM::Deep in there and will (at some point) get the rest of my distros in there, too. I'm also an admin there, if anyone wants a committer bit.

    But, yes, it would be nice to have both svn.cpan.org and git.cpan.org available. I wonder what it would take to get that up and running?


    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
      it would be nice to have both svn.cpan.org and git.cpan.org available. I wonder what it would take to get that up and running?

      Jarkko Hietaniemi owns cpan.org; I have trouble just pushing CPAN mirror updates to him, good luck trying to get new subdomains set up. DrHyde tried getting deps.cpan.org to point to cpandeps.cantrell.org.uk last year and Jarkko refused.

      On the other hand, svn.perl.org already exists, although it's currently only used for a few Perl projects. I'm not sure whether that's by chance or design.

      When 5.8.9 is released the perl5 porters will move from Perforce to git for the revision control system. RafaŽl Garcia-Suarez has said he will host git.perl.org at $work. There is talk of making things easier for maintainers of dual-life modules to use it for both blead and CPAN releases. I don't think there are any plans to open its use out to the wider community.

      Be that as it may, if you have a host and bandwidth to donate, I suggest you get in touch with Robert Spier and see about getting a different A record, like git-contrib.perl.org, set up.

      • another intruder with the mooring in the heart of the Perl

        I moved DBM::Deep from svn.perl.org to svn.ali.as because it was too hard to deal with the svn.perl.org admins. *sighs*

        My criteria for good software:
        1. Does it work?
        2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re^3: Losing faith in CPAN - unresponsive module authors (ownership--)
by brian_d_foy (Abbot) on Jun 25, 2008 at 18:45 UTC

    I don't think hosting more CPAN modules in revision control would matter to most people or most modules. Although a small group of people will actually make a patch against the repository, most people don't.

    Even if a few people do patch against the repo, I've only ever had one person also update the tests. It might be hard for some hard-core contributors to realize that, but most people either don't have the time or inclination, and even more people don't have the skills to do it.

    You don't need anyone's permission to upload a new version of an abandoned module. PAUSE just doesn't index it if you don't have privileges. A restive period is not helpful because some modules don't need updates in that time frame. There's no timeframe that you can apply across of all CPAN.

    Don't refrain from patches because you think the author won't help. Make the patch public, such as in RT, and other people can get it even if the author doesn't apply it. Don't hide from the rest of the world what you know because one person doesn't respond to you. And, making the patch public starts the history of an author not responding to you and makes it easier for the PAUSE admins to see what happened so they can transfer the module when necessary.

    --
    brian d foy <brian@stonehenge.com>
    Subscribe to The Perl Review

      Let me roll up replies to several items in this one reply...

      Some people may say that this is too much harassment,

      I say that it is excellent evidence that the system is broken. If anybody has had to resort to such drastic measures as often as you appear to have, then the system is requiring way too much from people who have to fight it just to get a patch applied. And the fact is that quite a few of us know darn well that without such drastic efforts, there is a reasonable chance that some patch that we produce to a CPAN module will never get applied.

      I don't think hosting more CPAN modules in revision control would matter to most people or most modules. Although a small group of people will actually make a patch against the repository, most people don't.

      Under the current system, most times when I have a patch for a module, I don't bother to submit it. And I've heard a ton of similar stories. So the fact that you see few patches being written has more do with most CPAN modules not offering easy and reliable ways to get a patch applied. Why bother to produce a patch when history shows that most of them will get ignored unless you start a personal vendetta?

      Even if a few people do patch against the repo, I've only ever had one person also update the tests. It might be hard for some hard-core contributors to realize that, but most people either don't have the time or inclination, and even more people don't have the skills to do it.

      I'm not a hard-core contributor. I'm a lazy bastard. I don't have the time to try to track down and then to nag some guy who submitted some implementation of a cool idea to CPAN months / years ago. When I run into a problem with a module, I do have a bit of time to fix the problem. If there is a system in place that makes submitting that fix easy and also likely to be useful, then I'm likely to use that system. And the easier it is and more importantly, the more likely it is to actually be applied, the more likely I am to patch documentation or tests as well.

      You don't need anyone's permission to upload a new version of an abandoned module. PAUSE just doesn't index it if you don't have privileges.

      Oh, no, it is much more than that. It also means that your module will be tagged with "Unofficial Release" in big red letters (at http://search.cpan.org/). There are probably even more important consequences with regard to using standard tools for downloading modules, but I haven't tried using any of those for quite a long time so I wouldn't know much about that.

      A restive period is not helpful because some modules don't need updates in that time frame.

      If a module needs no update, then why is somebody trying to upload a new version of it?! Your logic makes no sense. The time period isn't about "Oh, module X needs to be released every 3 months". The time period is about how long it is reasonable to wait for a module author to apply a patch and release a new version of the module.

      It would still be a big improvement if the clock started ticking as soon as an RT was opened against the latest release of a module. But that would still squash the spirit that motivated the creation of the first patch.

      The point is encouraging contribution. The point is trusting people to contribute good stuff while providing for ways to deal with the rare bad contribution.

      If I haven't released a version of my module in months and PAUSE / CPAN has been changed then somebody uploads a new version of "my" module that sucks, then I can upload an even newer version and fix the problem. Then the "bad" contributor is locked out again. If they are a serial bad contributor, then they you have a situation where administrative oversight is worth doing and take their upload rights away.

      I'd even be fine with the new version of the module being "unofficial" and unindexed for a period of time in order to give the official maintainer(s) time to respond to the upload. But it would be much more encouraging if the timeouts were automatic such that I could upload something useful and know that some person would have to make an effort to counter it in order to prevent it becoming official.

      When I find a problem, there is usually a pretty short window of time over which I'll remain motivated enough to work on fixing the problem. About 90% of the time when I find a problem with a CPAN module, it is a module that hasn't been released in quite a few months. And if I'm interested in patching it (which happens surprisingly often), then that means that the module is of reasonable enough quality that I think it can at least be made genuinely useful.

      So, if it were easy to create a patch, apply it, and release a new version based on it (rather quickly), then I'd just do it right then while I've recently worked out what needs to be fixed and whether the fix can fit in with the existing module or whether I just need to fork it or find a different solution, etc. Part of that process is evaluating whether the module is currently under active maintenance. If so, then I just submit to the maintainer. If not, then I should just release a new version right then.

      If there is a repository, then I can see what maintenance has recently been done, even though no new release has been produced in quite a while. And I can even see that some other bug fixes have been applied and deal with / release those.

      The point is to allow whoever comes along with the motivation to work on a module for an hour or few to find tools to help them to contribute usefully. They shouldn't find a system that values module ownership over all other considerations along with a system of red tape and long delays and appealing to the privileged cabal to deign to permanently annoint them with the burdensome tag of "I now maintain this module".

      What a perfect system for crushing contribution.

      CPAN is great for getting people to contribute new modules. It is also very good at getting people to not contribute to existing modules.

      And, making the patch public starts the history of an author not responding to you and makes it easier for the PAUSE admins to see what happened so they can transfer the module when necessary.

      Don't you love beaurocratic red tape? I guess if it is your own empire that you have built, then maybe you do. CPAN has become a little empire that allows people to stake out their own little empires within it.

      Don't refrain from patches because you think the author won't help. Make the patch public, such as in RT, and other people can get it even if the author doesn't apply it. Don't hide from the rest of the world what you know because one person doesn't respond to you.

      Like I said, I'm a lazy bastard. I have never grabbed a module, rooted around in RT looking for useful patches, downloaded the patches and tried to apply them, etc. That is way too much work. Heck, for many modules, I can implement my own replacement for it easier than that. So I'm quite unlikely to find appealing the idea of putting together a patch with the hope that it would be used as you describe.

      You are correct that CPAN makes this about the best that can be done. And I encourage people to follow your advice. But I'd much rather that people have much better options.

      The few times I've been successful at getting changes made to a module were the times I just wrote the author and said "make me a maintainer and I'll update it".

      And even when they do that, I'm still not allowed to give co-maintainer access to anybody else. PAUSE/CPAN doesn't even support their being more than one person who can dish out permissions to upload "authorized" releases of modules. It is just sad (and frustrating).

      The PAUSE system (because we're really not talking about CPAN) does not have a single point of failure as you describe. Some modules have only one maintainer, but the system allows for multiple maintainers. The system can handle multiple maintainers just fine. This isn't a problem with the system.

      Heh. Yes, keep telling yourself that. The system it perfect.

      There's no official channel for bug reports. I'm not sure what you submitted or where you submitted it to. Some authors use RT, some don't. It's not a single system.

      Note that I'll be encouraging people to use RT for reporting bugs even though my git repository has tools for bug tracking. RT is integrated with search.cpan.org (and PerlMonks itself recognizes it via the rt:// linking short-cut). If a module maintainer doesn't want bugs submitted to RT, then they should document how they want bugs submitted, of course.

      If you don't get a response, try another way.

      Yes, e-mail and then wait a week. If you don't get a response, then pretend you are stalking the person and try to find alternate contact methods. Oh, except that when that week is up you've probably lost most of your enthusiasm about getting the patch applied anyway. So make a new version of the module with a slightly different name and upload it to CPAN. CPAN likes it when you do that; they make that very easy.

      You don't want just anyone to upload any module. We'd quickly make CPAN useless as no one would know which version of a module to use. Do we use your version, or Joe's version, or Bob's version?

      Yes, just look at Wikipedia. This whole wiki "anybody can contribute" idea is an abject failure. It is much better that we encourage everybody to upload new modules that solve the same problem but with a slightly different name. That makes picking between them so much easier. If we had just one module with several versions written by several different authors arranged as a linear progression, each based on the previous one, that would be terribly confusing and all would be lost.

      You can always make local patches as the PAUSE admins figure things out.

      And as your interest in fixing things drifts away and your enthusiasm for contributing to CPAN gets converted to disdain with each successive long wait for the unraveling of the red tape...

      - tye        

        I think you've mistaken me for someone else. I do everything I can to make it easier for people to contribute not only to my modules, but to other people's modules. I'm not sure where your hostility to me is coming from. I'm here to help anyone with what exists right now. When something better is available, I'll help with that, too.

        I don't think the system is perfect, but I work with it to do the best I can and to help other people do what they need to do. Until someone implements something better, I'm going to keep trying to make what we have right now work, and to make it work as well as possible for everyone.

        PAUSE certainly supports more than one person being able to give out permissions to a module. If you're having trouble with that, write to modules@perl.org and one of us will help you if that's what you really want that. I've set that up for Parrot and even wrote their HOW-TO on how to use it. If you need that for your project, we can set that up. We can't fix any of the problems you don't report to us.

        --
        brian d foy <brian@stonehenge.com>
        Subscribe to The Perl Review

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://693902]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (8)
As of 2014-10-24 21:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (137 votes), past polls