This post will probably make me quite unpopular with some people,
and quite popular with others. Please, vote your conscience, in any
case, it will get the cat out of the bag.
I've always thought that the administration of this site should be
more open. In particular, though perlmonks is based on an open source
engine, and is about an open source language, it is not open source.
I've tried to talk about this before, most notably in Let the doors open wide!, but I think it's time to say it again, and hopefully write
it better. Also, this time, I have working (if ugly as hell, and
incomplete) code to share... yes, on a social post.
The code is what gives the title to this node. It's a pair of
scripts (at present), which make it possible to fetch nodes from PM,
and put them into a DB in the same form. To put it simply, this will
allow you to clone PM... if you have the rights to read the relevant
bits. (The code itself will be in a reply, because I can edit it
better that way.)
That brings me to my other point. For the majority of the users of
this site, this code will be of no use. This is because the nodes
that form most of the infrastructure of this site, you cant read. I
think this is a terrible shame. We have here a community of motivated
hackers, who know the language that PM is implemented in. We have a
body of software. Why not apply one to the other?
There is currently a group called pmdev. Users in this group can
read most of the site's code, and can write patches to be considered
by the gods, and possibly apply. Note that they do not have the
power to find out anything about a user, or a user's node, that they would
not have the power to see anyway. They do not have the power to
change nodes, or code, they can only suggest it.
This limit to their power means that, barring bugs, they have no
power to mess up the site, or find out confidential information.
There are two reasons that these powers are reserved for
pmdevils. The first is the fear of bugs. There have, in the past,
been times when security flaws were found in PM code, specificly SQL
injection flaws. These bugs can be read with pmdev access, and then
used. For that matter, if discussion of security holes was public,
they could be exploited between discovery and fix.
The software development community has, in general, decided that
these arguments are poppycock. The security flaws will be found sooner
or later by people guessing at them. The people who guess at them are
less likely to behave nicely. As to the window between when a flaw is
discovered and when it is announced, that will get shorter if the flaw
is public, now won't it. Security through obscurity may make you
sleep better at night, but it's hardly security.
The other reason is that there are already more patches then the
gods know what to do with. More eyeballs would make that worse, not
better. My response to this takes several forks. The first is that
more gods should help, or the ones we have could take some interest.
The second is that patches should be votable on, and replyable to.
The current structure of things allows for discussion, but does not
make it easy, nor open. IMnsHO, the things currently spoken about on
the pmdev wiki should be talked about in the (public) perl monks
discussion. Getting patches to be public, commentable, and votable
would take some work, but IMHO the work is well worth it. For that
matter, it would become easier, and more likely to happen, as more
people could see the code and do the work.
Warning: Unless otherwise stated, code is
untested. Do not use without understanding. Code is posted in the
hopes it is useful, but without warranty. All copyrights are
relinquished into the public domain unless otherwise stated. I am not
an angel. I am capable of error, and err on a fairly regular basis.
If I made a mistake, please let me know (such as by replying to this
node).
|
Re: Run your own perlmonks!
by chromatic (Archbishop) on Sep 28, 2003 at 20:01 UTC
|
Disclaimer: I am one of the gods, but I speak for myself, not the gods as a whole or any of the other gods.
We have here a community of motivated hackers, who know the language that PM is implemented in. We have a body of software. Why not apply one to the other?
Patches welcome.
As to the window between when a flaw is discovered and when it is announced, that will get shorter if the flaw is public, now won't it.
Will it? What if the flaw isn't easily fixable? What if no god is around to apply the fix? I realize these questions apply whether a flaw is discovered by reading code or by guessing at opcode parameters. That doesn't mean handwaving will magically answer them in either case.
The first is that more gods should help, or the ones we have could take some interest.
Yeah, and I should be richer and better looking.
It seems pretty arrogant and selfish of you to tell volunteers what they should and shouldn't do, especially since those volunteers have been keeping this site running for years. I suppose in some perverse way your frustration and demands could be interpreted as appreciation, but my brain just won't do that today.
Getting patches to be public, commentable, and votable would take some work, but IMHO the work is well worth it.
Voting on patches is a terrible idea. Perl Monks is not run by consensus. It's run by people who've proven themselves knowledgeable, mature, and trustworthy over the span of years.
I don't know of a single open source project that votes on patches in the terms I think you're describing — I think it'd be a fiasco. You're welcome to prove me wrong, though.
...it would become easier, and more likely to happen, as more people could see the code and do the work.
In my experience, getting people to do work is more work than doing the work yourself around 90% of the time. I don't expect more than two new coders to show up and write anything of merit if the codebase were opened completely.
Besides that, what are you going to do if you know that the code for the Personal Nodelet is just "pull some fields out of the database, linkify them, and put them all in a string"? Saints in our Book is just "pull some fields out of the database, linkify them, add some colors, and put them all in a table in a string". I don't think even the backup script is that interesting — if you've done any sort of web programming with Perl and databases before, you've already got the hang of it and you can learn the features of the Everything Engine from existing, open sourced code already!
Besides all of that, it's not difficult to become a member of pmdev. Making you work a little bit for it is a way of making sure you're serious. Again, even that doesn't mean that every member of pmdev has proposed a patch.
There are a lot of shoulds and maybes and mights in your post. I'd rather have facts. What are you going to do? What have you been prevented from doing? What are you going to do with the Perl Monks code that hasn't been released that you can't do with the code that has been released?
| [reply] |
|
|
Patches welcome.
Patching Everything wont help PM. And I think the motivation of PM'ers who want to participate is to change PM, not the latest release of Everything, which by all indications PM will never use.
It seems pretty arrogant and selfish of you to tell volunteers what they should and shouldn't do, especially since those volunteers have been keeping this site running for years.
Im perturbed by this comment. Actually what theorbtwo is saying is that the volunteers should open up to more volunteers contributing. This means that by having the older hands do a different kind of work (essentially management and oversite of a busy pile of pmdevers) the sum total of work being done grows drammatically.
Let me put it this way, if Linus or Larry insisted that every line of code in Linux and Perl was written by them in their style using the solutions they wanted to see implemented then neither would be the raring success it is. The success of both is a credit to these gentlemens maturity. They simply sit back and throw the occassional veto in. They stay at the high level of oversight and planning. They tend not to grind axes with their community. And if they dont have a personal solution that is better than a reasonable proposal they sure dont ignore or block the reasonable proposal. From what I can see pumpkings are specifically chosen for their maturity.
The fact is that there are only a mere handful of patches out of several hundred that have been written by pmdev that have been applied. Just about all that have been applied have been written by on of the gods. So this says that the gods really are pmdev and that they aren't actually pumpkings in any sense of the word. Which leaves me to wonder what the point is. And before you speak about quality ask yourself, of the many patches posted by pmdevers that have been ignored or not applied can it really be true they _all_ suck? Does it make sense that _none_ of them have been applied because there is a better solution coming down the pike?
A pressing example of this is your signature patch. It was poorly thought out. I have posted patches to it which correct its flaws and to make it consistent for root nodes and for replies. You haven't applied it. Why? Personally my guess is that you are insulted by this. Given what I see of your comments about the audience here you probably look down at me and find it hard to accept that I could possibly fix or improve what you have made. Yet the fact remains that I have. If our situations were reversed very soon after I saw the audience response (not positive, somewhat irritated) I would have applied the patches provided to me that improved the behaviour. And then thanked you for cleaning up my mess.
The core issue here to me is that the gods need to either expand their group with conservative but interested individuals for handling minor patches. (A responsible god would not apply a patch they did not understand.) Or they need to start spending more time doing the same, or they need to tell the pmdev group that basically its unlikely that a patch will be accepted or applied so dont waste your time, or maybe just comission specific patches from the pmdev team so that people wont waste their time.
Whatever happens I think the frustration theorbtwo has expressed is a genuine positive comment about the monastery. There are committed interested individuals who wish to contribute as only a hacker can to something they love. Denigrating them for wanting to do so is beneath the character of the monastery In my opinion.
In my experience, getting people to do work is more work than doing the work yourself around 90% of the time.
Then you obviously dont know how to delegate. The only way I can see this being true in your life is if you like to micromanage people doing work for you. Its obviously not generally true otherwise organizations and businesses would not be able to function, having a situation like that that would just mean that financial collapse is on the way.
I don't expect more than two new coders to show up and write anything of merit if the codebase were opened completely.
Think about it chromatic. This comment is pure elitist nonsense.
---
demerphq
First they ignore you, then they laugh at you, then they fight you, then you win.
-- Gandhi
| [reply]
[d/l] |
|
|
demerphq, I respect you and I often agree with you. I think you're very wrong about my intentions and my character in this situation and I regret that.
I think the motivation of PM'ers who want to participate is to change PM, not the latest release of Everything, which by all indications PM will never use.
I believe we could use the latest release, which is much nicer, better documented, and easier to adapt.
if Linus or Larry insisted that every line of code in Linux and Perl was written by them in their style using the solutions they wanted to see implemented then neither would be the raring success it is.
That's exactly what Linus does. Larry is much less involved in the day-to-day decisions and he has amazing powers of compromise that nearly always lead to better decisions, but I've seen him shoot down ideas and implementations that he doesn't like.
A pressing example of this is your signature patch. It was poorly thought out. I have posted patches to it which correct its flaws and to make it consistent for root nodes and for replies. You haven't applied it. Why? Personally my guess is that you are insulted by this.
You are welcome to guess, but please believe me when I say that you are completely wrong. I wrote my patch for three reasons:
- Some signatures bother me and I'd love to be able to ignore them.
- I believe that code that sorta works is much better (even just as a discussion point) than a brilliant patch that isn't even written.
- I thought it would take less than five minutes.
The reason I haven't applied your patch is that I'm on vacation this week — I've been travelling, with sporadic access. My system administrator experience taught me never to apply patches before or during vacation. I'm willing to take responsibility for that.
Then you obviously dont know how to delegate... This comment is pure elitist nonsense.
"Merit" was obviously the wrong word, and I think that explains your interpretation of my comments. I don't mean to say that only people who've been on Perl Monks for four years are capable of writing patches. I simply mean that, in my experience, for every ten people who ask for the source code to a project, one will download it. For every ten people who download it, one will read it.
The numbers of people who patch open source projects are shockingly small — in my case, out of hundreds of modules I've used, I've probably written patches for a dozen or so. There's nothing wrong with that, but it doesn't match my experience that there will suddenly be a flood of bugfixes and new features if all of Perl Monks were to be made ready for download.
Your point about not enough patch checking and application is well taken — I think that's the most productive discussion we could have.
I don't think that saying, "Hey, volunteers! DO THIS YOU HYPOCRITES!" is productive, though, and I definitely got that tone from the original post. Whether that's fair or accurate, theorbtwo will have to judge. I could be misinterpreting his intent very badly.
| [reply] |
|
|
|
|
In my experience, getting people to do work is more work than doing the work yourself around 90% of the time.
Then you obviously dont know how to delegate. The only way I can see this being true in your life is if you like to micromanage people doing work for you. Its obviously not generally true otherwise organizations and businesses would not be able to function, having a situation like that that would just mean that financial collapse is on the way.
You overlooked a minor detail. It's the implication of the word "voluntary" in chromatic's sentence. Most businesses pay the people that do their work for them. So I'm not concerned about financial collapse, just yet.
While I appreciate your sentiment, I think you're reading chromatic's post much too negatively.
I do believe that he has a good point when he says we're not likely to attract a lot of volunteers by opening the code base. After all, getting adopted into pmdev is not very difficult (heck, I made it in just because I inquired a few minor details about the site from tye to write my newest nodes interface). That fact could, maybe, be advertised more, of course.
But as tye has explained time and again, and which is something only implied in chromatic's post, the lack of gods is a problem because patches have to be tested - both by themselves as well as in their interaction with the rest of the codebase. Obviously, it's easy for a god to do that for their own code on the "shadow" PerlMonks devel setup they have. Since that one runs on a backup of the live PM database, I can understand that they don't want to open that. (Or do you want all your /msg's out there for everyone to read?) So you have a situation where a small handful of people (with limited time on their hands to boot) have to validate all foreign code passed in from outside.
I think the biggest problem we face is what perrin has often complained about: that the code is stored in the database. Were it completely separate, it would be much easier to make the code available, and hand out a mock up database to test it with, without any of the security and privacy issues of the current setup.
Before that changes, contributing to PM development is always going to be a cantankerous matter.
Makeshifts last the longest.
| [reply] |
|
|
|
|
Voting on patches is a terrible idea. Perl Monks is not run by consensus. It's run by people who've proven themselves knowledgeable, mature, and trustworthy over the span of years.
I don't know of a single open source project that votes on patches in the terms I think you're describing — I think it'd be a fiasco. You're welcome to prove me wrong, though.
The Mozilla project allows users to vote on specific bugs in the bug tracking database. This gives developers feedback about which bugs to fix first and which features they should add. This is different from voting on the actual patches - the users don't vote for those, but they are free to comment on them if they want. This is a nice way to keep the users feeling like they matter to the project, without giving them the ability to make messes that the developers have to clean up.
Besides that, what are you going to do if you know that the code for the Personal Nodelet is just . . .
I'll give you a counter example - why does anybody need to know that /bin/ls is "just" calling readdir() and printing to stdout? On the other hand, why does it need to be a secret? All other things being equal, I would rather have code be open. 98% of the time, I don't even look at the code, but when I do dig into it, I get to have that "oh cool, this is how it works" feeling that drives hackers to take things apart.
It isn't harmful to open up the code to the site so that people can look - unless it's insecure code, and then it should be fixed.
| [reply] |
|
|
The first is that more gods should help, or the ones we have could take some interest.
I interpreted that as "there should be more gods, because then the work would be spread over more people", not as "the gods we have should help more"
Also, i though that voteing and commenting on patches was for pmdevils only, and while voting might not be that great, it could give gods an idea as to which patches the pmdevils thought were best/most urgent.
English is a crappy language that is open to many different interpretations. :)
___________
Eric Hodges
| [reply] |
|
|
| [reply] |
Re: Run your own perlmonks!
by demerphq (Chancellor) on Sep 28, 2003 at 16:42 UTC
|
I agree that there are issues here. I have personally been disinclined to propose patches becuase the response and feedback of the patches and proposals has been limited and frankly a little discouraging. However I am mature enought to realize this situation may be because of legitimate concerns about the quality of the patches, the need of the patches and the skills and responsibility of the developers. (In this case me.)
I think there are issues with opening up code access, and responsibilty. Serious hacks could cause us problems with our kind hosts. Responsibility comes with expanded commit rights to the DB. A poorly judged patch could bring the site to its knees, and require serious time commitments to clean up and resolve. Those that would have to do the dirty work would not be impressed or happy about it.
OTOH A process of review and consensus amongst the developers and an expanded gods and pmdev group would probably be a good middle ground. However assessing responsibility and and skills of those involved is difficult and I can see why vroom and tye are shy of the subject. Theres a lot of possible solutions to opening up PM to more development, but all of them require risk and trust and work and responsibility and time.
---
demerphq
First they ignore you, then they laugh at you, then they fight you, then you win.
-- Gandhi
| [reply]
[d/l] |
Re: Run your own perlmonks!
by dws (Chancellor) on Sep 28, 2003 at 20:10 UTC
|
Before you get to far down the road of thinking how much better things would be if we were "open", read Clay Shirky's essay, A Group Is Its Own Worst Enemy.
| [reply] |
Re: Run your own perlmonks!
by LazerRed (Pilgrim) on Sep 28, 2003 at 21:53 UTC
|
This is going to be a bit wierd...
I like my Truck. It was the first new vehicle I have ever purchased. I know all the specifications right out of the sales brochures, and all the statistics from magazine reviews. It's a good truck, and I'm really happy with it. I do not know what the torque specification for the head bolts are, when it's time to change the oil, I have to lookup the filter in the manual at the store. Even though I like my truck, I do not want to become a mechanic, nor make it a personal hobby to know that much about it. I just want to get in, turn the key, and drive.End of wierdness...
I hate statistics without supporting documentation just as much as the next guy, but I'm going to go out on a limb here. I'm guessing that 90% of the members of the monastery feel the same way about perlmonks.org as I do about my truck. They like it, and just want it to work without having to make a hobbie out of it. Sure, if we were given the option to vote on patches, upgrades, or changes, I'm sure several people would indeed vote because they feel that community spirit. But is that really necessary? Personally, I don't think so. Most people just want to drive.
I think you want to have additional participation here, and that's great, more power to you! Without people that are motivated and willing to spend their personal time this site would be toast. My thanks to you for your eagerness, and to the current gods & pmdev folks.
LR
Whip me, Beat me, Make me use Y-ModemG.
| [reply]
[d/l] |
Re: Run your own perlmonks!
by liz (Monsignor) on Sep 28, 2003 at 22:16 UTC
|
If anything would need to be patched or developed right now on PerlMonks, it would have to be to make PerlMonks faster: I find the response times way too large, even when there are only a few users lurking in the Monastery.
Anything else is icing on a cake that is being squashed. We need to build a stronger cake before we put more icing on it and we can all have it too (pardon my mixed metaphores).
Liz | [reply] |
|
|
Here here! Let's start be removing as many features as possible. How about:
- Junk the chatterbox. If people want to chat they know where IRC is.
- Drop configurable nodelets. If people want a configurable set of links and widgets next to their Perlmonks view they can setup a Sidebar.
- Cancel SuperSearch. Just find a way to let Google index the site and link to that. Everyone's doing it.
- Offline The Past. Pick a cutoff date a couple years back and move everything older than that into cold-storage. Make it accessible if you must, but put it on a different machine.
I love PerlMonks, but the enormous feature creep has resulted in a site that's a hair's breadth away from unusable. Let's do one thing, host the best Perl disussions on the web, and do it well.
-sam
| [reply] |
|
|
Some remarks:
- Junk the chatterbox
- I disagree with that. I think the integration with the site is essential and has significant advantages above IRC by itself. However, what I don't understand is why the ChatterBox is not in an Iframe, so it could be refreshed by itself. Surely most browsers nowadays support this?
- Cancel SuperSearch
- I hope to be able to come up with a solution for that before too long.
Also with regards to voting: each time a vote is done, the whole page needs to be regenerated. Even though I'm not that interested in knowing the result. And double voting is already handled (i.e. ignored) by the system.
So why not have an option in your user settings that, if set, would return a status 204 (no change) on a vote? The page would remain on screen for you to continue to read and/or go to a next node. And the page would not have to be regenerated. In my average morning run through the monastery, that would save about half of the pages from having to be generated.
Liz
| [reply] |
|
|
|
|
| |
|
Junk the chatterbox. If people want to chat they know where IRC is. I can't use IRC at work. Chatterbox has kept me sane for the last 2 weeks.
| [reply] |
|
|
Killing the CB: Add another vote for keeping the chatterbox.
IRC isn't ways available (i.e. because it's being blocked by the corporate firewall), but web traffic is OK.
Getting quick help and taking a break in the CB is really helpful and aids in maintaining a sane outlook on the World :-)
Offlining the past: IMHO, if a node has a certain amount of views or rep, regardless of it's age, it might be worthwhile keeping, rather than placing in cold storage.
If the information in this post is inaccurate, or just plain wrong, don't just downvote - please post explaining what's wrong.
That way everyone learns.
| [reply] |
|
|
And while you are at it, cancel the AnonyMonks as well. I guess they cause a higer workload than the monks romping around the monastery.
| [reply] |
|
|
Re: Run your own perlmonks!
by Anonymous Monk on Sep 28, 2003 at 18:27 UTC
|
The software development community has, in general, decided that these arguments are poppycock. The security flaws will be found sooner or later by people guessing at them.
The 'software development community' isn't exactly the epitome of quality engineers. As for the abused phrase "security through obscurity" it only partially applies here. The code is not being distributed in any form. It cannot be reverse-engineered. The only risk is people guessing at query strings and hoping they'll break something. So you're only partially correct.
| [reply] |
Re: Run your own perlmonks (code)
by theorbtwo (Prior) on Sep 29, 2003 at 07:16 UTC
|
As promised, the following is the code for getnode.pl. This will take a node out of pm and into your local perlmonks. Note that it's only as good as the data that it can actualy get -- it won't allow you to see things that you otherwise could not.
To use, first replace pmusername, pmpassword, dbusername, and dbpassword with the approprate values (preferably for a pmdev or god on the PM side, and with a user that has create table privileges in the named database on the database side). Then call it with a nodeid, a nodename, or a nodename and type. I suggest you first start with the pmmodule nodes, then the dbtable nodes, then the nodetype nodes (which you have to do partaly manualy for the time being, mostly because I'm lazy and haven't written it).
There are also many things you have to patch or set up manualy to get a semi-working PM. I'll document those at a later point.
Note that this code is horribly written, horribly innefficent, horribly incomplete, and generaly horrible. Please, watch this space for updates.
Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).
| [reply]
[d/l] |
|
|
