Should it live or die?

[Jazz]

My fellow monks,

I am in a quandary and solicit your input. For some, this may be a confession, as I have not let this fact be previously known to more than a few.

I run a site called the Perl Archive, a CGI Resources-type site that lists various CGI programs written in Perl. At first, the site was a by-product of my natural-(link-)hoarding tendencies and a personal reference while I was primarily a graphic designer/webmaster who knew just enough Perl to be dangerous. I never dreamed that it would grow to the proportions that it has (1.2m page views/mo).

Shortly after the site was launched, the site began generating a respectable sum of ad revenue. Then, over the years, I grew hesitant about the less-than-average code that was being submitted and added to the site. Of course, it was being used as a directory of scripts for other webmasters (for their or their client sites) or programmers (primarily for learning).

So in an effort to encourage better programming practices, we launched a forum, PerlGuru. I wasn't aware of PerlMonks 2-3 years ago (was it here then?), and thought to offer it as a more "friendly" venue to newer programmers (I can't count how many times I admonished "Children, play nice with each other").

The forums have had quite a bit of success, mostly thanks to the diligence and expertise of japhy, converter, Kanji, mhx, rGeoffrey, and pro_4 (not sure if the last 3 handles are the same here on PM).

Then, in an attempt to further instruct potential Perl programmers, we launched a "Learning Center", which featured beginner to intermediate tutorials. japhy, Uri Guttman and Simon Cozens were hired to author articles, and were kind enough to see my ultimate goal and be extraordinarily kind with their fees (and donate a few articles as well). Even Lincoln Stein granted permission to reprint a chapter from his "Network Programming in Perl" book. Since then, we've had an influx of articles that either I was not able to edit for errors/omissions (mostly Perl/Tk stuff) or I do not believe is of a quality that would be useful to others.

We never wanted to compete with PerlMonth (which sadly seems to have gone defunct), but rather act as a "beginner/intermediate" supplement to them. Now, with the Perl Journal being a supplement to SysAdmin, it's an even sadder loss. I wouldn't mind trying to step into the void and offering whatever is needed for a free online version if author fees were doable and we can locate a devoted and qualified editor. But I digress.

Anyways, fortunately or unfortunately, the bulk of our traffic (1.2m page views/per month) is still to the guide of programs. While the guide is the "moneymaker", the rest of the site is what I perceive to be of the most value to newer programmers, and what I would have liked to see have the most success.

The final straw for me was the learning boost that I had during the PerlWhirl, YAPC and TPC last year. After that, very few programs seemed to be worthy of promoting for actual use.

My demise is that I have learned enough Perl to feel that the site I have lovingly created and maintained merely propagates bad and insecure programming (and sometimes, worse). It seems as though other monks feel the same way about the site (based on the node and its reputation). But on the flip side, some have told me privately that they like the fact that someone who actually loves Perl is running the site (as opposed to those who just does it for ad revenue) and doing what they can to promote Perl in and of itself.

At times I think of killing the entire site completely, but at other times… I think that the site still has the potential to promote good practices as well as make apparent the better programs that should be in wider use (e.g. NMS instead of the "other guy’s"). It's quite obvious that NMS probably won't make it to CGI-Resources, though it's a superior alternative.

PA is extremely well-linked from a variety of sites, search engines, a few books, and even from CPAN’s FAQ – it would be a shame to allow that visibility to go fallow when the opportunity to promote Perl in its glory is present.

I try to list all excellent scripts, but most authors of excellent scripts simply don't list them. The site no longer earns enough to warrant my time in hunting them down personally and list them. But as soon as I encounter any well-crafted scripts, I add them manually and make them as prominent as possible (such as NMS). Perhaps that can be considered censorship, but as both a Perl lover and a business owner, I'm constantly torn between what is right for Perl and what is right for everything else.

So, I beseech you your opinions and suggestions. Is the Perl Archive doing more harm than good? Should it just go away and not contribute to the existing, overwhelming evils? Should the site go into a different direction?

Your humble servant,
Jasmine

[chromatic]

Here's an idea. Take your weakness (an influx of programs of dubious quality), your goal (to improve the level of Perl programmers and of available programs), and turn the weakness into a strength.

Offer submitters the opportunity to have their programs reviewed by a qualified programmer. Given the number of "code review" questions here and the Ask-a-Guru sessions at TPC, there may very well be a market for this.

This provides review of scripts and some guarantee of quality and safety. Besides that, if the reviewer put his or her notes online, it would be a Learning Experience similar to Dominus' Red Flags Program Repair Shop series.

(Given the number of underemployed Perl hackers out there, you could probably find several people qualified to write articles or reviews for a nominal fee. It might not be hard to find an editor either. Drop me a /msg.)

With your levels of traffic and influence, it would be a shame not to seize the opportunity.

[footpad]

After that, very few programs seemed to be worthy of promoting for actual use.

You've heard of Sturgeon's Law, haven't you?

I mention this because I consider it a challenge, not an immutable fact. Since you have consistent traffic and a loyal userbase, why not let them loose and help improve what you've got? Make a list of the top 10, 20, 50, or how-ever-many things you wish were better--and more securely--implemented and then rewrite them. While doing so, keep track of the questions that people keep asking, for that suggests that a) people still aren't learning how to find Perl information or b) the information they're finding isn't explaining itself well. In either case, it suggests additional content and focus areas.

IOW, if you don't like the job you're doing, do a better one. Since you're the "manager" of the site, you might consider delegating some of it to the people with the time/desire to do it. In short, let it live a better life and it should be even more prosperous. If there's more than one way to do things, then there should be more than one way to find information about doing them.

After all, it's not like we're dealing with Isildur's Meme, the WebRing of Power, or other attempts to be amusing.

--f

[ask]

You should link to learn.perl.org. We don't have much content there yet, but it's the easiest way to subscribe to the very popular beginners mailinglists at perl.org.

Just yesterday I made a web interface to the nntp interface to the mailinglist archives, it's at nntp.perl.org. For example http://nntp.perl.org/group/perl.beginners.

Oh, and of course every perl site should link to jobs.perl.org. ;-)

But to answer your question; it would be a shame to just let it die. Maybe you can get other people to help you maintain the catalog? Or you could merge it with other similar projects?

 - ask

-- 
ask bjoern hansen, http://ask.netcetera.dk/   !try; do();

[BMaximus]

Imagine all the work you have put into this site. Now shut down the site and see where all that work has gone. You've done so much to get the site to where it it is now. You should ask yourself if you're willing to let all that hard work go and turn away the following you have so lovingly cultivated.

Your site has become a hub where individuals who are either in the know or not can find programs to download or use so as not to reinvent the wheel. Or they can have the opportunity to promote their own program. I agree with chromatic's suggestion. You see the need for a stricter regulation in the quality of programs that are being submitted to the list in order to protect the sites that use them and also to promote better Perl programing practices. Sometimes you have to say no to an individual who submits a program that you think is of dubious quality. Don't just flat out say no, but give them a reason as to why it has been rejected.

As for the programs that already on the list. My opinion could earn a slap on the hand but I'll write it anyways. Sometimes in order to bring order out of chaos the herd must be culled. Remove the programs from the list which you deem dubious. This will upset some of the programers who have submitted their works to the list, but kindly inform them as to why their programs were removed. Mayhap it is censorship, but its necessary. You've seen that a badly written program that your site has listed can hurt its reputation. So take this opportunity that has landed in your lap like a ton of bricks and use it to separate your site from others like it and lighten the load you bear on your shoulders.

Understandably you'd like to give the opportinity for all perl programers to show off their wares. However its ok to say no if they don't hold up to PerlArchive's standard. For me, this concern of yours shows me that you've grown and are ready to take your site to another level.

I would help but I'm not sure if I would be considered knowlegeable enough to review someone elses code. If there was some way to tell I certainly raise my hand. I see an opportunity.

/me steps down from soap box

BMaximus

[Jazz]

My sincere thanks to everyone who replied to this node, either here, by email, on irc or the chatterbox. I am now assured that the site has enough perceived value to not die, and that changes need to be made.

ask, links to various sites, including nntp.perl.org and learn.perl.org are now on the site. Thank you for the suggestion :) I'm sure that will prove to be the easiest change to have implemented :)

It seems to be the consensus that scripts need to be reviewed and marked (at the very least) as safe or unsafe. The idea of a Quality Assurance-type incentive is sublime. How then, should commercial scripts be reviewed? It's dubious that that authors will provide a review copy at no charge.

In order to implement a QA program, we will need to have a basic set of standards to which a script must adhere to pass the quality assurance tests. Here's a simple starting point for critique:

• Use strict and warnings (or know/explain why they are not)
• Scripts that work with files on the server must have –T (and useful untaintaing mechanisms)
• Should use CGI.pm
• Should use HERE_DOCS for lengthy text
• Bonus points for clear and consistent style :)
• _____Anything else?_____

As chromatic noted, there definitely appears to be an increasing interest in code review. The hard part about this is what should be charged for a personal code review? Because this was brought up to me previously with a potential reviewer, I've had discussions about what it would cost (and decided at the time that it was too expensive for the average coder offering a free script). We definitely need to steer clear of "get a review and get a QA listing" or something like that. The code review must be separate from the increased exposure that a Quality Assured listing would receive, lest the programmers think they are paying for an improved listing instead of a code review. We also need to tie in the version number of the reviewed code to maintain assurance and write disclaimers, disclaimers, and more disclaimers.

Several, including footpad have recommended that we take the initiative to either write or have written better versions of scripts in various categories. In a perfect world there would be at least one rewritten script for each category (eek!) with the exception of the Remotely Hosted category so that all bases are covered. For this, I would like to defer to other programmers, as it's probable that my "free time" will be spent implementing and maintaining the other improvements.

And last but not least, articles. If anyone has interest in writing or reviewing articles and/or programs, or helping with anything else that was discussed on this thread, please /msg me. I'm not certain what we can offer in terms of payment, but we'll do our best.

One last question, if I may impose a little more... In my opinion, the forums are redundant to PerlMonks and now nntp.perl.org. Should perlguru simply point to those sites instead? It seems to scatter knowledge quite a bit. It's uncertain how throwing 600+ users onto PerlMonks and nntp.perl.org would be received by existing participants, though. Opinions?

Again, thanks to everyone for their input. I look forward to revamping the structure of the site to make it a more useful and safe resource.

Many, many thanks,
Jasmine

[Chrisf]

It's uncertain how throwing 600+ users onto PerlMonks and nntp.perl.org would be received by existing participants, though. Opinions?

heh, probably the same way as when merlyn mentioned perlmonks in Learning Perl ;-)

Seriously though, I think it would be fine to provide a link to the monastery. There might be a few more RTFM responses, but we'd be working for the greater good, right? =D (you might also want to consider linking to the Q&A and FAQ sections)

[ask]

It's uncertain how throwing 600+ users onto PerlMonks and nntp.perl.org would be received by existing participants, though. Opinions?

There are +3000 users on the beginners lists at perl.org; there'll probably be room for a few more somehow. The PerlMonk community seems to be handling lots of users nicely too.

 - ask

-- 
ask bjoern hansen, http://ask.netcetera.dk/   !try; do();

[BMaximus]

The forums may be redundant but some of those users call it home and may not be comfortable anywhere else. Not to mention the knowlege that's stored there. I'd keep the forums on the site.

BMaximus

[social_mandog]

Thanks for posting this.

> I'm constantly torn between what is right for
> Perl and what is right for everything else.

For the original monks this was the answer.

However consider also that this organization has stayed as faithful as it could and achieved only local and intermittent success and that this organization has from time to time compromised and achived more global success

>How then, should commercial scripts be reviewed?

Well, you could refuse to post scripts that haven't been reviewed.

Alternately you could charge $50 per year to list scripts that can't be reviewed and mark those links as advertising.

>The hard part about this is what should be charged
>for a personal code review?

Maybe nothing. I've learned by reviewing scripts here and by reading people's review's of my reviews. I can justify the time to myself. (Arguably my review time is not worth paying for)

You could probably post 1-3 scripts a week here for review...

People could ++ the node if they thought it worth distribution and -- if not.

>Use strict and warnings (or know/explain why they are not)
>Should use CGI.pm

A couple regex's would probably test for this pretty quickly.

There is probably other simple QA code that could help you quickly winnow through the 90% ....

>Should perlguru simply point to perlmonks and those sites instead?

Yes, you should feed strength.

One question you might ask yourself is:

If my forums should be phased out because they are not as rigorous as perlmonks, Are my script archives as rigorous as CPAN?"

It wouldn't be hard to argue that your site fills a niche that CPAN does not. It is not for me to make the case.

If you are looking to increase quality, you might look at CPAN's methods...

Other questions you might ask yourself are:

“If I take the site down will people wind up writing their own even more insecure and crappy scripts?”

"Are insecure and crappy scripts better than no scripts at all?"

[Chrisf]

I agree that shutting down a site with that amount of traffic would be a shame. If you were to take it down, your visitors would end up finding another script archive site, and probably get lower quality scripts in the process (there's a chance they wouldn't even be perl!).

I think footpad's approach would be far better. Focus on some of the more popular scripts and try to improve them. You might want to consider talking to the Not Matt's Scripts people and see how they're doing. I'm sure you'll also be able to find some help with the articles from people around here.

Best of luck :)

[impossiblerobot]

For me personally, the Perl Archive was one of the better Perl resources I found before wondering into the Monastery. Although there is a lot of bad code on there, it doesn't seem to be all bad (as with some of the other "CGI resource" sites out there), and it is offset by those articles you mentioned, which I found very helpful.

Unfortunately, the bad code keeps pouring into the site, while the articles don't. :-)
But even if you can't devote enough time, etc., to make the site what you would like, my vote is that it's better to light a candle...

Impossible Robot