Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Status of Recent User Information Leak

by Co-Rion (Monk)
on Jul 30, 2009 at 21:01 UTC ( #784737=monkdiscuss: print w/ replies, xml ) Need Help??

What Happened

Some time on May 20, 2009, an unused (but still on line) perlmonks server was hacked, and its root password obtained by unknown individuals. The hacker(s) dumped contents from the perlmonks user database on that machine, data which is estimated to be current as of approximately September 2008.

The exploit was published in a hacker e-zine published on July 28, and was brought to the attention of PerlMonks administrators later that night.

The published material included the passwords, email addresses, and "real" names of all of the members of janitors and Saints in Our Book*. However, the hackers presumably obtained, and could distribute, the user info of all perlmonks users — at least those existing as of last September.

As far as is known, the main perlmonks servers have not been hacked.

* The list of Saints used appears by some indications to be a more recent one, from perhaps mid-April.

What Is Being Done In Response

Notifying Users

Alert reader OverlordQ brought the leak to the attention of PerlMonks administrators late in the evening of July 28.

At approximately 0130 UTC of July 29, an administrator of the perlmonks group on Facebook sent a broadcast message to all members of that group, notifying them of the event and advising them to change their PerlMonks passwords.

At about 1600 UTC of July 29, a notice was posted on the Monastery Gates.

At about 2100 UTC of July 29, PerlMonks administrators sent an email to the email addresses of record of the approximately 580 users whose user info was published, notifying them of the event and advising them to change their PerlMonks passwords.

Unfortunately, not all of the ~580 users whose passwords were published had working emails. In many cases, the gods have attempted to contact those individuals by alternate email addresses or other means. If you think you should have received such an email but have not, please check your spam folder for email from perlmonks@corion.net.

At some time prior to that, the gods changed the passwords of those users (out of the 580) who had not yet already changed them. (Noted by tye in Re: It's Time for Everyone to Change Passwords! (changed))

Lastly, this post is an official notification and status message to the members and visitors of the PerlMonks web site.

Any changes to the site as a consequence of this event will be announced in Tidings.

Closing the Hole

PerlMonks admins are working with the Pair.com folks (who manage our hardware and connectivity resources) to evaluate and strengthen security on the servers. No information is available at this time as to the status of this effort.

Strengthening Authentication

The administrators are planning to implement hashed passwords (allowing more than 8 chars).

What Should You Do?

If you have already changed your password, you are set (at least until the next time someone steals the info from our user database). If you have not, and you are one of the ~580 users whose user info was leaked, your password has been changed for you. Use What's my password? to request an e-mail containing your new, randomly generated password.

All PerlMonks registered users are strongly encouraged to have a current email address in their profile in case further administrative password resets are necessary. Emails can be set/changed by going to your homenode and clicking "Edit your Profile".

Caution: If you used your PerlMonks password on any other service (other sites, email, etc.), you should change those other passwords now — and for, FSM's sake, do NOT reuse passwords! Ever!

If you are unable to log in due to a lost/changed password and email isn't working, you may send a message to the gods via the form at Retrieving a forgotten username or password. Alternatively, you may contact the PerlMonks administrators via email, at perlmonks.org@gmail.com.

Many thanks must go out to jdporter who collected all this information and wrote it up in a presentable manner.

Co-Rion for the gods

PS - The perlmonks maintainers wish to extend a hearty high-five of gratitude to noble monk OverlordQ, who had the integrity and presence of mind to bring the security leak to our attention. Although, we do have to wonder what he was doing reading a hacker e-zine in the first place... ;-)

Comment on Status of Recent User Information Leak
Select or Download Code
Re: Status of Recent User Information Leak
by moritz (Cardinal) on Jul 30, 2009 at 21:59 UTC
    Thanks to all the gods for the hard to work to handle this breach as gracefully as possible, and for all the other helpful monks who helped other monks changing their passwords, pointing them to relevant threads etc.

    I know that you earned lots of criticism in the CB and in other discussions here for not handling this breach the way they wanted, or not using hashed passwords in the first place.

    With 50k accounts that's to be expected. There'll always be somebody who's unhappy with your reactions, and many diverging opinions. Still I think that you did an incredible job, especially if we consider that you're all just volunteers with day jobs and real lives.

    Update: And please take it with some sense of humor ;-)

      And I would like to add my thanks++ as well.

      There is a saying "sh*t happens".

      But your response is quite impressive and I hold a deep respect for all of the hard work you do in your "spare" time!

        Posting as anonymous because I can't log in right now.

        I'm furious this site used plaintext passwords. What a??holes even consider launching a basic site, let alone one used by professionals worldwide, storing passwords as plaintext?

        Sh*t happens, sure, but setting up the most loyal users to be f*cked is NOT cool.

        I sincerely doubt I'll use this kindergarten site again. It was NOT worth the sainthood.

      "Thanks to all the gods for the hard to work to handle this breach as gracefully as possible ..."

      Um. I do not share such sentiments. Perhaps I am wrong, but it seems to me that the "gods" knew about the password being stored in plain text for a long long time and did nothing to alert us or fix the problem.

      So, no. I don't thank them for this at all.

        Everyone who bothered to find out knew it was stored as plaintext, no claims were ever made to the contrary. Fixing this was in the TODO... I still thank them, they're volunteers

        Calling someone with a legitimate grievance a 'troll' simply because they make their point forcefully is simply inaccurate.

        I agree that this site is maintained by volunteers. I humbly thank you all for the years of effort that you have donated. I and everyone else here have enjoyed the free ride.

        I will point out that "free ride" means a very low expectation level.

        But are we not all software developers? Do we not practice what we preach? I do not expect us to provide the same level of service that a bank does - in a way I want MORE - since we are trying to set an example to follow.

        But again, 'volunteers' means that I do not get to expect that - as much as I would like it to be so.

        However - that the volunteers had the time to modify the voting and experience system but no time for security - is a damned shame.

        It is more embarrassing still when I read in TheRegister that maybe people will not trust perl as much because of this.

        That strikes me as a larger problem.

        So has anyone volunteered their time to work on security & fix the barn doors after the horses have eaten our children?



        Wait! This isn't a Parachute, this is a Backpack!
      FWIW, I'm reasonable sure my password stolen from here was just used to spam from my twitter account (iq tests and acai berry weightloss spam, in case anyone's interested.) big
      While I'm not thrilled to know passwords were stored as plain text, I beleive that this is quite excusable, and quite forgiven in my book. Good job of handling the issue, and thanks for you honesty and not pointing blame anywhere, but instead just working to solve the issue once and for all.
        Just because a good job was done of handling the issue does not equate to forgiveness in many of our books. No one is looking to punish anyone, so please stop being apologists and always remember that the volunteers chose to update the experience and voting system rather than protect the privacy of the users.
Re: Status of Recent User Information Leak
by Nkuvu (Priest) on Jul 30, 2009 at 23:20 UTC

    "Some time on May 20, 2009, an unused (but still on line) perlmonks server was hacked...
    The exploit was published in a hacker e-zine, and was brought to the attention of PerlMonks administrators later that night.
    "

    From the text here, it sounds like the exploit was known to the Perlmonks administrators for two full months before anything was done about it. Can someone explain to me why this security breach was wide open for two months without any notification for the people who may have been affected?

    Edit to add: The original node has been updated since I posted this, clarifying the point. Specifically, the second sentence I quoted now shows a date.

      It could be worded better but the intent is clear -- they found out about it shortly after the ezine was published (which was just recently). If that ezine link went to the actual ezine instead of a Wikipedia entry then we could see the publication date and Co-Rion's intent would be much clearer. Update: reworded previous sentence to be clearer.

      Elda Taluta; Sarks Sark; Ark Arks

      The gods found out about the leak sometime on 07/28/2009. The May 20 date would seem to be the result of forensic sleuthing by the gods/Pair.com. As far as anyone knows, the info was only leaked, by the hackers, as of approximately Jul 28, 2009 at 18:00 CDT (my best guess based on when certain links were posted on certain blogs - I don't feel the need to give any more direct pointers to the hacker 'zine).

      HTH,

      planetscape

        It's just the way the original node is worded -- apparently the gods were notified twice, two months apart. I was hoping that Co-Rion just mis-wrote the text and the gods were only notified once, on July 28th.

        Added: Note that I'm not looking for a link, and I agree that the fewer links to it, the better (no reason to make them all happy by giving them extra attention, not an attempt at "security through obscurity").

      Re: the hacker "article" see here: http://staynalive.com/articles/2009/07/30/theres-more-than-one-way-to-store-a-password-perlmonks-hacked/ They claim not to have any further malevolent purposes:
      There is a really simple reason we owned PerlMonks: we couldnít resist more than 50,000 unencrypted programmer passwords. Thatís right, unhashed. Just sitting in the database. From which they save convenient backups for us. In case you guys are worried, we did NOT backdoor dozens of your public Perl projects. Honest. Why would we want to do that?
Re: Status of Recent User Information Leak
by Zen (Deacon) on Jul 31, 2009 at 01:38 UTC
    I want to know why everyone who has had their passwords and email addresses taken are not being emailed, and only those who are facebook club or published did.

      Do you have effective bulk e-mailing services to offer? We are still having problems just getting the first batch of 700-odd e-mails actually successfully sent due to anti-spam measures that are nearly ubiquitous.

      Trying to figure out how to send 58,000 e-mails to people who, for the most part, haven't visited here in years is not my highest priority. There is plenty of other work to do in response to this incident, and most of it isn't happening as fast as I would like.

      If you feel strongly that this needs to be done, than please demonstrate your sincerity by composing a proposed e-mail body and finding an effective bulk e-mail delivery method that the vast majority of PerlMonks users would not mind having their e-mail address provided to. If you or somebody else provides those two items, then I will try to find a resource to select and extract the pertinent e-mail addresses.

      I personally believe that this incident is already very widely publicized and the number of people who would be reached by a mass e-mailing who are not already aware of the problem and who also left interesting personal information here that is still pertinent, would be vanishingly small. Never the less, I would like to e-mail everybody just in case there are some people in that category. So I would appreciate the help. But I will be spending the time that I already don't have on tasks related to this incident that I consider more important.

      Thank you.

      - tye        

        Tye,

        Does anyone here use Qmail on a linux server? If so, all it would take is to copy the 58,000 email addresses into a text file, one address per line, place it on the Qmail server as a mail list, and then bounce a generic email off the list. Everyone would receive the same message (not personalized) but it would be very efficient and only to them (like bcc:).

        I have my own box here at home, and I suppose if folks trusted me, I could send them out from here.

        Qmail allows mail lists for its user accounts (as I suppose other mail programs do as well). In Qmail, it is nearly as simple as this: adduser perlmonks, copy text file into perlmonks home directory as "newslist", then write an email to "perlmonks-newslist@qmailserver.domain.org" and watch the mail go out. (No guarantees that I'm not missing a step or two, but it's about that simple if qmail is already installed and running.)

        Features of Qmail, including the mail list (for which there is no size limit), can be found here: http://cr.yp.to/qmail.html. According to the statistics there, it might take under two hours to send the email out to all 58,000 addresses.

        Blessings,

        ~Polyglot~

        UPDATE: On second thought, I am remembering that I am not in a good position to send out bulk emails like this. Someone else would have to do it from a more trusted IP address. You see, I am in Taiwan, and many ISPs seem to block entire IP address ranges for Taiwan, as apparently much spam and mischief originates here. In other words, much of what I would send from here might not be delivered, or it would land in the "spam" box. But the Qmail solution would still be viable if used from a trusted source. ~ Polyglot ~

        I personally believe that this incident is already very widely publicized....
        This is precisely why I asked about the legal ramifications in 784719. Specifically, are there any applicable state or federal laws that require notification? Establish what you have to do (or what your lawyers advise you to do) first before worrying about what you would like to do.

        Update: And if possible, after things have settled down let us know what the response was. This is a great opportunity to learn more about the legal side of security breaches like this, especially for open source foundations, organizations, etc.

        Elda Taluta; Sarks Sark; Ark Arks

        http://www.expeditesimplicity.com/features.php

        $60 or use any of a number of linux freeware tools to send mail in phased batches, or from the pair server itself (notify them first).

        Body:

        Your Perlmonks account has been compromised! Your password, email address, and any information stored about you on our site was unencrypted and thus visible to the attackers. We have more information at the following link: http://www.perlmonks.org/?parent=784806

        We encourage you to change your password and visit our site for more information as it becomes available. This will be the last notification on the topic.

        ----------------------------------

        How's that? I may have done the link wrong. Updated link.
        How's that mailing going?
Re: Status of Recent User Information Leak (jdporter++)
by tye (Cardinal) on Jul 31, 2009 at 05:25 UTC

    I wanted to take a quick moment to offer thanks and public praise to jdporter who has done a great deal of work in response to this incident. I believe jdporter is the person who has done the most among those who do not have access to do much of the required work. (Though, I apologize if I missed substantial work done by others, as I surely have since I haven't had time to even read much of the discussions that have been spawned.)

    jdporter went to great lengths to contact as many PerlMonks as he could and reached out to me via quite a few routes, some surely requiring some research. Unfortunately, the timing was such that I didn't notice his attempts until after I caught the tail end of some aftermath to OverlordQ speaking as vroom and as me in the chatterbox. Also thanks to bobf for being the first to successfully communicate the situation to me in a manner that I was able to clearly understand.

    Thanks to many others who have expressed their support and to many more who have simply demonstrated patience, calm, and/or clear-headedness in the face of this crisis.

    Finally, I would like to apologize, again. In particular, for my part in not re-implementing enough of the password system at PerlMonks. There are quite a lot of improvements that I've long wanted to get to related to passwords at PerlMonks. Hashed passwords was certainly one of them. Quite a few of the published passwords (and tons of others) would have surely been quickly found even if they had been hashed due to well-established dictionary attacks. But the plain text password storage was a stated motivation of the attack.

    - tye        

      When you hash 'em, hash 'em well. With a grain (or a hundred odd bits) of salt. And preferably with a suitably expensive KDF based on a hash that's not known to be totally hosed. glibc 2.7+ crypt() with the $5$ method should be reasonably strong; Crypt::SaltedHash with SHA-256 is less strong, but the best thing I can think of that's reasonably portable.
      I imagine this been a huge time sink and very stressful. And if Don't Panic! briefly came to mind I'm sure it was quickly discarded. I can only imagine what it's like to be in those shoes and I appreciate the effort that has gone into handling this situation. Thanks jdporter and the rest and I hope we can get a more complete list later on so that we can properly thank everyone!

      Elda Taluta; Sarks Sark; Ark Arks

      Finally, I would like to apologize, again. In particular, for my part in not re-implementing enough of the password system at PerlMonks. There are quite a lot of improvements that I've long wanted to get to related to passwords at PerlMonks. Hashed passwords was certainly one of them.
      Thanks, I'm glad to hear the acknowledgement, and the apology -- but it should be part of the above summary, not buried in the comments. And while I personally agree that one should never re-use passwords, the admonishing tone about that advice is a bit much... you're pointing the finger in the wrong direction.

        you're pointing the finger in the wrong direction.
        Not to worry, because every time you are pointing a finger at someone else you are pointing three back at yourself!!! (^_^) [Think about how you fold the non-pointer fingers in.] Of course after a co-worker used that on me back at my first job I made sure to always point all of my fingers, toes/feet, and even my nose directly at him!! (^_^;)

        Elda Taluta; Sarks Sark; Ark Arks

Re: Status of Recent User Information Leak
by SFLEX (Chaplain) on Jul 31, 2009 at 09:59 UTC
    Don't we know better as developers to not store raw passwords in databases?
    I have been told many time that The "everything" web portal is the most secure, event though this current hack was from root and the web portal would have no security to combat against that. Except for encrypting the raw passwords in that database so they cant be read.
    I remember back this has happened before and still there has not been a change to secure the passwords.
    Why hasn't that been done?

    Information is knowledge.
Opportunity to excel
by pemungkah (Priest) on Jul 31, 2009 at 20:57 UTC
    http://www.securityfocus.com/blogs/262

    Highly-recommended article on making password attacks unprofitable, especially "rainbow table" attacks.

    Summary: make computing the hash slow. Really slow. Like 2 or 3 seconds, or more. Provos-Maziereís Bcrypt scheme lets you make it as slow as you want. This lets you make computing the rainbow table so prohibitively slow that breaking in by password guessing is unprofitable.

    (Rainbow tables are huge tables of hashes with the corresponding text that made them. Look up a hash; if you found it, you win and can get in. So "dumb" passwords are almost certainly in a big rainbow table. But! If you use a really slow hashing algorithm, it's too unprofitable to compute more than a "few". If you added on the analysis from http://www.infoworld.com/d/security-central/myspace-password-exploit-crunching-numbers-and-letters-983 and outright disallowed the most common "dumb" passwords, you'd be way ahead.)

    There's definitely an opportunity to step on this business of "wow, those Perl guys are so clueless they stored their passwords in plaintext" hard. If I can help out, let me know.

      I would truly appreciate any of the code that might be developed being made public; along with the process of developing it (stages, discussion, critiques). I'm okay on security but I'd like to be a rock and code/techniques for Bcrypt and salting hashes are two things that would be edifying for JAPHs everywhere.

      I don't think rainbow tables are big tables of hashes. ... they seem to have more to do with hash chains than big tables of precomputed hashes. I don't really understand it myself, but I'm pretty sure I have that much right.

      oechslin

      -Paul

      Making the hash algorithm really slow makes the hash unusable in most practical situations. If you need two seconds on a fast machine for the hash to compute, login to a website would make the server unresponsive for two seconds. Everyone could do a denial-of-service attack to that website by simply doing one (failed) login attempt per second!

      Also rainbow tables will find good passwords with the same chance as dumb passwords. Rainbow tables are precomputed lists. For every entry you hash a random password, then hash the resulting hash, and so on for (lets say) 100000 times. Store the last hash as key and the starting password as data in your database. Now do this precomputation for a year or so

      If you now want to crack a hash, just hash it, then hash the resulting hash and so on until you find a hash that corresponds to a key in your database. If you find one after rehashing less than 100000 times you have a winner. Now you only need to use the starting password of that database entry and do the rehash cycle from there again until you find the value in the chain just before the hash you got. That is your password (in reality always a collision with the original password).

      So rainbow tables in fact allow a costly brute force search, because you need to do it only once for a specific hash algorithm and can do it in preparation for the actual cracking.

      The often mentioned seeding does not prevent rainbow tables but makes them expensive again because you need a separate rainbow table for each individual seed value.

        The often mentioned seeding does not prevent rainbow tables but makes them expensive again because you need a separate rainbow table for each individual seed value.

        Hash the userid and password together.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Status of Recent User Information Leak
by spectre9 (Beadle) on Jul 31, 2009 at 21:17 UTC
Re: Status of Recent User Information Leak
by chazzz (Pilgrim) on Aug 01, 2009 at 22:36 UTC
    It would be great if perlmonks supported OpenID login then I would not need to use a password at all! Passwords are /so/ 20th century. Supporting OpenID would allow users to authenticate with any method supported by their provider, including X.509, PGP and RSA.
Re: Status of Recent User Information Leak
by Anonymous Monk on Aug 01, 2009 at 23:57 UTC
    I really think the plain-text thing is the elephant in the room that PerlMonks admins aren't talking about.

    I would really appreciate a statement from someone explaining why they were stored in plain text.

    The whole thing is roughly as ironic as when Amazon removed "1984" from Kindles -- this is a place of wisdom, of Best Practice, of knowledge and years of study, and as everyone else is saying, we always tell people to encrypt passwords. Not doing so is a rookie mistake.

    The fact that PerlMonks didn't follow its own wise advice makes the whole site a laughing-stock, and Perl by association.

      Maybe you should ask the creators of Everything Development Engine, chromatic, jaybonci, n_oostendorp, paul_the_nomad, tvroom? Count them, they are only 6.

      The fact that PerlMonks didn't follow its own wise advice makes the whole site a laughing-stock, and Perl by association.

      So you want I should invoke Godwin now? perl -e " die if 6 x 3 eq 666 "

      This only approaches the root of the problem in my mind.

      This place was the core of the Perl community. If the community core could not mold Perl into the best choice for web development than I have to decide that no one can. The technology may be suited to the task, but there is no guidance on how to get there. As someone who regularly recommends technology my faith in the the Perl community has been more than shattered. A lot of smart people in one room who still can't make it work. In short, it makes Perl seem like a fruitless endeavor.

      I also feel that the response to this incident was very poor. A placeholder for perlmonks.org notifying users of the situation would have been appropriate, within 10 minutes of realisation of the hack. All user data should have been locked down immediately, and announced as such until further notice - futher notice being when everything was guaranteed secure and the hack completely understood. Instead, for some bizarre reason the site continued to operate with clear confusion and indecision dominating the chatterbox for hours. Stern advice to immediately change passwords despite persisting ignorance surrounding the circumstances was paramount idiocy. Why give away one perfectly good password, when you could give away two?

      To add insult to injury, after a painfully long waiting period of inaction, some monk with appropriate access decided that a leet-speak banner on the front page would be the best way to announce the site being compromised, followed by a joking reference to The Hitchhikers GTTG. Really? I failed to see the humour in it and found it to be one of the most publicly unprofessional acts I have ever witnessed.

      To be sure, I laughed all day long as I removed Perlmonks from all my browsers bookmarks on all my computers. I continued laughing as I re-visited every site I've been to since 2002 checking to see if the password I used there was the same as perlmonks. Then I laughed as I checked all my personal accounts, my servers, and any other place I use a password to make sure it wasn't that one I favored long ago when I setup my perlmonks account. It was hilarious. Wasting the time of 50,000 people in addition to compromising their personal details is FUNNY.

      I commend the few monks with any amount of public facing professionalism, such as tye. Conversely I vehemently censure the others who lean on the crutch of mediocrity waving slogans like, "it's a forum, not a bank", and "no site is secure". Before you posture as The Oracle, you need to be the oracle.

      So now, to me, there is no trusted community core for Perl. That is the elephant in my room.
        I also feel that the response to this incident was very poor. A placeholder for perlmonks.org notifying users of the situation would have been appropriate, within 10 minutes of realisation of the hack.
        ...I vehemently censure the others who lean on the crutch of mediocrity waving slogans like, "it's a forum, not a bank"...
        It's a forum, not a bank. I continue to not see what would possibly have been gained to make it worth shutting down legitimate access to the website.
        There seem to be an awful lot of overreactions going on here. Breakins happen from time to time. Storing the passwords cleartext is embarassing, sure, but it was probably considered handy for mailing passwords to people back in 1996 or whatever.

        Also, hashing the passwords does not make them that much safer. Are you talking md5/sha1 hmac stuff like the Linux shadow files? Well, a few hours with john will get you a huge majority of the passwords I imagine, even with salts. And for the patient (or the botnet operator), even the really good ones will be discovered in relatively short order.

        Pfft, I say. This is why you should use a randomly generated unique password on each site.

        It doesn't really have anything to do with Perl or the Perl community either. I imagine the everything 2 engine has crypted passwords -- I don't really know that, I just imagine. Probably this was a bad design decision unique to this particular e2 site.

        I'd guess more forum sites store passwords cleartext than don't though, doesn't really matter what language. It was really common to send your clear text password over cleartext email when you clicked "forgot password." A lot of sites changed this behavior, for good reasons, but a lot didn't. It's historical, not a Perl-the-language problem.

        Basically, people were just too lazy to change it, because that's how it's always been.

        -Paul

        This place still is the core of the Perl community and therein lies your anger. You expected more. You got less.

        Your anger at wasted time is legitimate. But the blame is not on "the gods", but on all of us - me included, you included. Perhaps you may not have noticed but the number of monks actively involved in patching and enhancing this site can be counted on less than one hand. This is a large complex site with +80 database tables, 100 different node types, and 50-100K* lines of code - managed day to day by less than 5 people. With so few volunteers managing so many resources, it is almost guaranteed that even very important things will fall through the cracks or get pushed back onto the round tuit table.

        Did you volunteer for pmdev? If you did, when did you last contribute? If not, did you seek out ways to help developers better prioritize an ever growing to-do list? Did you donate money to hire a paid full-time developer? Did you volunteer when we discussed how to better manage enhancement requests? Did you take up mr_mischief on his request for information on how other on-line volunteer organizations manage volunteers? Or tye on his request to improve the patch application system? Add an insight to the discussion about the site improvement process? Volunteer to help coallate the numerous suggestions for improvement spread across tens, maybe hundreds of nodes?

        Nothing in life is free. We can each pay money for a site that is managed by paid staff. We can volunteer our time and expertise to "pay" for a site that is free. But if we do neither and then the site fails our expectations, we have only ourselves to blame.

        Best, beth

        * very approximate "guestimate" based on number of nodes containing operational code * 100 lines avg.
        Who are you Mr. Professional?
        To add insult to injury, after a painfully long waiting period of inaction, some monk with appropriate access decided that a leet-speak banner on the front page would be the best way to announce the site being compromised, followed by a joking reference to The Hitchhikers GTTG. Really? I failed to see the humour in it and found it to be one of the most publicly unprofessional acts I have ever witnessed.

        This reminds me of The Parable of the Old Man, The Boy, and The Donkey. Thus far, site admins have been vilified for notifying all users, not notifying any users, notifying only FaceBook users, notifying only users whose e-mails and passwords had been leaked, and on and on. Frankly, I am surprised we still have admins to bitch about and a site allowing us to bitch, given the "damned if you do, damned if you don't" mentality of so many. Fortunately, even more seem to ask what they can do to help rather than stand back and pour more kerosene on the fire.

        HTH,

        planetscape

        Stern advice to immediately change passwords despite persisting ignorance surrounding the circumstances was paramount idiocy. Why give away one perfectly good password, when you could give away two?

        Oh right, passwords are really so expensive that losing two would really put a dent into your pension plan. Where do you buy your passwords?

        But seriously, the reason to change the passwords was to prevent copycats from using the (freshly) published passwords. Sounds sensible to me

        I can understand your anger, but please direct some of that anger to your own "unprofessionality" (if I may reuse your words) to use web passwords also on your personal accounts or your servers. Are you sure your password would have been safe from a dictionary attack? If not, all that checking and changing would have been necessary even if the passwords had been hashed.

Re: Status of Recent User Information Leak
by Trimbach (Curate) on Aug 02, 2009 at 22:00 UTC

    I just wanted to throw out a big thank-you to the PM Gods for the time and effort they've spent responding to this situation. Maintaining the site is a thankless job on a good day, and the last few days have been more thankless than most. Just trying to shift the karma the other way a little bit.

    I have donated $$$ to PM several times in the past, and now is a good time to do so again. This is a resource that I value and I encourage everyone to look past this unfortunate incident and look towards the future.

    As for the whole "hashed/unhashed" debate this doesn't bother me at all. "Convenience" and "Security" are at opposite ends of a continuum and PM is not a bank. I do not blame the Devs for choosing convenience over security... anyone who did not reuse their PM password has lost exactly nothing in this incident, and anyone who did reuse their PM password on another site deserves what they get. (Anyone who used their PM password on something that matters, like a bank account, a root server, or a database, should be publicly humiliated with extreme prejudice.) This is not/should not be a big deal.

    Thanks again for your time and professionalism.

    Gary Blackburn
    Trained Killer

      When developers and designers continue to ignore how people actually behave then said developers and designers are the ones at fault. Studies have shown over and over that people write complicated passwords down, reuse passwords, etc. What we really need is a decent and inexpensive two-factor auth solution.

      And if you want to play the "professional" card then you might want to avoid saying things like "[certain people] should be publicly humiliated with extreme prejudice".

      Elda Taluta; Sarks Sark; Ark Arks

        When developers and designers continue to ignore how people actually behave then said developers and designers are the ones at fault. Studies have shown over and over that people write complicated passwords down, reuse passwords, etc.

        Yes, people do dumb things. And they use their birth date for their ATM pin. The natural (and even universal) tendency to do dumb things doesn't absolve users from taking responsibility for their actions.

        What we really need is a decent and inexpensive two-factor auth solution.

        Sure. And maybe (maybe) we'll get one of those someday, but until then the game is all about risk mitigation. The risk for me for a security breach at PM is zero. So therefore I don't care what PM does or does not do to secure my information. YMMV.

        And if you want to play the "professional" card then you might want to avoid saying things like "certain people should be publicly humiliated with extreme prejudice".

        No, if I wanted to play the "professional" card I'd use much harsher terms, like "fired." Any professional, who has been trained in IT security procedures, and who is fully aware of the risks and hazards of password security, who nevertheless uses the same same password on PM that they use on a server or a bank account deserves much more punishment than mere humiliation.

        Gary Blackburn
        Trained Killer

Re: Status of Recent User Information Leak
by Polyglot (Monk) on Aug 03, 2009 at 05:49 UTC
    I realize many have already given their two cents here. As my two cents are of a slightly different shade, I will add them to the pile.

    1. I used a random password on PerlMonks.
    2. I did not use this password anywhere else.
    3. The password had upper/lower case and digits.
    4. The password was as strong as any password could have been on PerlMonks.
    And since I didn't use the password anywhere else, you might think I had nothing to lose, right? Wrong. Some people here have focused on the password as the only critical piece. But along with the password, the crackers also got our email addresses and potentially our real names (in my case I have not supplied a real name).

    What advice do the "professionals" have regarding email addresses? Should we apply for a different email account for every webservice we use as well? Spam is a big problem these days, as is identity theft. If we were to use a fake email address, we would be unable to sign up on Perl Monks. So, I did have something to lose after all.

    It is also my understanding that it was not the user passwords that were cracked, but the server root itself. Is there any way our data could have been protected from a root-level attack? I doubt it.

    I begin to wonder if the real security problem here had little to do with passwords, and everything to do with general server security procedures. On my linux server, I use a firewall, I ban for twenty minutes any user who fails thrice to correctly enter a password, and use private/public keys with SSH on a non-standard port which will not allow anyone to login as root. Logging in as root requires a separate step. The database is password protected with a separate password, and I do not keep dumps of the DB's user table. And I do not think my server is especially secure. There are many more steps one might take. But it sounds like from the way PM was cracked, it was almost a giveaway.

    If you want to discuss having more secure passwords here, then can we talk about having more than 8 characters in our passwords? But a chain is only as strong as its weakest link, and it seems that even the weakest of passwords belonging to users here may not have been the weak link in this case.

    Blessings,

    ~Polyglot~

      I'm sorry, but that is really incoherent (well, everything after the straw man you might think I had nothing to lose, right? ).
        "I'm sorry, but that is really incoherent"

        How do you get away with writing code then?

Re: Status of Recent User Information Leak
by SilasTheMonk (Chaplain) on Aug 03, 2009 at 06:38 UTC
    I am not amongst those most likely to be affected however I thought I better change my password anyway. I notice that the password has a maximum length of 8 which is annoying. Also I have tried to log onto the PAUSE website but I get a message:
    pause.perl.org uses an invalid security certificate.
    
    The certificate is not trusted because the issuer certificate is not trusted.
    
    (Error code: sec_error_untrusted_issuer)
    
    
        This is silly, because it's quite easy to get a valid certificate for free or very cheap.
Re: Status of Recent User Information Leak
by inman (Curate) on Aug 03, 2009 at 11:29 UTC
    It looks as though the Gods have been getting some rough treatment by some monks. OK so storing the passwords as plaintext is embarrassing and it is fundamentally disappointing that this whole affair took place. However, I got an e-mail this morning informing me of the problem and then found that I had to reset the account password since it had been changed for me. This is a reasonably pro-active response and the gods should claw back some credit for that.

    On a separate note - the job of changing passwords on other sites was easy because I use PasswordSafe. This allows me to create random passwords for each new site. The ones that had my old password in were really old accounts.

    Ever noticed how difficult it is to delete an account?

      Everybody at Perl Monks make you feel not-alone or left-out in front of an angry boss, a lazy soul that whispers at pulling you offtrack from learning Perl consistently or any monstrous authority for that matter, a great win-win to everyone PM is. Instead of criticizing the admins and debating whether they have neglected a security aspect we should be glad that they could bear the hassle of running this free site and all the bedeviling shots that come along, without PM all of us would be punching in the dark sometime somewhere to get a certain code working or would be going through noncontiguous Google's results in order to extract some knowledge which is a frustration. Risk arises if the displayed passwords were shared for different user-accounts, but is this not against the very strict rule that says "each account somebody has should have it's own independent password" too?, so maybe all of us share the blame if we were among those who pooled their passwords. As you said, crediting the Admins, the monks and the users in here for this site is worth it,supporting them is worth it and looking up to them is worth it as well..
      Excellence is an Endeavor of Persistence. Chance Favors a Prepared Mind
        Instead of criticizing the admins and debating whether they have neglected a security aspect we should be glad that they could bear the hassle of running this free site and all the bedeviling shots that come along, without PM all of us would be punching in the dark sometime somewhere to get a certain code working

        There is no question that the site bears some useful purpose. But such a fundamental mistake as to directly expose plain-text passwords absolutely deserves strong criticism.

        Only a few months ago I stood my ground (through a protracted potentially career-limiting argument) with a project manager about salting passwords used for a website I was involved with the creation of. He didn't get it because he wasn't technical. The Admins of this site, however, should have "got it".

        I can no longer post to this site using my original user name as my e-mail address has been published and could well be used to infer projects I am involved with. However that's inconvenient, not unforgivable, as hacks happen. Having to change other sites' passwords as a result of plain text leaks, however, is intolerable.

        each account somebody has should have it's own independent password

        This is an ideal. Personally I juggle about 10 different passwords in my head. A unique one for EVERY site that needs access? Impossible. A password manager? Not on every access point I use.

        I was using a password on this site I share with other non-critical systems so there was no risk of any commercial system being accessed using the exposed password. However I did have to go and change my generic password on the other websites on which I use it.

        The problem with computer development is that it is a small part science and a large part art. A large part of it is managing risk. How much risk do you take using 1 password vs convenience? How about 5 passwords vs convenience. How about 250 (pretty inconvenient for me, I can't even name every bone in the body.. might work for surgeons though..)?

        Now trade the effort required to salt and store passwords. Hmm, about 3 minutes using the crypt() function. How much risk is alleviated doing this? More than enough to mandate it for any web project.

        Instead of criticizing the admins and debating whether they have neglected a security aspect we should be glad that they could bear the hassle of running this free site and all the bedeviling shots that come along, without PM all of us would be punching in the dark sometime somewhere to get a certain code working or would be going through noncontiguous Google's results in order to extract some knowledge which is a frustration.
        How many users does it take to make the programmers of a "free software" bear some responsibility for its integrity?

        Consider the following "free" things, and whether you would be a bit indisposed if any one of them had a similar leak of your email address, password, and real name due to a server that wasn't secure.

        • linux
        • Firefox
        • Skype
        • Gmail
        • Hotmail
        • Yahoo Mail
        • MSN Messenger
        • Facebook
        Businesses have come to rely on these products. Should they not? Should everyone be content for the free service these products give, even if now and then their security is breached and passwords, emails, and IDs become public info?

        Blessings,

        ~Polyglot~

Re: Status of Recent User Information Leak
by Xenofur (Monk) on Aug 03, 2009 at 21:40 UTC
    I'm sure there is an answer this somewhere, but after reasonably extensive search i could not find it. As such I'd like to ask here and in relevance to the current events: Is the source code of perlmonks openly and without restrictions available for scrutiny in any place?

    If so, where? If not, why?

    Thanks in advance to whoever takes the time to type out an answer to this. :)
      ...after reasonably extensive search ...

      Something is wrong with your searching abilities if you didn't find this in the PerlMonks FAQ: Can I get the PerlMonks source code?

      Between the mind which plans and the hands which build, there must be a mediator... and this mediator must be the heart.
        ... if you didn't find this in the PerlMonks FAQ: ...

        I think this might be in the same category as forgetting to look in the closet for the coat you wore last night. :-)

        Best, beth

        I looked through that page and expected it to be under "About PerlMonks", which seemed to cover such general topics. When i looked at "Advanced PerlMonks Topics" i pretty much only read the first five titles and then assumed that the rest of it also was "How to do stuff on PM". Not actually anything about PM.

        Since you took no care to be diplomatic or polite, I'll be blunt as well and say straight out that the FAQ is, at least in that respect, badly structured.
      If so, where? If not, why?

      At present not even members of pmdev can get a download of the site and source code to install on their own machines to create a test environment.

      It does indeed seem odd that a site that is part of the open-source community does not itself provide open access to its own site's source code. Much of the problem lies in the way the site is structured. The code used by the site is split between a database and source code files. Bundling up the source code files into a tarball probably wouldn't be so difficult. Extracting the relevant information from the database is another story.

      The code stored in the database is spread across a plethora of database tables. We can't just dump the tables because some of these tables contain many different types of records: some store site content, some store code to layout and "skin" the site, and some contain integrity constraints and processing code for the various pages you see day to day at Perl Monks. For example, almost everything stored in the database has an entry in the "node" table.

      To tease apart the material relating to site structure, process, and look and feel from the actual site content may require a certain amount of hand tagging. Finally a script would have to be written to gather together all the relevant records from each database table. Writing a script wouldn't be such a big deal if it weren't for the fact that the site documentation is spread out all over the place. Simply knowing what is used where is a challenge. It would be all to easy to leave a crucial bit out without good documentation. We are working on trying to improve the documentation situation, but it will take time.

      Even after that script was written we would have another problem to deal with before we could publish the site's nodes: the database portion would (should?) be reviewed for any material that might be a security issue. These nodes would need to be refactored into separate components, one that could be published and one that should not be. There are somewhere between 600 and 1000 units to review.

      I am not saying any of this as an excuse, merely as an explanation. I think the process of creating such an export would be very good practice for us. It would encourage good documentation and confirm the truth of what we had documented so far. It would give us a focused reason to keep publishable and site specific security code separate. That separation would make the site more secure. It would also make it easier to integrate new pmdevs. It would enable much faster implementation of bug fixes and site enhancement suggestions since PM devs could experiment and test their work more freely.

      Best, beth

        This is an excellent write-up explaining the situation and i think it should in fact replace the "Can I get the PerlMonks source code?" node in its entirety, not only because its content is much more clear, but also because it approaches the question with a lot more respect.

        Something I'm curious about though is: How is the social situation regarding this? Putting the technical reasons aside, is there resistance from the gods or pmdevs towards moving the code into a direction that would make opening it more easy? Or would they welcome such efforts?

        I'm sure there is a number of people (including me) that would happily contribute to such a movement. However with the current communication regarding the issue it seems to me as if such efforts would not be welcomed and only be a waste of time. (Leaving aside the whole issue of being unable to look at the source and ascertain whether my skill level is up to it before committing to a join request and wasting someone else's as well as my time in going through the motions there.)
Re: Status of Recent User Information Leak
by boblawblah (Scribe) on Aug 04, 2009 at 06:56 UTC
    Shame on you. Volunteers or not - what a major betrayal of trust. Restore it.
      What were you hoping to hear, I quit?

        [From someone in the Saints list, who'd rather leave this Anonymously]

        How about some honesty? How about real honesty instead of the defensive egotistical self-centered crap that the "gods" have been posting about this?

        "We're sorry, we really screwed up."

        "We aren't competent enough to fix it quickly."

        "The PerlMonks code is so bad, even competent programmers can't fix it quickly."

        "The PerlMonks code is such crap, we can't even give it away and ask for help."

        "Many eyes will never make short work of PerlMonks bugs."

        "We laugh at places like Twitter for being insecure, but we're leaving a badly hacked, insecure system up anyway."

        "We give out lots of coding and security advice, but haven't really been listening to any."

        "We've really poorly represented the Perl community, and we're sorry."

Re: Status of Recent User Information Leak
by Anonymous Monk on Aug 04, 2009 at 07:57 UTC
    Just the sheer unperlish'ness of the act of having unencrypted passwords is so ugly. I fell like having been in bed with someone who has HIV, honestly.
    Just to be one of the 50k club dropping his mind here: Once I've been in the place to decide: plain pw's (5 minutes more of perl code) versus hashed ones. My immediate reaction was: GOSH, plain passwords suck - i'll never ever do that, and didn't. It was just against my believe of beeing cool.
    Frankly, someone here was a devil in disguise. He should be spotted and sent not to a security but "fundamental attitudes" seminar payed from the perl foundation.
Re: Status of Recent User Information Leak
by OverlordQ (Hermit) on Aug 04, 2009 at 08:31 UTC

    Can I plead the fifth?

    Honestly, looking back on the timing, if life was a network I think I was only a couple of hops from the source.

Re: Status of Recent User Information Leak
by Zen (Deacon) on Aug 04, 2009 at 18:03 UTC
    http://en.wikipedia.org/wiki/Victim_blaming
Re: Status of Recent User Information Leak
by ology (Initiate) on Aug 04, 2009 at 19:44 UTC
    hmm

      "This statement is further proof that this site is maintained by dorks"

      As you have an ology I would have expected better from you.

      Further no one is forcing you to frequent the Monastery.

      Update

      Better link

        hmmm
Re: Status of Recent User Information Leak
by Anonymous Monk on Aug 05, 2009 at 20:35 UTC
    It's seems that everyone is bragging about plaintext passwords, at the same time forgetting about compromised root account on the server involved. Also, about 90% of the posters are happy to declare perlmonks maintainers as stupid and careless because of that. So, I have one question: give me one good logical reason to have encrypted passwords on the user login machine I and not you have the root access for.
      1) Sh*t happens.
      On a technical level everything you say is correct. However, you have overlooked the "temptation" factor -- the hackers specifically stated that they "couldn't resist so many clear text passwords" (paraphrased).

      Elda Taluta; Sarks Sark; Ark Arks

Re: Status of Recent User Information Leak
by scorpio17 (Monsignor) on Aug 11, 2009 at 14:29 UTC

    I've come to realize the true damage of this security breach was not caused by leaking passwords - those can easily be changed - but by leaking emails. Since the leak I have been subject to a steadily increasing amount of spam email. Changing an email that you've had a long time is much more difficult and inconvenient. But the spam is slowly rendering it useless. Bummer!

      Suggestion: Use sneakemail.com or similar redirector addresses when signing up. If the address you give starts garnering spam, discard it and generate another one. Your real email remains intact and spam free.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The answer to spam (as long as spam exists) is not to run from it, but to put it in its place. Email addresses are nearly useless if they're not public. But you need a good spam filter. This is one of the reasons why I'm still using a Yahoo email address after well over a decade. Its spam filtering is quite good. I have no qualms about using my Yahoo address for all my ordinary public needs. (yes, I also have an account on a different email server which I only give to my most trusted contacts.) It is quite liberating to not have to worry about whose hands your email address will fall into.

        Email addresses are nearly useless if they're not public.
        That I disagree with. Email addresses are nearly useless if noone knows them, but addresses known to a proper subset of the world population can be quite handy. I've various public email addresses. I've also tons of email addresses only a few number of people know. I even have email addresses only one person beside myself knows. Except for some of the public addresses (which only get spam), all of them are useful.

        Email addresses aren't that different from phone numbers. Just because a number is unlisted doesn't mean it carries no use.

Re: Status of Recent User Information Leak
by baddy (Initiate) on Aug 13, 2009 at 00:15 UTC
    Thanks for posting this, luckily I was using a unique password for the site. I must admit, of all the sites I am on I figured this one would have been less likely to have a problem like this since it is about programming, shows you can never be to safe though.
Re: Status of Recent User Information Leak
by punch_card_don (Curate) on Aug 30, 2009 at 11:22 UTC
    Just discovered this after a busy summer away from programming.

    This sort of thing is why I never, ever, provide true information to anyone I don't absolutely have to.

    A throw-away Hotmail email, a unique username, no personal info in the profile, a false "real name", and a unique password. The worst anyone can do with the info is make rude posts to PM, which IPs will prove weren't me.

    Once in a while someone will get on his high horse about the paranoia of anonymity - I have conclusive confirmation that my dismissal of them is warranted.




    Time flies like an arrow. Fruit flies like a banana.
Re: Status of Recent User Information Leak
by samwyse (Scribe) on Aug 31, 2009 at 22:53 UTC
    I, too, just arrived back after some time elsewhere. Shit happens, I suppose. I do use my password elsewhere, but it's my low security one, used only for programming forums and the like. Fortunately, I keep unique passwords for any account that handles financial information (banks, 401-K, etc) or electronic communications (Gmail, Twitter, etc.)
Re: Status of Recent User Information Leak
by zebroz (Initiate) on Sep 09, 2009 at 17:13 UTC
    fudge. My password from my college account never got changed... time to change a bunch of passwords.
Re: Status of Recent User Information Leak
by Anonymous Monk on Sep 19, 2009 at 14:48 UTC
    Almost two months later... no update in Tidings. No update here.
Re: Status of Recent User Information Leak
by Ratazong (Prior) on Sep 21, 2009 at 13:04 UTC
    This node is still linked from the monastery gates - which is fine.
    The link there is titled with Users, please read the following important update:. However the last update seems to be from July 30th.
    Maybe someone can add the date of the last update to that link? Then we can easily see if we already read the update or if there is something new...

    just an idea by Rata
      Unfortunately I don't think there have been any major updates to the root node (certainly not any recent ones). Which sadly makes 786810 look more and more relevant/accurate. Please, can we get some status updates!?!

      Elda Taluta; Sarks Sark; Ark Arks

OpenID Now
by brycen (Monk) on Oct 07, 2009 at 16:46 UTC

    This is EXACTLY the place for and reason for OpenID. With openID your less trusted site (e.g. perlmonks) never gets your true password. And yes, there is a perl module to support OpenID. Ooops, several perl modules.

    -Bryce

      I was surprised that this thread receives so much attention, considering when I go to my profile I see
      "Change password: Note: Eight (8) characters max!".

      I thought it was strange that when you perform a text search for 'password policy', but this is all you find on it?

      For example, considering the enormity of this thread, why not further enhance the security posture, perhaps with something like this regular expression, extended password policy, which might fit into the modular user authentication system?

      I certainly appreciate these proactive security breach notifications, but I vote to please enable unimpeded view of the monastery gates without this diabolical banner, innocence is bliss. Thanks!
Re: Status of Recent User Information Leak
by Fedot (Initiate) on Jan 13, 2010 at 19:49 UTC
    Despite this experience, registration page still asks new user to give 'real name'. This can be dangerously misleading for some newcomers.

      I accidentally swapped my real name with my user name.

      How can I remove my real name, and user jcrush only.

      Thanks for all the help,

      jcrush
Re: Status of Recent User Information Leak
by AI Cowboy (Sexton) on Jan 19, 2010 at 03:00 UTC
    So, any news on if this has been fixed or when it will be fixed? So far I've seen nothing about it, but it may just be that my senses have shut down from sleep deprivation, and they already fixed it.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: monkdiscuss [id://784737]
Approved by ww
Front-paged by virtualsue
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2014-08-30 08:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (291 votes), past polls