Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

What happened?

by jrsimmon (Hermit)
on Jul 29, 2009 at 04:16 UTC ( #784123=monkdiscuss: print w/replies, xml ) Need Help??

Subject: user database compromised
"all perlmonks users are advised to change their passwords immediately."
Once things have settled down a bit, I would like to have a bit more of an explanation as to what happened, how, etc.

Removed the facebook link that was confusing some people

Update 2:
Please see node Status of Recent User Information Leak for the official status/response from the gods.

Replies are listed 'Best First'.
Re: What happened?
by mzedeler (Pilgrim) on Jul 29, 2009 at 08:28 UTC

    I didn't get this message and found that its because I'm not a user with 3000+ xp. But that just made me even more worried - does the code really store passwords in different places and with different encoding schemes depending on the user status?

    Also, what steps are the janitors taking to restore in order to ensure that the hackers doesn't have access any longer?

      What really worries me is that the attackers claim that the passwords were stored UNENCRYPTED. We tell each and every wannabe-coder to salt and encyrpt passwords, and the perlmonks code doesn't? If that is true, the monastery has a really big problem, and just changing our passwords once or twice, as advised in It's Time for Everyone to Change Passwords!, is just trying to cure the symptoms.


      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
        Evidently they were stored plain text. Until someone updates the users that the breach has been closed and the passwords are actually being stored in a sane manner, you should expect that people who care to do so have full access to your profile.
      does the code really store passwords in different places and with different encoding schemes depending on the user status?

      No, only 3000+ xp were selected for exposure

Re: What happened?
by scorpio17 (Canon) on Jul 29, 2009 at 19:54 UTC

    I don't get it... why were the passwords in plaintext? One would think the powers-that-be behind this site are all perl gurus, but this is a really bad nooby mistake. It's pretty embarrassing actually. Once the encryption part is fixed, the login page should probably be changed to use https, too.

    And what about The server is on their network, right? What are they doing to assist?

      Who the hell stores passwords in plain text - especially on a programmers' web site? I knew this place had its flaws - hundreds of SQL queries per page view, extremely slow page loads (25 seconds to process my votes on this page's nodes), but really. I kept passwords unhashed for about the first 6 months I was programming. This is up there with using strict and warnings. Cannot... comprehend... stupidity...

      Why not just make this site static for historical reference and redirect everyone to Stack Overflow? The only thing I miss over there is the chat sidebar.

      On the Other Hand, the fact that the login page for PM was not https made me go *shrug* and pick a unique throw-away password from way back when I first created my account.

      #my sig used to say 'I humbly seek wisdom. '. Now it says:
      use strict;
      use warnings;
      I humbly seek wisdom.
      Since every request authenticates by cookie, everything would need to be https, not just the login page. And as far as I can see, that ain't gonna happen.

      Update: or, as tye says, we'd do something quite different with cookies.

        True, for our current cookies.

        But the cookie shouldn't depend on the password itself. The cookie should be more like a cryptographic hash of something including the hashed password. So sniffing the cookie would not get you very far. You'd have to learn the "secret" string and then reverse a hash function and that would then only get you the hashed password.

        The 'login' page should be https://. I've long wanted that. That'd also mean getting rid of the "login nodelet".

        Also, a good password hashing function takes significant CPU time. There is a balance to be struck there. Making op=login a very convenient denial-of-service attack point is a bad idea. We might want to implement a "you recently tried to log in (from this IP address or for this username), please wait before trying again" response, which means we only need to worry about attacks from botnets. Surely that would never happen. /:

        - tye        

Re: What happened?
by rowdog (Curate) on Jul 30, 2009 at 09:35 UTC

    zf0 is what happened to us. The cat's already out of the bag so go read zf05 for yourself.

    At least they kind of like us...

    In case you guys are worried, we did NOT backdoor dozens of your public Perl projects. Honest. Why would we want to do that?
    Not worth our time ;)

    Ah well, live and learn I guess.

      Thanks for the link to a copy of the haxor's newsletter.
      There is a really simple reason we owned PerlMonks: we couldn't resist more than 50,000 unencrypted programmer passwords.

      That's right, unhashed. Just sitting in the database. From which they save convenient backups for us.

      Believe it or not, there is actually debate at perlmonks about whether or not this is a good idea. Let's just settle the argument right now and say it was an idea that children with mental disabilities would be smart enough to scoff at. We considered patching this for you but we were just too busy and lazy. I'm sure you can figure it out yourselves.

      This isn't a bad set of passwords, either. Programmers have access to interesting things. ...

      And they also published that servers private ssh key, so that might be used to compromise other servers that trust it (depending on their config). And they published that server's password hashes, which is subject to a brute force attack.

      I'm shocked this site hasn't gone off-line for housecleaning. Bad enough to be hacked, glad there's a homepage announcement. Would like to see more repairs. Would like an announcement about how the original exploit, and how subsequent vulnerabilities caused by the info liberated during the breach, have been addressed.

      The one time I suspected a server had been hacked- didn't even have firm proof, just a good hunch- I took it off line, wiped the drive, re-installed the OS from CDs, gave all users new passwords, and restored the scripts/executables from known good sources and the data from backups. Pain in the buttocks but it had to be done. That was a small machine with half a dozen users and I know this site is much much bigger and thus more of an issue to take off-line, but please, it has to be done.

      ... we did NOT backdoor dozens of your public Perl projects. Honest. Why would we want to do that?

      There's more than a bit of wiggle room in that statement. Would code be considered a public project?

Re: What happened?
by jdporter (Canon) on Jul 29, 2009 at 17:19 UTC
    Subject: user database compromised

    That was a message I sent to the members of the "perlmonks" Facebook group. If you are not a member of that group on Facebook, you would not have received the message.

    Between the mind which plans and the hands which build, there must be a mediator... and this mediator must be the heart.
Re: What happened?
by Lawliet (Curate) on Jul 29, 2009 at 05:19 UTC

    I did not get this memo :3

    Were you emailed it?

    I don't mind occasionally having to reinvent a wheel; I don't even mind using someone's reinvented wheel occasionally. But it helps a lot if it is symmetric, contains no fewer than ten sides, and has the axle centered. I do tire of trapezoidal wheels with offset axles. --Joseph Newcomer

      here's the log i found on the web, snipped the pws and other sensitive stuff, but i can assure you they worked ;)

      update : was asked not to post the log so i removed it.
      if any janitor/god wants to have a look - i guess you know where to find it, else just pm...
      I didn't get a memo either; someone told me on twitter. I guess it is word of mouth or else reading davorg's blog or bumping into the information some other way.
Re: What happened?
by Mr. Muskrat (Canon) on Jul 29, 2009 at 19:22 UTC
    You have our email addresses so why were not notified via email??? By the way, I was already considering leaving but this might be the last straw. Thanks guys.

      "You have our email addresses so why were we not notified via email???"

      Very good point, I think we deserve more of an explanation.

      If there is not a site policy to cover such eventualities perhaps it would be a good time to implement one now

        While I do understand the desire for more detailed information, we cannot tell you more than we know. Once we learn more of the details of the breach, we'll inform you, but I consider it useless to post hourly updates of "No New Information".

        Also note that while we are taking the situation seriously, this site is still a hobby operation with people having a day job. This implies that we do not have pre-allocated time to handle such situations and not everybody has free time to spend working on the various things that need to be done currently.

      I know we haven't conversed in a long time but I'd still be sad to see you go. I hope you don't.

        Thanks. I am going to wait and see how this whole affair is handled before I decide.

Re: What happened?
by Anonymous Monk on Jul 29, 2009 at 06:24 UTC
    Pssst! I know your secret password, click here to change it.
      I though it was a joke, boy was I wrong :(

      I'm spitballing here, but I think they somehow injected code (cross site scripting?), and gained db server password, then remotely logged into the DB.

      Man this sucks :(

Re: What happened?
by Zen (Deacon) on Jul 29, 2009 at 20:31 UTC
    I doubt anyone will own up to being a noob with people's data. But, if it were in a company, management would be on the hook for this.

      I'm not so sure... I work for a *very*large* company, and there's just too much remediation work to do. Every year, I have to report known insecurities in the software we have. Every year, it's nearly the same report. There's no money/time for remediation, and the auditors are satisfied so long as all the insecurities are listed in our report.

      I really wish they'd allocate some time/money to get them fixed and off the list!

Re: What happened?
by ank (Scribe) on Jul 29, 2009 at 21:41 UTC
    OH NO! it looks like the hackers used perl 6 to penetrate the site! we are doomed!!!

    -- ank

      I think it was more like php fanatics that did it
        Oh hell no. The same guys report hacking Dan Kaminski and a few other noisy security people in the same zine, mostly because they were using insecure PHP CMS applications on their websites.
Re: What happened?
by mfollett (Initiate) on Jul 30, 2009 at 16:04 UTC
    Is anyone with the list of passwords testing it against CPAN authors with the same or similar username or registered email address to make sure the passwords get changed? It seems like it'd be a good idea to regenerate the passwords or lock the accounts of people who don't change their password and have it published.
Re: What happened?
by Anonymous Monk on Jul 30, 2009 at 23:19 UTC
    I didn't keep my email address up to date and as a result, didn't get the message, and can't get a new password either. What can I do, if anything?
Re: What happened?
by Zen (Deacon) on Jul 29, 2009 at 20:37 UTC
    Is it possible this was intentional; storing plaintext passwords for nefarious reasons? Not just a tinfoil hat, but a real possibility perhaps.
Re: What happened?
by leocharre (Priest) on Jul 30, 2009 at 15:20 UTC
    This post contained unsafe information. Sorry.
      "unsafe information"


      1. Malware distro site?
      2. Personal data?
      3. other?

      If the second, that's a bit of barn-locking after the horse has been stolen.

A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: monkdiscuss [id://784123]
Approved by GrandFather
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2020-09-28 03:21 GMT
Find Nodes?
    Voting Booth?
    If at first I donít succeed, I Ö

    Results (143 votes). Check out past polls.