Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Storing credentials in a cross-platform binary file?

by Anonymous Monk
on Sep 11, 2008 at 19:48 UTC ( #710721=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Looking for some wisdom here: I have a mandate (due to SOx and other audit requirements) to get DB, LDAP, web service, and other usernames/passwords out of my code and into a local file. Essentially if my code needs a set of credentials to access some service, those credentials should be stored in a separate file not subject to version control and not in the path of the web server. Further, those credentials should be encrypted in such a way that the calling script(s) can read them, but should the file fall into the wrong hands the passwords aren't easily recoverable.

Here is the difficulty: I need to be able to share this file across a linux server, two Windows Server 2003 servers, and two Solaris 8 servers without having to manually generate it on each architecture (which eliminates DBM). For futher difficulty I do not have access to compilers on the Solaris servers, so using a XML solution is right out as I can't compile Expat and do not have root access to install sun freeware packages. Additionally, I'd like the files to be binary to discourage casual browsing/modification from other team members who might fiddle with the file with good intentions but not understanding what they're doing.

So here are my current thoughts: Create a module to wrap around an SQLite DB. SQLite is available on all of the platforms, and the DB file itself is guaranteed to be portable (according to SQLite docs). I can add new credentials to the DB on one server and distribute the DB file to the others without having to modify it. I can use an encryption module to store the password before it goes in the DB file, so that meets the encryption requirement. However I can't help but think an SQL engine is overkill for this.

Next option would be similar to above but wrap around an SDM_File instead. Gotchas here would be the length restriction of the DBM files, and is SDM_File always going to be portable across environments? And is it reliable?

There are obviously plenty of other options, such as rolling my own binary file format or using some sort of obfuscated text or ini-file format.. But again with text I'm afraid others with access to the server might be tempted to tinker with the file.

Does anyone know of solutions for this that might already exist in CPAN? The closest I can find is DBIx::Password, but as far as I can tell the credentials are stored in plain text in perl modules, and this would not help for web services, LDAP, etc.

Any advice?

Comment on Storing credentials in a cross-platform binary file?
Re: Storing credentials in a cross-platform binary file?
by moritz (Cardinal) on Sep 11, 2008 at 20:27 UTC
    Additionally, I'd like the files to be binary to discourage casual browsing/modification from other team members who might fiddle with the file with good intentions but not understanding what they're doing.

    I thought that file was encrypted anyway? If so, the obfuscation is really superfluous.

    Or do you want to encrypt the passwords in that file, but not the file itself?

    I assume that the credentials file isn't very large. If that's true, you can store your login informations in something as simple as YAML or Storable, encrypt the whole thing and dump it to disk. In the other directory you read and decrypt it, and let your de-serializer handle that string.

      OP here: my original thought wasn't to necessarily encrypt the whole file, rather to just encrypt the individual passwords before writing them to the file. This would leave the file open to casual editing/browsing (even though the PWs are encrypted) if it's a text file. However using a serializer and encrypting the resulting output is an angle I hadn't considered - I'll explore that.. Thanks!
Re: Storing credentials in a cross-platform binary file?
by jbert (Priest) on Sep 11, 2008 at 20:42 UTC

    Why not use the good old one time pad.

    You can embed a big chunk of random data in your script (this is your decryption key) and then simply XOR your data file with that.

    The file format (after XOR) can be a simple text/ini format, but won't be readable to anyone without the key.

    You'll want to make sure the XOR data is longer than the possible length of your file, and generated randomly (man 4 random on your Linux box - you want /dev/random, not /dev/urandom).

    It's a simple approach, but provided you use real random data, longer than your file, it's effectively unbreakable crypto.

    Stick your random data (hex encoded) in the __DATA__ section of a module, and you're fine.

      I simply cannot express how bad an idea this is.

      First there is the problem of what you do if someone gets access to your script. But that is minor.

      Much more serious is the fact that one time pads only work if you only use them once. Suppose someone gets access to your data. If they xor an "encrypted" password with the real password, they get your xor data back. All they need to do is take a small dictionary of common passwords, xor it against 100 passwords, and look for some piece of xor text popping up more than once. (Lots of people use very bad passwords.) Once they find that, they now have your xor text and they have everyone's xor text.

      If you are doing this anywhere, be assured that any serious security audit should find the fact that you are doing this, and you will get a lecture about how wrong you are.

        I think he meant for the whole file to be encoded as one message, so the pad would only be used once.

        Once, that is, for every time the file is modified. Does it really make sense to update the key in the script every time the file is modified?

        And does it make it safer? The hacker now needs to get two files instead of one, but you'd think he'd be able to get both if he can get one.

        First there is the problem of what you do if someone gets access to your script. But that is minor.

        That's a given. He's storing credentials in an external file to be 'decrypted' by the script. The script has to have the info (key+algo) needed to decrypt the credentials file available to it. Anyone who has access to the script has access to the data, whatever scheme is used.

        Suppose someone gets access to your data. If they xor an "encrypted" password with the real password, they get your xor data back. All they need to do is take a small dictionary of common passwords, xor it against 100 passwords, and look for some piece of xor text popping up more than once. (Lots of people use very bad passwords.) Once they find that, they now have your xor text and they have everyone's xor text.

        Please elaborate. If I'm storing the repeated text in your scenario (10 passwords, all "banana\n") and XORing that with a random chunk of data *longer than the total password data*, you won't see repeated XOR text if you XOR the file with banana, since there is no repeated XOR text. You will have the XOR key but not know it. The XOR key needs to be bigger than the stored data (I stipulated this).

        Much more serious is the fact that one time pads only work if you only use them once.

        This is a more legitimate concern. If the file is repeatedly acquired by other people, and changes often, and changes in ways which allow multiple guesses of passwords to expose some of the XOR key, then yes, parts of the XOR key can be obtained.

        But that requires an environment in which regular password change is mandated, which is the sort of environment which doesn't allow easily guessed passwords. Changing the XOR key with approximately the same frequency as the passwords are changed defeats this.

        Edit: yes, using another crypt module is probably a better idea but perhaps harder to achieve given his platform constraints. Homebrew crypto is a bad idea. But we're only really obfuscating here since the script can decrypt at any time, and the script is presumably no better protected than the credentials file.

Re: Storing credentials in a cross-platform binary file?
by jdrago_999 (Hermit) on Sep 12, 2008 at 02:11 UTC

    Just encrypt the file using an algorithm that is available on all of the platforms you need to support.

    Then you still run into the question "But where do I store the key to decrypt that file?"

    You cannot stop a determined hacker. You can, however, help to keep honest people honest.

    An alternative is to "sign" the password file. If it is changed, it will no longer match the signature.

      Why would you ever need to decrypt it?
        Why would you ever need to decrypt it?
        So that you could use the password?
Re: Storing credentials in a cross-platform binary file?
by missingthepoint (Friar) on Sep 12, 2008 at 03:06 UTC

    I'd recommend encrypting the whole file for the security benefit. If you encrypt individual parts of file separately, you make it easier to break the encryption. Of course, this may not be an issue, depending on your definition of 'easily recoverable'.

    If the whole file is encrypted then it's indeed binary, and not likely to be fiddled with.

    If you do encrypt the whole file, you may have issues using SQLite. I could be wrong, but I think SQLite require a disk file to operate on. If you decrypt an encrypted SQLite database file and write a temporary plaintext version to disk, you render all the encryption pointless.

    With all that in mind, I think the best solution is what moritz suggested: use YAML or Storable and encrypt the whole file.


    email: perl -e 'print reverse map { chr( ord($_)-1 ) } split //, "\x0bufo/hojsfufqAofc";'
      There are standard encryption mechanisms that are so good as to be effectively unbreakable. As Bruce Schneier says, attempting to improve these systems by improving the encryption is like trying to extend the wall in the path from 2 km high to 5 km high while hoping that attackers won't just walk off the path and go around the wall.

        By 'breaking the encryption' I meant 'going around the wall', that is, rendering the encryption ineffective. I know, taken literally 'breaking the encryption' means finding/exploiting a flaw in the actual algorithm, but I was speaking idiomatically - think 'arrays of arrays' in Perl.

        Incidentally, and in agreement with you, tilly: I can't remember the source, but I recall someone from the NSA as being quoted as saying: "We don't need to break DES, because the implementations are so poor". For lots of cases, you can probably s/DES/modern cipher/.


        email: perl -e 'print reverse map { chr( ord($_)-1 ) } split //, "\x0bufo/hojsfufqAofc";'
Re: Storing credentials in a cross-platform binary file?
by Anonymous Monk on Sep 12, 2008 at 04:31 UTC
    OP here: just to provide some additional info on the audit requirement.. As some mentioned, you cannot stop a determined hacker, and that's not necessarily the point of the requirement. The idea is you shouldn't just hand credentials to anyone walking by whether they need them or not. According to the requirement, you shouldn't have code in your script like this:

    my $DBH = DBI->connect('dbi:mysql:host=server.example.com', 'dba', 'my +dbapassword');

    The idea is you store the credentials in an external file that never hits version control (many people don't think about passwords sitting there). The passwords in the external file are encrypted, so that should the file fall into the wrong hands (misconfigured web server, file symlinked into FTP root, etc) the passwords aren't easily obtainable. Likewise, if someone gains access to your source code repository they shouldn't be able to gain DB/LDAP/web service passwords from the code. Lastly, with passwords in a separate file you can be more pedantic about file-system ACLs to control access to the file, in addition to the other safeguards in place. So, ultimately it's a good requirement, I'm just surprised there's no existing CPAN module to implement it.

    Thanks to all for your help! I think I will go down the road of serializing the credentials and encrypting the resulting file. Who knows, maybe I'll try to contribute my work to CPAN (with my company's approval) for others.

      I think jdrago_999 hit the nail on the head and you may have missed it. I'm no cryptographic genius but you haven't mentioned the keys. Access to the "password file" is a moot point after passwords are encrypted... almost.

      I would encrypt each password with different keys so that the LDAP, web server, database password are encrypted with seperate keys. You could then limit access to the key files with ACLs. The scripts could only access the keys corresponding to the passwords they need.

      With this concept something as simple as a serialized hash could contain the encrypted password data. Then you would access whatever password you needed by name via hash element. You could use Crypt::CBC to encrypt/decrypt the cyphertext and use them in your scripts.

      The key here is (are?) the keys.

Re: Storing credentials in a cross-platform binary file?
by dHarry (Abbot) on Sep 12, 2008 at 14:19 UTC

    Wow, your post certainly triggered lots of discussion:-)

    For the record: I am/used to be a cryptologist. There ain’t no such thing as 100% secure but you can make it very safe and approach the 100% if needed. In your case that seems like overkill however (do you have specific requirements?).

    I must confess that I am surprised/shocked by some of the arguments/ideas put forward:

    “good old one time pad”
    I would say more old then good:-) During the cold war the hotline between Washington and Moscow was based on something similar. It’s extremely safe but can only be used once then you have to change key. They had to exchange the (many) keys prior to communication over a secure channel of course. IMHO: You can do a lot better then using this approach, you probably don’t have the same requirements as the “hotline” either;-)

    “If you encrypt individual parts of file separately, you make it easier to break the encryption”
    I wonder were this idea comes from it is simply not true.

    “We don't need to break DES, because the implementations are so poor”
    A surprising quote from the NSA? When I majored on this stuff DES was safe. However DES has been cracked/hacked and is considered unsafe nowadays (That’s why they use things like tripleDES). For the record: the DES algorithm is often implemented in hardware because it outperforms the software implementations by a factor. I wouldn’t call these hardware implementations pour. BTW: the security requirements of DES were kept secret by instigation of the NSA!

    So in your case a simple file containing the passwords with a standard off-the-shelve cryptographic algorithm should do the trick unless you have very specific requirements (well do ya?). When in doubt you can always hire a specialized company to assess your security to gain confidence in it.

    Last but not least there is the problem of Key management which is normally the most difficult component of the entire cryptographic system! You will have to change the password with some frequency and device a strategy for that.

    Rest my case

      “We don't need to break DES, because the implementations are so poor” A surprising quote from the NSA? When I majored on this stuff DES was safe. However DES has been cracked/hacked and is considered unsafe nowadays (That’s why they use things like tripleDES). For the record: the DES algorithm is often implemented in hardware because it outperforms the software implementations by a factor. I wouldn’t call these hardware implementations pour. BTW: the security requirements of DES were kept secret by instigation of the NSA!

      I think the quote referenced was actually talking about the fact that most times expensive crypto breaking is the hard route to take when looking for the data -- most times there are other simple paths to take to get at the data. The systems implementation, procedure and policy usually left gaping holes to exploit before needing to crack the crypto.


      -Waswas
Re: Storing credentials in a cross-platform binary file?
by DrHyde (Prior) on Sep 16, 2008 at 10:19 UTC

    For a cross-platform binary file format, take a look at Storable. In particular, the nstore() function to store data in network order as opposed to the machine's native byte order. It's core in all perl 5.8.x versions so you shouldn't have to build it yourself.

    Even so, you don't have compilers? Well then, if you need compilers to do your job then your boss will need to get you compilers. You can't just wave a magic wand.

    And so what if you don't have root? Your sysadmins will, and its their job to install stuff that users need, after checking that you really do need them. I can certainly understand why they wouldn't want to install compilers on live systems, but don't you have development machines? If not, why not? That's another thing that your boss should be providing if he wants you to do your job.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://710721]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (10)
As of 2014-07-23 08:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (137 votes), past polls