Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

Using the file system as a DB in a web app?

by jimbus (Friar)
on Apr 07, 2006 at 05:11 UTC ( #541792=perlquestion: print w/replies, xml ) Need Help??
jimbus has asked for the wisdom of the Perl Monks concerning the following question:

I'm converting an app from JSP to mason/mod_perl that allowed xls files to be uploaded and then hyperlinked them for later access. When I'd originally built the app, the customer had a limited number of reports and it made sense for me to set up a few columns in mysql to handle the three keyword columns (one keyword per), urls and display name for each file. I used the keywords to create the directory structure in the upload area. As part of the conversion, there is a change in requirements. The customer has decided he needs more/different granularity because he had more reports. Now different reports and resources have different numbers of keywords and things aren't as neat as they were and I'm wondering if there might be a better way to approach tracking the files.

My thought is, since I have a fetish for fully descriptive file and directory names, could I write a quick PM to travesrse the varying length directories, prettying up and capitaling the names and embedding them in hyperlinks to the next level to print out on the page. When I reach a leaf, I could extract the entire path and filename for the page title and the date from the filename for the hyperlink to the file.

OK, I guess the point of this.. other than it helps focus things in my head to try and explain them... is: How safe is this? Is this realistic from a a performance point of view? Of course, I'll regex all the input to make sure no one is trying to back door the system. Is there anything else I need to worry about?

--Jimbus aka Jim Babcock
Wireless Data Engineer and Geek Wannabe
  • Comment on Using the file system as a DB in a web app?

Replies are listed 'Best First'.
Re: Using the file system as a DB in a web app?
by graff (Chancellor) on Apr 07, 2006 at 07:03 UTC
    I don't see any intrinsic problem with the plan, so long as the application operates within reasonable limits, such as:
    • The number of entries created in a given directory: depending on what kind of file system it is, having many thousands of things in a single directory can be a real pain.

    • The nesting depth of directories: if you're going deeper than, say, 15 levels, you should probably rethink the process.

    • Overall length of a single path string: maybe your OS or various relevant apps won't really have a problem with this, but when deep nesting is combined with a verbose naming style, it becomes unwieldy and unpleasant.

    • Overall quantity/size ratio of data files: I've seen people create directory trees with a million itty-bitty little data files, and believe me, it's not just ugly, it's a serious liability for everyone involved (including the sysadmins who have to run backups on it). Anything that is high-quantity/small-size needs to be in a regular database, not a directory tree.

    Apart from that, I think doing capitalization is a waste of time, but that's just me.

Re: Using the file system as a DB in a web app?
by Zaxo (Archbishop) on Apr 07, 2006 at 07:08 UTC

    If you're just concerned about covering the client's existing file structure, File::Find may do the trick.

    If you're responsible for creating the thing, you'll need to understand the intent before you can use it or fix it.

    After Compline,

Re: Using the file system as a DB in a web app?
by dragonchild (Archbishop) on Apr 07, 2006 at 11:56 UTC
    If you need a persistent hash-of-hashes, why not use DBM::Deep?

    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
Re: Using the file system as a DB in a web app?
by Tanktalus (Canon) on Apr 07, 2006 at 17:48 UTC

    As long as we're disclosing our fetishes, mine is RDBMS's. I'd just create a table using a many-to-many relationship between keywords and filenames (or file IDs if you use them). The file identifier (name or number) would be a foreign key. Then you can use select with join to get whatever you need.

    The actual file could either be stored as a BLOB if your db is good at that, or it could just be a pseudo-random directory and/or name relative to some path (which could even be the root directory, i.e., an absolute path).

    So, because of my particularly-coloured glasses, I probably would use a DB for a DB in a web app ;-)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://541792]
Approved by Corion
and the grasshoppers chirp...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2018-05-22 18:16 GMT
Find Nodes?
    Voting Booth?