Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

File I/O structure

by silverbullet (Initiate)
on Jul 11, 2013 at 15:54 UTC ( #1043767=perlquestion: print w/replies, xml ) Need Help??
silverbullet has asked for the wisdom of the Perl Monks concerning the following question:

Good morning...

I am looking for an alternative to the sequential files that seem to dominate PERL. I have found only database structures and sequential flat files. Is there some sort of random access file with true file and record locking? If not, is there some sort of module that adds that?

Replies are listed 'Best First'.
Re: File I/O structure
by roboticus (Chancellor) on Jul 11, 2013 at 16:44 UTC


    As is often the case, the answer depends on the problem you're trying to solve. What difficulties are you having with flat files? You mention random access and record locking, so it may be that using a database back end might be the answer. Then again, as MidLifeXis suggests, DBM::Deep might ease your pain.

    Of course, XML might be good, so you might find something like MongoDB to be better.

    Different problems require different solutions ... there is no silverbullet..

    (Yes, the entire post was just for that punchline....)


    When your only tool is a hammer, all problems look like your thumb.

      Thank you for your reply. I will look into the DBM::Deep. Thanks for the help.
Re: File I/O structure
by marinersk (Priest) on Jul 11, 2013 at 20:23 UTC

    "Seem to" dominate? The very meaning of the name of Perl suggests it would be predominately focused on sequential text files.

    That said, and DBM::Deep already mentioned, if you want to spin your own module to support what we in the dinosaur days would have called Fixed-Length Random Access File processes, you could look into the following features of Perl:

    Input and output functions:

    • binmode
    • read
    • seek
    • sysread
    • sysseek
    • syswrite
    • tell
    • write

    Modern data processing (is that an oxymoron?) thinks more in terms of XML and DB for re-usable, rewriteable data; but if you must play in the old fashioned fixed-length random-access file space, the above tools will readily permit migration of your old C code where you last reinvented that wheel to Perl where you seem fixated on this somewhat dated approach to data storage.


    All that having been said, I myself could not resist the urge to port my old FLRAF routines to Perl when I first got here. I never bothered to write a module I would dare to put in front of other software engineers, however, since I knew going into the effort it was an exercise to produce a tool I was unlikely to ever need.

    DBI/SQLite has been a good friend:

    use DBI; my $dbargs = { AutoCommit => 0, PrintError => 1 }; my $dbscns = "dbi:SQLite:dbname=$Db_dbn"; &debug::debug("\$dbscns = '$dbscns\n"); my $dbshan = DBI->connect($dbscns,"","",$dbargs); &debug::debug("\$dbshan = '$dbshan'\n");
    Updated to computerize DBM::Deep
      I appreciate your suggestions and perspective. Thank you.
Re: File I/O structure
by MidLifeXis (Monsignor) on Jul 11, 2013 at 16:16 UTC
      Thank you for your suggestion. I will look into it.
Re: File I/O structure
by sundialsvc4 (Abbot) on Jul 12, 2013 at 03:16 UTC

    I have had a lot of success with SQLite, which is (by careful and deliberate design ...) a public-domain SQL system which stores everything in one file and requires no server.   The very nice thing about it is that, well, “it’s a [self-describing(!)] random-access file,” and a whale of a lot more.   All wrapped-up in a very small, high-performance engine that runs everywhere and with every language (not just Perl).   Quite frankly, I don’t “roll my own random-access files” anymore.   I can’t justify the effort, nor the bugs that I can avoid.

    There is only one caveat, but it is an important one:   use transactions.   SQLite is also designed for fail-safe situations, so it will re-read everything that it has just written, quite by-design, unless you are using transactions, which allows more-“lazy” writes.   But, hey, you get “a self-describing multi-user random-access file that is understandable everywhere,” with transactions!   Not bad.   Not bad at all.

      Thank you for the suggestion.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1043767]
Approved by toolic
Front-paged by toolic
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2017-12-17 04:44 GMT
Find Nodes?
    Voting Booth?
    What programming language do you hate the most?

    Results (462 votes). Check out past polls.