Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

I think that you've discovered that Storable provides a persistence mechanism that enables you to dump and load data structures. You are also looking for something scaleable - you were worried about the time to store the data; having the whole data structure in memory is going to eat into your process and operating system resources if it gets large.

I can suggest two related approaches: objects and tieing. An object can be a placeholder for some point in your arbitrarily large data. If your application keeps to the OO encapsulation rules, you are only holding the data in memory that you actually need for manipulation; the other parts can still be accessed via method calls. One approach is to hold the data in a database, and use Class::DBI as an object persistence layer to turn rows into objects.

Tieing provides an alternative interface your data, and can make it look like an array or a hash (or a filehandle, etc.) When the application performs operations on the tied variable, method calls happen to do the sleight of hand and get your data. The application does not need to know that this is happening - it just sees an array (or hash). See perldoc perltie for more on this.

Hope this helps

--

Oh Lord, won’t you burn me a Knoppix CD ?
My friends all rate Windows, I must disagree.
Your powers of persuasion will set them all free,
So oh Lord, won’t you burn me a Knoppix CD ?
(Missquoting Janis Joplin)


In reply to Re: Saving big blessed hashes to disk by rinceWind
in thread Saving big blessed hashes to disk by b888

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2024-04-19 07:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found