Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re: Caching hash

by gaal (Parson)
on Jun 29, 2009 at 20:35 UTC ( #775801=note: print w/ replies, xml ) Need Help??


in reply to Caching hash

What's being slow, the IO of reading it from disk, or do you have some calculation going on in making it? How is it actually stored on disk? The fastest format for that is probably Storable::nstore(\%data, $file); ... Storable::retrieve($file). If you have some long-running process that can hold a copy of the data and you want to use that, is a real copy (over a named pipe or something) really too expensive? How big is your data? Is it immutable?


Comment on Re: Caching hash
Download Code
Re^2: Caching hash
by vit (Pilgrim) on Jun 29, 2009 at 20:45 UTC
    The reading is slow. I use Storable and the data size is around 1g. It is processed (calculations) in less then one sec, but read much longer.
      That's a lot of data! I bet you don't need it all at once on the transient CGIs, though.

      Should your long-running process provide a query interface? That way the random scripts that come and go only ask it for the stuff they need. You can memcache the results of these queries if you like. No need to spend 1gb on the cache: experiment with some values, see what your hit rates are.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://775801]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (9)
As of 2014-09-21 18:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (175 votes), past polls