|Problems? Is your data what you think it is?|
Storable Objectsby BioLion (Curate)
|on Jul 13, 2009 at 16:59 UTC||Need Help??|
BioLion has asked for the
wisdom of the Perl Monks concerning the following question:
I wanted to get your opinions before committing to anything, so here goes:
Having read around (Burned by Storable, Re: Caching or using values across multiple programs, How to duplicate objects?, Serializing data ,Data::Serialize, Data::Dumper, YAML, ), I am a bit put off by the frequency of people having bad times with Storable. Especially with serializing code refs and the need to use Safe compartments for inflating code refs.
So my question is this - what is the (IYHO) currently preferred method for (recursively) storing objects?
By way of warning, I want to apply this method to a very large object that is a container for ~50-100,000 smaller objects/code refs, that in turn may contain further objects... Much calculation goes into building it up, and i would like to able to cache it at key stages, so that in future i can choose whether to bypass building it from scratch each time.
Thus, if i could also test for the existence of a stored object at each checkpoint too that would be a huge bonus for me:
Thanks in advance,
Update edited my ramble a bit
Update 2 (16 July 09)After much helpful discussion (esp. clinton/ELISHEVA on the CB, and the posts below), i came to the realisation that storing code refs and objects, while possible, doesn't make sense in my situation because the testing post-inflation ( Re^2: Storable Objects ) would take more or less as much time as i could save by having 'storage checkpoints' during the build-up of my mecha-objects, and moreover loses me/users much flexibility (i.e. what if they have multiple DB/flat files that they want to work from).
So in the end I decided ( thanks perrin! )that it would make more sense to put that time into making my code more efficient, so that users don't need to go for a coffee while their computer thinks...
My codebase is fairly large / diverse, but it has all been handled wonderfully by Devel::NYTProf, which seems to be the profiler-du-jour round here ( Re: Profiling and Performance ). There was also much useful info found here. I particularly found the nytprofhtml utility very useful and nice to work with, especially the way it gives you your code right next to how it is being used, and your can very easily click-navigate right across any code that was called! I set it up with these options:export NYTPROF=trace=2:start=begin:file=/tmp/nytprof.out:addpid=1:savesrc=1
So, just a big thanks all round!
Just a something something...