Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: Reducing memory footprint when doing a lookup of millions of coordinates

by nikosv (Chaplain)
on Feb 27, 2011 at 16:29 UTC ( #890431=note: print w/replies, xml ) Need Help??


in reply to Reducing memory footprint when doing a lookup of millions of coordinates

Maybe this is a case for Memoization

In computing, memoization is an optimization technique used primarily to speed up computer programs by having function calls avoid repeating the calculation of results for previously-processed inputs

The Memoize module makes it easy to use by wrapping around your function.

It might increase the memory footprint but make the processing faster so it better to benchmark it,and if you do use it afterall I would be interested in hearing the test results.

Memoization is not suitable for your project. I've read your question hasty and misinterpreted it
  • Comment on Re: Reducing memory footprint when doing a lookup of millions of coordinates

Replies are listed 'Best First'.
Re^2: Reducing memory footprint when doing a lookup of millions of coordinates
by richardwfrancis (Beadle) on Feb 28, 2011 at 08:33 UTC

    Wow, I've definitely got plenty to work with here. Thank you all for your help.

    Kind Regards,
    Rich

      Be aware that memoisation will not help your problem at all.

      Memoisation speeds up repetitive calculations by caching the results of those calculations to avoid recalculating them. It uses (often prodigious amounts of) memory to gain speed.

      Since your stated goal is to reduce memory usage; since you have no repetitive calculations; any attempt to use the linked module could not help your performance and would increase your memory footprint, probably to the point of out-of-memory failure.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Thanks BrowserUK,

        I had a read up on Memoization and came to the same conclusion. However I realised I'd implemented it in another recent project without actually knowing what it was!

        Regards
        Rich

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://890431]
help
Chatterbox?
[perldigious]: Tanktalus: Ha, understandable, that does make it harder when you don't want to move. I assume you are looking at remote work from home jobs too?
[Tanktalus]: yeah, I'm looking at a lot of remote work - I've been working from home since 2002, so I know how to do the work from remote, I just need a new employer ;)
[Lady_Aleena]: Discipulus, I wouldn't know how to get a script to do what the greps do in one line.
[perldigious]: Even working for a large software engineering contractor that's willing to do remote could be an option. The fortune 500 company at $job-- I worked for had quite a lot of that work to hire out to large contractors.
[perldigious]: The pay and benifits leave a lot to be desired though, or so I've heard...
[Lady_Aleena]: Discipulus, grep doesn't have an option to go recursively through a directory as far as I know.
[Your Mother]: I work remotely and my pay and benefits are quite nice. :P
[choroba]: Lady_Aleena Perl's grep is not related to directories
[Your Mother]: It does recurse.
[Lady_Aleena]: choroba, I know.

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (10)
As of 2017-05-23 19:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?