Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: Memory issue with large array comparison

by Eliya (Vicar)
on May 24, 2012 at 21:21 UTC ( #972325=note: print w/ replies, xml ) Need Help??


in reply to Memory issue with large array comparison

The task doesn't require that you have all pathnames in memory at once. It can just as well be solved by processing one path at a time.  So, for example, read them one by one from a file

... while (my $path = <PATHS>) { # check if path component isn't in @safe_list ... }

That should significantly cut down on memory usage.

Also, iterative solutions (plain old nested loops) are generally less memory hungry than functional implementations (grep, map and friends) of the same task. This is because the former avoid creating additional lists on the stack.

BTW, do you really need the regex /$_\w+$/, i.e. wouldn't a simple hash lookup (of the appropriate path fragment) work, too?


Comment on Re: Memory issue with large array comparison
Select or Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://972325]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (2)
As of 2014-09-20 03:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (152 votes), past polls