Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Memory usage while tallying instances of lines in a .txt file

by dave_the_m (Prior)
on Dec 05, 2016 at 16:44 UTC ( #1177244=note: print w/replies, xml ) Need Help??


in reply to Memory usage while tallying instances of lines in a .txt file

How many lines is the input file?

columns 2 and 3: are they always integers; do they have any well-defined min and max possible values; is their disturbution sparse (e.g. is it possible col 3 might have values 336158, and 336159, but then nothing till 491797, or is it likely that most of the gaps inbetween will appear at some point)?

Dave.

  • Comment on Re: Memory usage while tallying instances of lines in a .txt file

Replies are listed 'Best First'.
Re^2: Memory usage while tallying instances of lines in a .txt file
by TJCooper (Beadle) on Dec 05, 2016 at 17:23 UTC
    The input files can contain millions of lines - with around 1-1.5m unique entries that will be tallied up. The entries are indeed sparse.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1177244]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2018-08-15 22:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Asked to put a square peg in a round hole, I would:









    Results (165 votes). Check out past polls.

    Notices?