Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: memory use array vs ref to array

by NetWallah (Canon)
on Sep 08, 2016 at 05:59 UTC ( [id://1171363]=note: print w/replies, xml ) Need Help??


in reply to memory use array vs ref to array

According to this article, Each array element consumes a minimum of 24 bytes.

Your 4.5 Gigs of memory will accommodate a maximum of 185,834 elements.

How does this compare with the record count in your file ?

More relevant is the question - WHY do you need to read the entire file into memory?

Typical/efficient parsing handles the file one line at a time.

IF it is necessary to store the file in memory, would it suffice to store the file in a scalar ? (You could do that by setting $/=undef; prior to reading the file).

        ...it is unhealthy to remain near things that are in the process of blowing up.     man page for WARP, by Larry Wall

Replies are listed 'Best First'.
Re^2: memory use array vs ref to array
by hippo (Bishop) on Sep 08, 2016 at 08:36 UTC
    According to this article, Each array element consumes a minimum of 24 bytes. Your 4.5 Gigs of memory will accommodate a maximum of 185,834 elements.

    Respectfully, I think you are out by a factor of 1000 (or 1024) here. Should it not be around 185 million elements?

      There are 22,186,287 lines in the file so about 22M records.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1171363]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (3)
As of 2024-04-19 17:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found