Re: Out of memory

[fishmonger]

How big is Data2014.txt?

I doubt that the out of memory problem is due to slurping that file into an array instead of looping over it line-by-line.

I suspect that the out of memory issue is related to the retrieval of the url/file. Instead of copying the content into one or more vars, I'd output it directly to the file.

EDIT:
The LWP::Simple module has a getstore() function that will simplify that process.

[wrkrbeee]

Size of Data2014.txt is 1.5 MB, so you are probably correct. Any tips for the output directly to the file? Thank you!

[Corion]

See LWP::UserAgent on Request Methods. Especially the :content_file parameter allows you to have LWP store the data directly to disk.

[wrkrbeee]

Thank you!