|Problems? Is your data what you think it is?|
Possible to have regexes act on file directly (not in memory)by Nocturnus (Sexton)
|on May 02, 2014 at 16:50 UTC||Need Help??|
Nocturnus has asked for the
wisdom of the Perl Monks concerning the following question:
I have some files which I would like to parse somehow using regular expressions. The files could be several gigabytes or even up to a terabyte in size, so loading the files to memory is not an option.
The files are not guaranteed to have line breaks, and it is not known in advance if the search pattern exists in a certain file, where it is and how often it occurs.
Hence the question: Is it possible to let a regular expression act directly on a file *without* reading any part of the file to memory?
Sorry if this has been answered already; of course, I have tried to do my homework, but probably used the wrong search keywords (I just found the usual posts or tutorials where people suggest to read the whole file into a scalar variable and let the regular expression act on that variable, or to split the file in chunks and test each chunk etc.).
Thank you very much for any thoughts,