http://www.perlmonks.org?node_id=1084839


in reply to Re^3: Possible to have regexes act on file directly (not in memory)
in thread Possible to have regexes act on file directly (not in memory)

The size of the data doesn't matter, it's the maximum size of the regex which counts. Suppose you have a maximum regex size of n, then your chunk size should be n (or greater). The algorithm is then:

  1. load 1 chunk into RAM
  2. load another chunk into RAM
  3. concatenate the 2 chunks for testing
  4. Discard the older chunk
  5. Goto 2

HTH.