Re^3: File handles in regular expressions

in reply to Re^2: File handles in regular expressions
in thread File handles in regular expressions

Hi Lotus1,
If so there is a problem with concatenating all the output into a scalar. $matched_lines could end up holding the whole huge file. One possible solution is to replace $matched_lines .= $match.$/; with print $fh $match.$/; Just print incrementally.

Not so, am afraid your suggestion will further affect the performance of the script, because the print function would be call as many times as the strings matches, meanwhile with the scalar used no call is placed.
Using a Profiler (NYTProf) made that very clear.
Try it.

If you tell me, I'll forget.
If you show me, I'll remember.
if you involve me, I'll understand.
--- Author unknown to me

Comment on Re^3: File handles in regular expressions Download Code

Replies are listed 'Best First'.
Re^4: File handles in regular expressions by Lotus1 (Vicar) on Oct 19, 2012 at 02:16 UTC
If you run out of memory the performance won't be so good. I was questioning the logic of using a tied file while holding all the output in memory. The OP didn't state the file sizes or performance requirements. But since you seem to be focused on performance wouldn't it perform better to read the file into an array? For larger files the tied file will only keep part of the file in memory so it will end up rereading the file many times.	[reply]

Replies are listed 'Best First'.

Re^4: File handles in regular expressions
by Lotus1 (Vicar) on Oct 19, 2012 at 02:16 UTC

If you run out of memory the performance won't be so good. I was questioning the logic of using a tied file while holding all the output in memory. The OP didn't state the file sizes or performance requirements. But since you seem to be focused on performance wouldn't it perform better to read the file into an array? For larger files the tied file will only keep part of the file in memory so it will end up rereading the file many times.

[reply]

In Section Seekers of Perl Wisdom