in reply to Performance Question
I will second the notion that this is really more a systems issue than a perl issue.
Others have mentioned pre-sizing your destination file (which could help guarantee contiguous spacing on disk for the file), splitting the file into pieces and processing in parallel.
But what caught my eye that no one else has responded to (yet) is that you are reading and writing the file from the same disk. This means that for every physical read and write (which will depend on O/S-level buffering) your disk needs to seek. If there is any way you can read from one disk and write to another you should see some speedup there, too.
Best of luck,
--JAS
Others have mentioned pre-sizing your destination file (which could help guarantee contiguous spacing on disk for the file), splitting the file into pieces and processing in parallel.
But what caught my eye that no one else has responded to (yet) is that you are reading and writing the file from the same disk. This means that for every physical read and write (which will depend on O/S-level buffering) your disk needs to seek. If there is any way you can read from one disk and write to another you should see some speedup there, too.
Best of luck,
--JAS
|
|---|
In Section
Seekers of Perl Wisdom