|Perl: the Markov chain saw|
Diagnosing a pregnant pause.by BrowserUk (Pope)
|on Sep 03, 2011 at 22:28 UTC||Need Help??|
BrowserUk has asked for the
wisdom of the Perl Monks concerning the following question:
When I run an evolution of this script with a particular set of arguments: Ibufd.pl -IBUF=20971520 -NBUF=6000 1GB.csv, about 18 seconds into a 2 minute run, the cpu and IO activity suddenly drop to 0 and stay there for 3 seconds. Memory requirements are exactly stable at that point in time and the overall memory requirements are well within the bounds of my free physical storage.
I've posted a screen grab graphing the cpu/memory/IO history of the entire process run here. The vertical divisions represent 3 seconds. This is entirely consistent regardless of what else is running or how many times I re-run the test.
I'm trying to track down the cause of the pause. I've tried several methods of trying to instrument the process, but all of them completely change the performance characteristics of the process rendering all the information produced meaningless. My purpose in posting is to try and get answers to the following questions:
To aid anyone thinking of trying to reproduce the problem, this script (with default options) will create a 1GB CSV file consistent with the input expectation of the script:
And this is the evolved script, the required command line is shown at the bottom:
A hearty thank-you to anyone taking the time to help me with this.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.