|Problems? Is your data what you think it is?|
Counting concurrent event jobsby vagnerr (Prior)
|on Apr 24, 2006 at 16:55 UTC||Need Help??|
vagnerr has asked for the
wisdom of the Perl Monks concerning the following question:
Greetings fellow monks! ...
I am doing some performance analysis on some batch processing that we run (log processing). I need to determine how many concurrent jobs are running at set intervals over a day/month period (and of what type), based on a log of the start and finish times of each job. For arguments sake lets say the log files are in the following format.
I need to be able to convert that to some sort of report along the lines of
The simplest solution would appear to be to create a nice big array, or hash. Each node would represent a 1-5 minute window (depending on the required granularity) referencing an array of the individual types. The logs would then be processed line by line. As we find a matching set of "start" and "finish" lines we update the counters for that type for all the time segments between the start and the finish times.
The problem as I see it with this solution is that whilst it does work, its a little messy and can create quite a large data structure. Does anyone have any suggestions on a better approach to the problem?
Remember that amateurs built Noah's Ark. Professionals built the Titanic.