Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Dynamic thresholds in report

by mhearse (Chaplain)
on Jan 04, 2013 at 22:32 UTC ( #1011729=perlquestion: print w/replies, xml ) Need Help??
mhearse has asked for the wisdom of the Perl Monks concerning the following question:

I've written an hourly report which utilizes static thresholds. Which are stored in a config file. I think the report would be more meaningful with dynamic thresholds. I'm wanting input about the best way to do dynamic thresholds. Anything at all is appreciated.

The report contains http error code counts and offending urls.

Why use thresholds? So the script which generates the report can intelligently send pages.

My plan is to calculate a normal/standard distribution based on a floating 24 hours worth of results.

Replies are listed 'Best First'.
Re: Dynamic thresholds in report
by muba (Priest) on Jan 04, 2013 at 22:53 UTC

    Dynamic how?

    Based on previous results? For example, if there are a lot of anomalies (unwanted http error codes, lots of offending urls), the threshold are tuned down so that not every anomaly triggers a page? Or if things have been quiet for the last couple of days, but suddenly in the last hour there's a peak of anomalies, you somehow want to reflect this in the value of your threshold? You'd have to define the possible scenarios and how the thresholds are to react to this. Then you'll have to work out a system that will detect these scenarios and apply the required dynamics. Then you'll have to implement that in code.

    Based on the time of the day? For example, the threshold vary from hour to hour — medium in the morning, high during the afternoon, medium again until midnight, and low from midnight until 7am. You could write a subroutine that looks at the time of the day and adjusts thresholds accordingly. Or your report script could accept threshold values as command line arguments, as in hourly-report-tool --http404=minimal --http500=low --http302=maximal --offurl=supermax and then set up cron tabs to call the tool with the right arguments at the right times.

    Based on the phase of the moon and a little divination? Sorry, I'm not into this kind of esoteric knowledge.

    Just kidding, of course. But what I mean to say is that, without a little more context or background information, it's hard to know what you mean with "dynamic." What kind of reports are you building exactly? What kind of information is collected? How is this information processed? What is reported? What does it mean to "intelligently send pages"? Do you want (your users) to be in control of the 'dynamicity' or do you want your tool to be able to adjust those thresholds yourself? What do you even mean by thresholds? How many are there? What do they regulate? How?

Re: Dynamic thresholds in report
by mhearse (Chaplain) on Mar 07, 2013 at 00:22 UTC
    I found what I needed here Using the standard deviation and the 689599 rule I had my dynamic thresholds.
    sub stdev{ my($data) = @_; if(@$data == 1){ return 0; } my $average = &average($data); my $sqtotal = 0; foreach(@$data) { $sqtotal += ($average-$_) ** 2; } my $std = ($sqtotal / (@$data-1)) ** 0.5; return $std; }

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1011729]
Approved by Perlbotics
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (6)
As of 2017-11-22 08:32 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (316 votes). Check out past polls.