Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re^4: Finding local maxima/minima in noisy, pediodic data

by kikuchiyo (Pilgrim)
on Oct 12, 2009 at 18:39 UTC ( #800768=note: print w/replies, xml ) Need Help??

in reply to Re^3: Finding local maxima/minima in noisy, pediodic data
in thread Finding local maxima/minima in noisy, pediodic data

The data file posted on pastebin contains data from two channels recorded during the same measurement. 2nd col. vs. 1st col. is one dataset and 3rd vs. 1st. is the other.

So you are plotting the right values.

You are also correct about misses and false hits.
Anyway, here is your first graph with missed peaks marked with ugly blue arrows and false peaks marked with smaller but equally ugly gray arrows.
  • Comment on Re^4: Finding local maxima/minima in noisy, pediodic data

Replies are listed 'Best First'.
Re^5: Finding local maxima/minima in noisy, pediodic data
by BrowserUk (Pope) on Oct 14, 2009 at 16:39 UTC

    How automated does this have to be? In other words, is it acceptable to your process to supply some initial parameters, derived by inspection, to the processing?

    Because your dataset has the signal you are seeking to isolate, superimposed upon a much lower frequency, but still relatively high volume "carrier wave", with the whole thing overlain by a considerable amount of high frequency noise; single pass filtering--low pass; high pass; band-pass; simple smoothing--don't achieve the goal. And the asymmetric nature of the saw-tooth, combined with the variablity of the frequency, and the strength of both the random noise, and the underlaying carrier wave, makes isolating the required signal quite complex.

    However, if all the waveforms you wish to analyse fit the same basic structure; or if it is acceptable to supply some initial parameters, derived by inspection, it considerably simplifies things.

    For example, this graph shows your sample data processed using 4 input parameters:

    1. -PERIOD=75;

      The approximate duration of 1 cycle.

      Inspecting a simple plot of the data, you can see that peak 2 coincides with 100, and peak 5 with 400; giving ( 400 - 100 ) / 4 = 75. It's not a perfect fit, and the frequency drifts over the sample, but it's easy to derive and greatly simplifies the process.

    2. -IDIR=0;

      The initial direction (trend) of the first cycle in the data is down; (set IDIR=1 for up).

      This can be derived by comparing the relative ordering of the extremas found within the first PERIOD, but it is so easy for the human eye to see, it's better done by inspection.

      It also doesn't make a huge difference if you specify it wrong, as the algorithm syncs after the first cycle, but it does ensure a clean first cycle.

    3. -ATTACK=3;

      This indicates tha the attack of the waveform is fast. Ie. The next maxima occurs within 1/8th (1/2**3) of a cycle from the preceding minima.

      It is used to suppress consideration of a possible maxima until at least (PERIOD >> 3) time units have been collated. It helps exclude single point high-peak anomolies.

    4. -DECAY=1;

      This indicates that the decay of the waveform is qute slow. Ie. The next minima occurs not less than 1/2 cycle (1/2**1) after the preceding maxima.

      It is used to delay consideration of local minima until we've got at least half way through the cycle.

    On the graph linked above, I've displaced the second dataset for clarity. The keys are:

    • Raw data, red: first dataset; green: second dataset.
    • Respectively, blue & yellow: moving average (used for crossover detection).
    • Respectively, purple & cyan: resultant square waves.

    The algorithm now requires two passes for each dataset. The additional one to compute the moving average. This can be done in-line with the second pass, but it considerably complicates things.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://800768]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2018-03-17 11:00 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (223 votes). Check out past polls.