Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Re: Finding local maxima/minima in noisy, pediodic data

by kikuchiyo (Monk)
on Oct 14, 2009 at 17:17 UTC ( #801134=note: print w/replies, xml ) Need Help??

in reply to Finding local maxima/minima in noisy, pediodic data

Thank you all for your comments.

I ended up doing two smoothing passes before I attempt to find any extrema: one is smoothed twice by a running average with a window of 1.1*$period (which I find with a simplified version of the autocorrelation func.), while the second is smoothed with a running median algorithm. The first (almost) completely removes the pulses, leaving only the "carrier wave" in place, while the second preserves the shape of the pulses but removes the spiky noise from them.

This gives me a robust crossover detection: if the running median is higher than the large scale smoothed wave, then I have to look for a maximum, else a minimum.

This algorithm is somewhat complex, however, it is (or seems to be) robust enough.

Ultimately, this algorithm will be the core of a Gtk2 GUI application that lets the user (my boss) visualize the data, find the peaks and mark/move/delete/add/export them at will. However, I'll write this later.

As for Perl not being the best tool for this particular problem, or a particular class of DSP-related problems -- well, I hesitate to say it out loud on this hallowed ground, but yes, quite likely that is true. There maybe other languages, platforms or tools which may be more efficient or easier to use for this problem than Perl; however, Perl is something I'm comfortable with. I happen to like Perl. :)

There is also the factor that I need my application to be portable, specifically, I develop it on linux but it'll have to run on windows. It is my experience that this works with Perl; I wouldn't be sure about other languages (for example, I never wrote anything for Windows in C).

Additionally, Perl is free, which is important to me for both practical and ethical standpoints.
  • Comment on Re: Finding local maxima/minima in noisy, pediodic data

Replies are listed 'Best First'.
Re^2: Finding local maxima/minima in noisy, pediodic data
by BrowserUk (Pope) on Oct 14, 2009 at 21:36 UTC

    You have your solution, but are you aware that there is an anomoly in your data (n=47.00) that doesn't stand up to statistical analysis? It's more than just a random discontinuity, it's almost like a chunk of data was deleted from the set.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://801134]
[marto]: I saw this robot the other day, pretty fancy stuff. Things are pretty good jedikaiti, improving :) How are things with you?
[jedikaiti]: Doing OK, but a little achy this AM - I think the cats were using me as a wrestling mat and mattress while I was asleep last night.
[marto]: I can relate to that somewhat, I'm a light sleeper and frequently get woken up by kids doing much the same as your cats
[jedikaiti]: Yea, the kids (& the cats) are good at that

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (12)
As of 2017-07-25 15:10 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (374 votes). Check out past polls.