Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Dear monks,

The Code

I have some code which resembles the following:

sub remove_unwanted { my ( $self, @objects ) = @_; foreach my $filter ($self->filters) { @objects = $self->$filter(@objects); last unless @objects; } return @objects; }

There might be thousands of @objects and tens of filters. The filter will return zero or more objects. The number of filters is static (for a given software release), but the number of objects will always change (duh).

The Problem

This is some of the hottest code on a high performance system spread across 75 servers. It needs to run fast, fast, fast. The order in which the filters run can impact this greatly. I've had anywhere between 7% to 41% performance speedups by choosing the order carefully. This means that the slowest filters should usually run later and filters which filter out the most @objects should usually run sooner. However, what if we have a slow filter which filters out many objects? Or a fast filter which filters virtually nothing? Further, the criteria on which things get filtered is very complex and rapidly changes and requires that we use real-world data to determine what's really faster.

What Solution?

At first glance, this really looked like a candidate for genetic programming. One or two of our servers could run this code, instrumented to keep mutating the order of the filters until we find a "fast enough" solution.

I don't want to go down that route if someone can suggest a much easier way. On the other hand, if you think a genetic algorithm is the way to go, I wouldn't mind seeing how you would approach it.


In reply to Evolving a faster filter? by Ovid

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others musing on the Monastery: (10)
    As of 2015-07-31 00:49 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









      Results (274 votes), past polls