Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

I'm try to speed up some data collection on a bunch of Windows pc's.

Basically I need to parse some info out of the the Event Log. I then munge some data from a bunch of files and then pass data back in an array of arrays.

My current programs takes about a minute per pc to run and I am testing about about 70pc's at the moment and could potentially need to run this on 1,000's.

Here's a stripped down version of the code.

my ($file_listing_pcs) = @ARGV; #slurp in file my $data_ref; while (<$file>) { my $pc = $_; if ( ping_it($pc) ) { ($data_ref) = process_pc($pc,$data_ref); } else { next; } } # post process data exit(0); sub process_pc { my ($pc, $data_ref) = @_; my ($event_log_data) = get_event_log_data($pc); my $dir_path = qq(//$pc/c\$/temp); ($data_ref) = process_dir($event_log_data,$dir_path,$data_ref) return($data_ref); } sub get_event_log_data { my ($pc) = @_; # play with Win32::EventLog return($event_log_data); } sub process_dir { my ($event_log_data,$dir_path,$data_ref) = @_; # read directory # get list of files to read # read files # plunder data # munge it about return($data_ref); }

After spending time googling, reading the docs and PerlMonks I just need to use Coro (I don't think I need AnyEvent but I'm not sure).

The biggest time suck is dealing with the Event Log.

My first naive attempt to use Coro is to just change the while loop that processes the list of pc's to be:

my @pids; while (<$file>) { my $pc = $_; if ( ping_it($pc) ) { push @pids, async{ ($data_ref) = process_pc($pc,$data_ref) }; cede; } else { next; } } $_->join for @pids; #data profit?

This didn't change the time it took to process my test horder at all.

I can't recall the last time I event pondering doing anything event related let alone thread related.

Any and all guidance much appreciated.

gizmo


In reply to Use Coro for reading EventLog on many pc's? by gizmo_mathboy

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others exploiting the Monastery: (8)
    As of 2014-08-29 23:44 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      The best computer themed movie is:











      Results (289 votes), past polls