Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I'm try to speed up some data collection on a bunch of Windows pc's.

Basically I need to parse some info out of the the Event Log. I then munge some data from a bunch of files and then pass data back in an array of arrays.

My current programs takes about a minute per pc to run and I am testing about about 70pc's at the moment and could potentially need to run this on 1,000's.

Here's a stripped down version of the code.

my ($file_listing_pcs) = @ARGV; #slurp in file my $data_ref; while (<$file>) { my $pc = $_; if ( ping_it($pc) ) { ($data_ref) = process_pc($pc,$data_ref); } else { next; } } # post process data exit(0); sub process_pc { my ($pc, $data_ref) = @_; my ($event_log_data) = get_event_log_data($pc); my $dir_path = qq(//$pc/c\$/temp); ($data_ref) = process_dir($event_log_data,$dir_path,$data_ref) return($data_ref); } sub get_event_log_data { my ($pc) = @_; # play with Win32::EventLog return($event_log_data); } sub process_dir { my ($event_log_data,$dir_path,$data_ref) = @_; # read directory # get list of files to read # read files # plunder data # munge it about return($data_ref); }

After spending time googling, reading the docs and PerlMonks I just need to use Coro (I don't think I need AnyEvent but I'm not sure).

The biggest time suck is dealing with the Event Log.

My first naive attempt to use Coro is to just change the while loop that processes the list of pc's to be:

my @pids; while (<$file>) { my $pc = $_; if ( ping_it($pc) ) { push @pids, async{ ($data_ref) = process_pc($pc,$data_ref) }; cede; } else { next; } } $_->join for @pids; #data profit?

This didn't change the time it took to process my test horder at all.

I can't recall the last time I event pondering doing anything event related let alone thread related.

Any and all guidance much appreciated.


In reply to Use Coro for reading EventLog on many pc's? by gizmo_mathboy

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others perusing the Monastery: (7)
    As of 2018-05-25 09:15 GMT
    Find Nodes?
      Voting Booth?