RFC Win32::Event2Log ..gimme back my logfiles

[Discipulus]

Hello monks,

introduction

I haved produced my first, at least in my intention, serious module: Win32::Event2Log for the moment on github (current version). I tried to follow all best practices for module creation (a long read..) and I announced on prepan last week but I had no comment back.

The windows Event Viewer, in my experience, it's good just to lead you to a carpal tunnel syndrome so in the past I have arranged a bounch of Perl programs to inspect it's registries using Win32::EventLog to trigger some action. This approach it's difficult and everytime I had to restart from scratch. So I had the, cool, idea to write an engine that read events and, if a given rule matches, write them on a plain logfile, then the road it's plain for a Perl programmer.

presentation

Essentially what the module do, as it is explained in it's POD, is using Win32::EventLog and parsing windows events and writing them to plain logfiles. This module is rule based: a rule it's a minimal set of conditions to be met to write an entry to a logfile. You must add valid rules before starting the engine. Once started, the engine will check events every x seconds (specified using interval argument) and for every registry (System, Application, Security, Installation or a user defined one) that is requested at least in one rule will check for an event's source specified and optionally for some text contained in the event's description.

The resulting engine it's designed to survive to shutdowns and user's interruption issued with CTRL-C in the console or a kill of the PID: next run of the program will read just unparsed events on. This is achieved storing numbers of last event read (for each registry) in a file specified with the lastreadfile argument.

A simple example of it's usage is (as in the example section of the module) is the following:

use strict;
use warnings;

use Win32::Event2Log;

my $main_log = $0.'.mainlog.log';
my $last_numbers_log = $0.'.last_numbers.log';
my $sys_errors_log = $0.'.System_err_warn.log';


my $engine = Win32::Event2Log->new(     
                                   interval        => 60,
                                   endtime         => 0,
                                   mainlog         => $main_log,   
                                   verbosity       => 2,
                                   lastreadfile=> $last_numbers_log,  
+             
);

$engine->add_rule (
                    registry => 'System',
                    eventtype=> 'error|warning',
                    source   => qr/./,
                    log              => $sys_errors_log, 
                    name     => 'System errors and warnings',         
+              
);

$engine->start;

But since I've always produced modules as private containers of almost related functions, I'm a bit a newbie in regard to CPAN standards. Infact I plan to release it on CPAN soon, but not before having listen your advices. So my Request for Comments are:

RFC

1) name:
I think the Win32 is naturally the correct one but what about Event2Log ? it seemed the best choice for me

2) testing:
in this field I read a lot in the past but, my sin, practiced almost no times.. I've done my best writing 01-basic.t (here(current version)). How the test can be improved? I need to bail out in the test if $^O is not MSWIn32? I tested only the public methods I offer: should I test also private functions?

3) design and enanchemts:
Even if the module runs well enough in my tests on various scenarios, I already plan to modify it. Infact actually the core of the engine is a while (1) {.. loop where new events are checked and rules applied (you can see it here(current version)).

I plan to abstract the reading part, maybe adding a Win32::Event2Log::Reader submodule. Infact I want also the user to choose if use Win32::EventLog as reader or a wrapper around wevtutil.exe that I plan to write soon. How achieve this? Having Win32::Event2Log::Reader using Win32::EventLog by default and Win32::Event2Log::Reader::Wevtutil subclassing Win32::Event2Log::Reader ? What is the cleanest design for such modification? Which tests I must add?

4) design of an eventual Win32::Event2Log::Reader :
This seemed to me a good use for an iterator: $reader->next will replace a lot of odd code in my current module. The fact I'm wondering about is for the wrapper around the system call wevtutil.exe

Since system calls are expensive I plan the first time the iterator it's initialized, to query all previous events and return them one at time: the array of events this first time can be many Mb and in successive calls possibly just few bytes. This seems against the good design of a ligth sized iterator. It's justificable to avoid possibly many system calls?

Thanks for reading.

L*

[Anonymous Monk]

Hi,

congratulations brave programmer

This seems like a good project for you to think out loud, some more, cause you're already doing it, congratulations!

For now concentrate on how a user would use this thing, forget about backends and implementation details, write a few dozen user programs , a few a day, and see what you like the most, concentrate on verbs/actions the user would perform, the programs don't have to actually work they just have to be written like they should work like a user would want them to work

Ask yourself why would anyone use your module instead of EventLog? Answer the question as many times as you can (write a program), you're trying to attract/catch users for your module(?or app?) ... is your bait delicious enough?

If you're only writing for yourself (valid reason) then you don't have to get imaginative with api names, just do the first thing that pops into your head and you can write -- one-offs are easy

For some good/bad ideas take a look at Re: Win32::EventLog searching the wrong logs (MyEventLog for Win32::EventLog, Re: Translating Win32 EventLog Category ) and POE::Component::Win32::EventLog - A POE component that provides non-blocking access to Win32::EventLog.

Some comments about your code

local is used for temporarily overriding globals like this

local $Win32::EventLog::GetMessageText = 1;

This is a "Red flag" localtime ... $year + 1900, and an extremely ridiculous one when on the next line you have  DateTime->new but we've all been there and you probably know just need more perl DateTime FAQ ( hint DateTime->strftime, DateTime::Format::Strptime , DateTime::Format::Human::Duration, DateTime::Format::Natural, )

Pod convention is uppercase for sections and

=head1 NAME

Module::Name - one line description

=head1 DESCRIPTION

...

Regarding the name?

Well name has to follow from function, and since the function of your module is kinda up in the air ...

See Namespace/advice for new CPAN modules for Thai & Lao, Re^2: Naming a module that handles SIP2 (library protocol stopwords), Modules: Creation, Use, and Abuse , Re: RFC: Automatic logger module,

yes Win23 might belong in the name

yes Event might belong in the name

No "2" does not belong in the name, not as a substitute for "To" in "ThisToThat". Don't be like "Pod::2::DocBook" thats maximum terrible :) Exception is the terribly named "POD2::IT - Italian translation of Perl core documentation " series cause it should have been lowercase "it" :D

No, "log" does not belong in the name. Sure you're making log files, and "events" are logs, but you're not log framework anything .... use the module DESCRIPTION to help users find your log file creating win32 event log reading parsing module :D

Is your module really an application? Are you sure you're not really working up to App::Win32EventsFilter? Thats where the research/trialerror/userprograms come in .... if 80% of the use cases, the rules, would look identical in a myevents.pl or a commandline  win32eventsfilter --add "snackarooony" then the name kinda writes itself

 

I'm sure there is more stuff I could say if I spent more time thinking about it, but its like tennis :)

[Discipulus]

Many thanks Anonymous Monk for your review, full of good links and suggestions!

let me comment back..

> write a few dozen user programs ... concentrate on verbs/actions the user would perform ..

I'm already doing this: I need to use my module already so I'm testing on various scenarios.

> Ask yourself why would anyone use your module instead of EventLog

Well I'm confident enough with my lure: it abstract all the boring part of using Win32::EventLog as counting events, get them back in the right way (consider the module has a bit weird interface, not easy to use under strict and warnings, populating globals variables like in  my $base; $handle->GetOldest($base) )

More: my engine tracks and logs last numbers read for each registry, so if you restart the program you just read newer events: this is achieved via the lastreadfile option in the engine contruction.

It also automatically open and close your destination logfiles with an autoflushed handle: this is the only way I found to have no buffering done by the OS.

Finally providing a callback in the rule creation lets the final user to really do what they want. Rethinking about this probably it's worth to be redisigned: I now constrain the rule to have a destination logfile associated and optionally a format callback.

Perhaps it will be better to let logfile optional if a callback it's specified: so a better name will be action or callback instead of format This can lead to an extreme flexibility: no more the output to a logfile but, for example: if an event is found write a mail and restart a service.

The above abstraction will affect the name too: Win32::EventTrigger or Win32::Event::Engine ?

Links provided are useful (apart from the la-berre one unavailable..) and you are probably the same Anonymous Monk of Re: Win32::EventLog not returning all events? and if so what a pity you are no a named one to track of your post.. anyway thanks!

I have add a FURTHER READINGS section (and put them all uppercase..) to my pod pointing to these and more articles.

about code style

I have localized my $Win32::EventLog::GetMessageText occurence: as I said probably i'll strip out the reading part into a Win32::Event2Log::Reader submodule. The best place for this variable will be just after the, eventual, require Win32::EventLog one. Anyway this means you read my code.. someone read my code!! I must tell to my wife! ;=)

The red flag about dates is received, mea culpa, I'll clean the example (following your hints) before publishing the module.

about the name

ok no 2 in the name: Win32::EventFilter (without the S?) sounds good. Considering the above abstraction I mentioned probably Win32::Event::Engine will be more appropriate, no?

Anyway no, I do not think i'm writing an App::* like think. It's more similar modular engine, rule based. As side note, during my preliminar readings, I read perlmodstyle and I noticed i broke Designing your API suggestions, precisely Separate functionality from output one. But what can I do? The engine must show itself some output (on verbosity at least). It is designed to run forever (or to terminate at endtime ..) so it's like a webserver so some output directly from the module can be useful.

Thanks again for your help

L*

update Feb 09: asked here how to subclass and separate module functionalities.

[alexander_lunev]

Good day to you, Disciplus!

If I understand you correctly, you want Win32 analog of swatch perl utility? I beg your pardon if you already know about the swatch, but i think that maybe you should look in it to see some new or good-old for yourself.

[soonix]

Sounds interesting. However, I think the swatch project summary would be a better link