Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Win32::EventLog searching the wrong logs

by FloydATC (Chaplain)
on May 02, 2014 at 06:07 UTC ( #1084734=perlquestion: print w/replies, xml ) Need Help??
FloydATC has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to extract Windows Event Logs, apply some filters to reduce the chatter, and then relay the result to a syslog server.

Unfortunately, while I can extract the "System" and "Application" logs with no problems and the "Security" log by running as Administrator, I have serious trouble accessing the "Setup" and "Forwarded Events" logs.

When searching the "Setup" and "Forwarded Events" logs, I get no error message but instead I get the "Application" log events. This leads me to believe that either there must be something odd about those two logs, I'm using the wrong name, there's something wrong with Win32::EventLog or Windows is playing a trick on me.

The platform is Windows 2012 Server R2, running Strawberry Perl 5.14.2 and Win32::EventLog 0.077

Does anyone else have experience with this module and can tell me what on earth is going on? If I can't figure this out, this whole script is pointless because what I'm really interested in are those Forwarded Events :-/

#!/perl use strict; use warnings; use Win32::EventLog; use DateTime; use Net::Syslog; my @servers = ( "MAN-EVENTLOG-01" ); my @logs = ( "Setup", # appears to work but actually reads from "App +lication" "System", # works as expected "Security", # works as expected for "administrator", other +wise returns 0 events "Application", # works as extected "Forwarded Events" # appears to work but actually reads from "App +lication" ); my %priority = ( EVENTLOG_ERROR_TYPE => 'error', EVENTLOG_WARNING_TYPE => 'warning', EVENTLOG_INFORMATION_TYPE => 'notice', EVENTLOG_AUDIT_SUCCESS => 'debug', EVENTLOG_AUDIT_FAILURE => 'informational' ); my %DEBUG = ( loop => 0, event => 0, rule => 0, syslog => 0, history => 1, ); my %last = (); my $HISTORY_FNAME = 'syslog-events.dat'; load_history(); my $need_save = 0; my %rules = (); # EventID processing rules go here as they are loaded while (1) { my $syslog = Net::Syslog->new( SyslogHost => '' ); foreach my $server (sort @servers) { foreach my $log (@logs) { my $handle = Win32::EventLog->new($log, $server) or die; if ($handle) { my $recs; print "$server $log Get number of events...\n" if $DEBUG{loop} +; $handle->GetNumber($recs); next unless $recs; my $base; print "$server $log Get oldest event...\n" if $DEBUG{loop}; $handle->GetOldest($base); print "$server $log has $recs events, first one is $base\n" if + $DEBUG{loop}; my $index = 0; if ($last{$server}->{$log} && $last{$server}->{$log} >= $base) + { my $skip = 1 + $last{$server}->{$log} - $base; $index = $skip; print "$server $log Skipping $skip already seen\n" if $DEBUG +{loop}; } while ($index < $recs) { my $hashref; my $discard = 0; # As demonstrated by Win32::EventLog docs, read one event by + record number if ($handle->Read(EVENTLOG_FORWARDS_READ|EVENTLOG_SEEK_READ, + $base+$index, $hashref)) { $hashref->{'_EventID'} = $hashref->{'EventID'} & 0x0000fff +f; # Seems to solve the "-2xxxxxxxxx" event IDs my ($timestamp) = unixtime_to_timestamp($hashref->{'TimeGe +nerated'}); $hashref->{'_DateTime'} = "$timestamp"; Win32::EventLog::GetMessageText($hashref); # This is where + the Registry & DLL nightmare takes place if ($DEBUG{event}) { # Dump the event $hashref for debugging foreach my $key (sort keys %{$hashref}) { print "\t$key: + $hashref->{$key}\n"; } } # Trim the host name, we're all friends here my ($host) = split(/\./, $hashref->{'Computer'}, 2); # Reduce the Message to a single line by replacing CR/LF w +ith spaces, then eliminate double spaces my $message = $hashref->{'Message'} || ''; $message =~ s/[\r\n]/ /g; $message =~ s/\s+/ /g; my $original = length($message); # Check for matching rule for this EventID # A rule consists of a short perl script that aims to redu +ce the size of $message # A rule may also set $discard to a true value, meaning 'd +o not syslog this event' # A rule may also wipe your hard disk or install SilverLig +ht so be careful # Example filename: # We keep track of "modified" timestamp to reload changed +rules on the fly my $rule = 'event-'.$hashref->{'_EventID'}.".pl"; if ($rules{$rule} || -e $rule) { # A rule exists for this EventID, load if necessary print "A rule exists for this event ID: $rule modified " +.(-M $rule)."\n" if $DEBUG{rule}; print "Cached version modified ".$rules{$rule}->{'modifi +ed'}."\n" if $rules{$rule}->{'modified'} && $DEBUG{rule}; if (!defined $rules{$rule}->{'modified'} || -M $rule < $ +rules{$rule}->{'modified'}) { # Rule is not cached or it has been modified print "Loading rule $rule\n" if $DEBUG{rule}; $rules{$rule}->{'code'} = ''; open(my $fh, $rule) || die "Error reading $rule: $!"; while (my $line = <$fh>) { $rules{$rule}->{'code'} .= $line; } close $fh; $rules{$rule}->{'modified'} = -M $rule; } # Now execute the rule using eval() print "Executing $rule:\n" if $DEBUG{rule}; eval $rules{$rule}->{'code'}; warn $@ if $@; # Perhaps syslog this? # If the rule set $discard to a true value then throw th +is message away next if $discard; } else { print "No rule exists for this event ID: $rule\n" if $DE +BUG{rule}; } # Time to assemble the actual syslog message and send it my $syslog_msg = "$message Log=".$log." EventID=".$hashref +->{'_EventID'}." EventType=".$hashref->{'EventType'}." Source=".$hash +ref->{'Source'}."\n"; if (length($message) < $original) { $syslog_msg .= " [f~".($original-length($message)."]"); +# Indicate that we tampered with this message } print $syslog_msg if $DEBUG{syslog}; # TODO: Set meaningful facility values somehow. Use the de +fault 'local5' for now. my $fac = 'local5'; my $pri = $priority{$hashref->{'EventType'}}; $syslog->send("$syslog_msg", Timestamp => $timestamp, Host + => $host, rfc3164 => 1, Facility => $fac, Priority => $pri ); # We logged an event, make a note of it for later and reme +mber we have to save that info $last{$server}->{$log} = $base+$index; $need_save = 1; } else { warn "Could not read event ".($index+$base)." from $server + log $log"; } $index++; } } else { warn "Error collecting $log log from $server"; } } } # We finished a pass. Save information if needed, then take a (short +) break save_history() if $need_save; $need_save = 0; sleep 1; } sub unixtime_to_timestamp { my $unixtime = shift; my $dt = DateTime->from_epoch( epoch => $unixtime ); $dt->set_time_zone( 'local' ); # $unixtime from Win32::EventLog appe +ars to be in localtime, not UTC. return sprintf("%s %2d %s", $dt->month_abbr(), $dt->day(), $dt->hms( +)); } sub load_history { print "Loading history to $HISTORY_FNAME\n" if $DEBUG{history}; open(my $fh, $HISTORY_FNAME) || warn "Error reading $HISTORY_FNAME: +$!"; while (my $line = <$fh>) { chomp $line; # Line should contain "SERVERNAME/LOGNAME key=value [key2=value3] +.. [keyN=valueN]" if ($line =~ /^(.+?)\/(.+?) (([a-z]+\=[a-z0-9]+\s*)+)/i) { my ($server, $log, $pairs) = ($1, $2, $3); print "server=$server, log=$log, pairs: $pairs\n" if $DEBUG{hist +ory}; foreach my $pair (split(/\s/, $pairs)) { my ($key, $value) = split(/\=/, $pair, 2); # For now, the only supported key name is "last" but format is + extensible if ($key eq 'last') { $last{$server}->{$log} = $value; } } } else { print "Failed to parse '$line'\n" if $DEBUG{history}; } } close $fh; } sub save_history { print "Saving history to $HISTORY_FNAME\n" if $DEBUG{history}; my $tempname = $HISTORY_FNAME . '.temp'; open(my $fh, '>', $tempname) || warn "Error writing $tempname: $!"; foreach my $server (sort keys %last) { foreach my $log (sort keys %{$last{$server}}) { print $fh "$server/$log last=".$last{$server}->{$log}."\n"; } } close $fh; rename($tempname, $HISTORY_FNAME); } END { save_history() if $need_save; }
-- FloydATC

Time flies when you don't know what you're doing

Replies are listed 'Best First'.
Re: Win32::EventLog searching the wrong logs
by Anonymous Monk on May 02, 2014 at 07:11 UTC

    Try and take a look at MyEventLog for Win32::EventLog, Re: Translating Win32 EventLog Category

    I might add this constructor

    sub Win32::EventLog::new { package Win32::EventLog; die "usage: PACKAGE->new(SOURCENAME[, SERVERNAME])\n" unless @_ > +1; my ( $class, $source, $server ) = @_; my $handle; my $error; # Create new handle if ( $source !~ /\\/ ) { my $ret = OpenEventLog( $handle, $server, $source ); $ret or $error = [ [ int $!, $!], [int $^E, $^E] ]; } else { my $ret = OpenBackupEventLog( $handle, $server, $source ); $ret or $error = [ [ int $!, $!], [int $^E, $^E] ]; } return bless { handle => $handle, Source => $source, Computer => $server, error => $error, } => $class; }

    I also might add a check in this constructor for the behaviour you experience that is describe below (opens Application when it can't find what you ask for)

    lpEvtLog->hLog = OpenEventLogA(lpszUNCServerName,lpszSourceName);
    OpenEventLog function (Windows)
     lpSourceName [in]
    The name of the log.
    If you specify a custom log and it cannot be found, the event logging service opens the Application log; however, there will be no associated message or category string file.

    The linked example ( Querying for Event Information (Windows) ) says The source name (provider) must exist as a subkey of Application.

    Regarding that comment one answer in How to open system event log? says For Vista or higher use EvtQuery, EvtNext etc to query the XML based event logs. and links to Querying for Events (Windows) using EvtQuery

    Other answer links to WindowsNT Event Log Viewer - CodeProject which talks about the win32 registry

    I've not deciphered the registry clues , but you might be able to :)

    Enjoy :)

      I should probably have pointed out that I don't enjoy working with Windows and therefore tend to avoid it, mostly because of its tendency to do braindead stuff like what you referred to;

      If you specify a custom log and it cannot be found, the event logging service opens the Application log; however, there will be no associated message or category string file.

      Well, thanks! Basically, this confirms my suspicion that I'm asking for the wrong thing and Windows, instead of returning an error message like any civilized OS would, decides to "help" by giving me something completely different. (Stupid joke about toilet paper vs. sand paper goes here)

      After reading the articles you linked, I ventured into the most unholy land of the Windows Registry and found the following keys:

      HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Applicat +ion HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Hardware +Events HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Internet + Explorer HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Key Mana +gement Service HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Security HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\System HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Windows +Powershell

      Uh... okay? So what about the "Setup" and "Forwarded Events" that very clearly show up in the Event Log viewer? There's something going on here that I just don't understand.

      Examining the Event Log viewer closely, you may notice that those other logs have a different icon than the ones I can't read (which do not appear in the Registry). I have no idea why, but there's probably a connection.

      Further, examining the "Properties" of each log, I see the following "Log paths":

      %SystemRoot%\System32\Winevt\Logs\Application.evtx %SystemRoot%\System32\Winevt\Logs\Security.evtx %SystemRoot%\System32\Winevt\Logs\Setup.evtx %SystemRoot%\System32\Winevt\Logs\System.evtx %SystemRoot%\System32\Winevt\Logs\ForwardedEvents.evtx

      If I try to open those files (obviously replacing "%SystemRoot%" with "C:"), Win32::EventLog->new() succeeds but ->GetNumber returns undef. Probably because they're not meant to be accessed that way.

      I don't have a clue how to fix this...

      -- FloydATC

      Time flies when you don't know what you're doing

        Do you have a
        HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Applicat +ion\Setup

        Where do you find  Setup.evtx in the registry?

        Hmmm Eventlog Key (Windows)

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1084734]
Approved by GrandFather
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (2)
As of 2016-09-25 07:18 GMT
Find Nodes?
    Voting Booth?
    Extraterrestrials haven't visited the Earth yet because:

    Results (465 votes). Check out past polls.