Win32::EventLog searching the wrong logs

[FloydATC]

I'm trying to extract Windows Event Logs, apply some filters to reduce the chatter, and then relay the result to a syslog server.

Unfortunately, while I can extract the "System" and "Application" logs with no problems and the "Security" log by running as Administrator, I have serious trouble accessing the "Setup" and "Forwarded Events" logs.

When searching the "Setup" and "Forwarded Events" logs, I get no error message but instead I get the "Application" log events. This leads me to believe that either there must be something odd about those two logs, I'm using the wrong name, there's something wrong with Win32::EventLog or Windows is playing a trick on me.

The platform is Windows 2012 Server R2, running Strawberry Perl 5.14.2 and Win32::EventLog 0.077

Does anyone else have experience with this module and can tell me what on earth is going on? If I can't figure this out, this whole script is pointless because what I'm really interested in are those Forwarded Events :-/

#!/perl

use strict;
use warnings;
use Win32::EventLog;
use DateTime;
use Net::Syslog;

my @servers = (
  "MAN-EVENTLOG-01"
);

my @logs = (
  "Setup",              # appears to work but actually reads from "App
+lication"
  "System",             # works as expected
  "Security",           # works as expected for "administrator", other
+wise returns 0 events
  "Application",        # works as extected
  "Forwarded Events"    # appears to work but actually reads from "App
+lication"
);

my %priority = (
  EVENTLOG_ERROR_TYPE           => 'error',
  EVENTLOG_WARNING_TYPE         => 'warning',
  EVENTLOG_INFORMATION_TYPE     => 'notice',
  EVENTLOG_AUDIT_SUCCESS        => 'debug',
  EVENTLOG_AUDIT_FAILURE        => 'informational'
);



my %DEBUG = (
  loop => 0,
  event => 0,
  rule => 0,
  syslog => 0,
  history => 1,
);

my %last = ();
my $HISTORY_FNAME = 'syslog-events.dat';
load_history();
my $need_save = 0;

my %rules = (); # EventID processing rules go here as they are loaded

while (1) { 
  my $syslog = Net::Syslog->new( SyslogHost => '10.80.8.252' );
  foreach my $server (sort @servers) {
    foreach my $log (@logs) {
      my $handle = Win32::EventLog->new($log, $server) or die;
      if ($handle) {
        my $recs;
        print "$server $log Get number of events...\n" if $DEBUG{loop}
+;
        $handle->GetNumber($recs);
        next unless $recs;
        my $base;
        print "$server $log Get oldest event...\n" if $DEBUG{loop};
        $handle->GetOldest($base);
        print "$server $log has $recs events, first one is $base\n" if
+ $DEBUG{loop};
        my $index = 0;
        if ($last{$server}->{$log} && $last{$server}->{$log} >= $base)
+ {
          my $skip = 1 + $last{$server}->{$log} - $base;
          $index = $skip;
          print "$server $log Skipping $skip already seen\n" if $DEBUG
+{loop};
        }
        while ($index < $recs) {
          my $hashref;
          my $discard = 0;
          
          # As demonstrated by Win32::EventLog docs, read one event by
+ record number
          if ($handle->Read(EVENTLOG_FORWARDS_READ|EVENTLOG_SEEK_READ,
+ $base+$index, $hashref)) {
            $hashref->{'_EventID'} = $hashref->{'EventID'} & 0x0000fff
+f; # Seems to solve the "-2xxxxxxxxx" event IDs
            my ($timestamp) = unixtime_to_timestamp($hashref->{'TimeGe
+nerated'});
            $hashref->{'_DateTime'} = "$timestamp";
            Win32::EventLog::GetMessageText($hashref); # This is where
+ the Registry & DLL nightmare takes place
            if ($DEBUG{event}) {
              # Dump the event $hashref for debugging
              foreach my $key (sort keys %{$hashref}) { print "\t$key:
+ $hashref->{$key}\n"; }
            }
            
            # Trim the host name, we're all friends here
            my ($host) = split(/\./, $hashref->{'Computer'}, 2);
            
            # Reduce the Message to a single line by replacing CR/LF w
+ith spaces, then eliminate double spaces
            my $message = $hashref->{'Message'} || '';
            $message =~ s/[\r\n]/ /g;
            $message =~ s/\s+/ /g;
            my $original = length($message);
            
            # Check for matching rule for this EventID
            # A rule consists of a short perl script that aims to redu
+ce the size of $message
            # A rule may also set $discard to a true value, meaning 'd
+o not syslog this event'
            # A rule may also wipe your hard disk or install SilverLig
+ht so be careful
            # Example filename: event-7001.pl
            # We keep track of "modified" timestamp to reload changed 
+rules on the fly
            my $rule = 'event-'.$hashref->{'_EventID'}.".pl";
            if ($rules{$rule} || -e $rule) {
              # A rule exists for this EventID, load if necessary
              print "A rule exists for this event ID: $rule modified "
+.(-M $rule)."\n" if $DEBUG{rule};
              print "Cached version modified ".$rules{$rule}->{'modifi
+ed'}."\n" if $rules{$rule}->{'modified'} && $DEBUG{rule};
              if (!defined $rules{$rule}->{'modified'} || -M $rule < $
+rules{$rule}->{'modified'}) {
                # Rule is not cached or it has been modified
                print "Loading rule $rule\n" if $DEBUG{rule};
                $rules{$rule}->{'code'} = '';
                open(my $fh, $rule) || die "Error reading $rule: $!";
                while (my $line = <$fh>) {
                  $rules{$rule}->{'code'} .= $line;
                }
                close $fh;
                $rules{$rule}->{'modified'} = -M $rule;
              }
              # Now execute the rule using eval()
              print "Executing $rule:\n" if $DEBUG{rule};
              eval $rules{$rule}->{'code'};
              warn $@ if $@; # Perhaps syslog this?
              
              # If the rule set $discard to a true value then throw th
+is message away
              next if $discard;
              
            } else {
              print "No rule exists for this event ID: $rule\n" if $DE
+BUG{rule};
            }
            
            # Time to assemble the actual syslog message and send it
            my $syslog_msg = "$message Log=".$log." EventID=".$hashref
+->{'_EventID'}." EventType=".$hashref->{'EventType'}." Source=".$hash
+ref->{'Source'}."\n";
            if (length($message) < $original) {
              $syslog_msg .= " [f~".($original-length($message)."]"); 
+# Indicate that we tampered with this message
            }
            print $syslog_msg if $DEBUG{syslog};
  
            # TODO: Set meaningful facility values somehow. Use the de
+fault 'local5' for now.
            my $fac = 'local5';
            my $pri = $priority{$hashref->{'EventType'}};
  
            $syslog->send("$syslog_msg", Timestamp => $timestamp, Host
+ => $host, rfc3164 => 1, Facility => $fac, Priority => $pri );
            
            # We logged an event, make a note of it for later and reme
+mber we have to save that info
            $last{$server}->{$log} = $base+$index;
            $need_save = 1;
          } else {
            warn "Could not read event ".($index+$base)." from $server
+ log $log";
          }
          $index++;
        }
      } else {
        warn "Error collecting $log log from $server";
      }
    }
  }
  
  # We finished a pass. Save information if needed, then take a (short
+) break
  save_history() if $need_save;
  $need_save = 0;
  sleep 1;
}

sub unixtime_to_timestamp {
  my $unixtime = shift;
  my $dt = DateTime->from_epoch( epoch => $unixtime );
  $dt->set_time_zone( 'local' ); # $unixtime from Win32::EventLog appe
+ars to be in localtime, not UTC.
  return sprintf("%s %2d %s", $dt->month_abbr(), $dt->day(), $dt->hms(
+));
}

sub load_history {
  print "Loading history to $HISTORY_FNAME\n" if $DEBUG{history};
  open(my $fh, $HISTORY_FNAME) || warn "Error reading $HISTORY_FNAME: 
+$!";
  while (my $line = <$fh>) {
    chomp $line;
    # Line should contain "SERVERNAME/LOGNAME key=value [key2=value3] 
+.. [keyN=valueN]"
    if ($line =~ /^(.+?)\/(.+?) (([a-z]+\=[a-z0-9]+\s*)+)/i) {
      my ($server, $log, $pairs) = ($1, $2, $3);
      print "server=$server, log=$log, pairs: $pairs\n" if $DEBUG{hist
+ory};
      foreach my $pair (split(/\s/, $pairs)) {
        my ($key, $value) = split(/\=/, $pair, 2);
        # For now, the only supported key name is "last" but format is
+ extensible
        if ($key eq 'last') {
          $last{$server}->{$log} = $value;
        }
      }
    } else {
      print "Failed to parse '$line'\n" if $DEBUG{history};
    }
  }
  close $fh;
}

sub save_history {
  print "Saving history to $HISTORY_FNAME\n" if $DEBUG{history};
  my $tempname = $HISTORY_FNAME . '.temp';
  open(my $fh, '>', $tempname) || warn "Error writing $tempname: $!";
  foreach my $server (sort keys %last) {
    foreach my $log (sort keys %{$last{$server}}) {
      print $fh "$server/$log last=".$last{$server}->{$log}."\n";
    }
  }
  close $fh;
  rename($tempname, $HISTORY_FNAME);
}

END {
  save_history() if $need_save;
}

[Anonymous Monk]

Try and take a look at MyEventLog for Win32::EventLog, Re: Translating Win32 EventLog Category

I might add this constructor

sub Win32::EventLog::new {
    package Win32::EventLog;
    die "usage: PACKAGE->new(SOURCENAME[, SERVERNAME])\n" unless @_ > 
+1;
    my ( $class, $source, $server ) = @_;
    my $handle;
    my $error;
    # Create new handle
    if ( $source !~ /\\/ ) {
        my $ret = OpenEventLog( $handle, $server, $source );
        $ret or $error = [ [ int $!, $!], [int $^E, $^E] ];
    }
    else {
        my $ret = OpenBackupEventLog( $handle, $server, $source );
        $ret or $error = [ [ int $!, $!], [int $^E, $^E] ];
    }
    return bless {
        handle   => $handle,
        Source   => $source,
        Computer => $server,
        error    => $error,
    } => $class;
}

I also might add a check in this constructor for the behaviour you experience that is describe below (opens Application when it can't find what you ask for)

https://metacpan.org/source/JDB/Win32-EventLog-0.077/EventLog.xs

lpEvtLog->hLog = OpenEventLogA(lpszUNCServerName,lpszSourceName);

OpenEventLog function (Windows)

 lpSourceName [in]
The name of the log.
If you specify a custom log and it cannot be found, the event logging service opens the Application log; however, there will be no associated message or category string file.

The linked example ( Querying for Event Information (Windows) ) says The source name (provider) must exist as a subkey of Application.

Regarding that comment one answer in How to open system event log? says For Vista or higher use EvtQuery, EvtNext etc to query the XML based event logs. and links to Querying for Events (Windows) using EvtQuery

Other answer links to WindowsNT Event Log Viewer - CodeProject which talks about the win32 registry

I've not deciphered the registry clues , but you might be able to :)

Enjoy :)

[FloydATC]

I should probably have pointed out that I don't enjoy working with Windows and therefore tend to avoid it, mostly because of its tendency to do braindead stuff like what you referred to;

If you specify a custom log and it cannot be found, the event logging service opens the Application log; however, there will be no associated message or category string file.

Well, thanks! Basically, this confirms my suspicion that I'm asking for the wrong thing and Windows, instead of returning an error message like any civilized OS would, decides to "help" by giving me something completely different. (Stupid joke about toilet paper vs. sand paper goes here)

After reading the articles you linked, I ventured into the most unholy land of the Windows Registry and found the following keys:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Applicat
+ion
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Hardware
+Events
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Internet
+ Explorer
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Key Mana
+gement Service
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Security
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\System
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Windows 
+Powershell

Uh... okay? So what about the "Setup" and "Forwarded Events" that very clearly show up in the Event Log viewer? There's something going on here that I just don't understand.

Examining the Event Log viewer closely, you may notice that those other logs have a different icon than the ones I can't read (which do not appear in the Registry). I have no idea why, but there's probably a connection.

Further, examining the "Properties" of each log, I see the following "Log paths":

%SystemRoot%\System32\Winevt\Logs\Application.evtx
%SystemRoot%\System32\Winevt\Logs\Security.evtx
%SystemRoot%\System32\Winevt\Logs\Setup.evtx
%SystemRoot%\System32\Winevt\Logs\System.evtx
%SystemRoot%\System32\Winevt\Logs\ForwardedEvents.evtx

If I try to open those files (obviously replacing "%SystemRoot%" with "C:"), Win32::EventLog->new() succeeds but ->GetNumber returns undef. Probably because they're not meant to be accessed that way.

I don't have a clue how to fix this...

[Anonymous Monk]

Do you have a

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Applicat
+ion\Setup

key?

Where do you find  Setup.evtx in the registry?

Hmmm Eventlog Key (Windows)