Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re^2: Creating thread to call Subroutines parallely

by kar_thik82 (Novice)
on Feb 21, 2013 at 14:14 UTC ( #1019953=note: print w/ replies, xml ) Need Help??


in reply to Re: Creating thread to call Subroutines parallely
in thread Creating thread to call Subroutines parallely

This is the complete code. The problem i am facing is sometimes. I am getting an error in the line while writing to the output file app_output.txt Nothing is writing to the file it is empty.

This is the error i am getting since the file app_output.txt is empty couldn't read C:\log_parsing\app_output.txt - at log_windows_new.pl line 444.

#!/usr/bin/perl #Import the package use strict; use File::Slurp; use Win32::EventLog; use Net::SNMP qw(:ALL); use threads; #my @jobs=(); #Global Variable Section my $path = "C:\\log_parsing"; my $eventSource1 = "Application"; my $eventSource2 = "System"; my $config_file1 = $path."\\app_config.txt"; my $config_file2 = $path."\\sys_config.txt"; my $outputfile1 = $path."\\app_output.txt"; my $outputfile2 = $path."\\sys_output.txt"; my $outputfile3 = $path."\\event_output.txt"; my @app_common_regex=(); my @app_iis_regex=(); my @app_powerpath_regex=(); my @sys_common_regex=(); my @sys_iss_regex=(); my @sys_powerpath_regex=(); my ($sys_string_common,$sys_string_iis,$sys_string_powerpath); my ($sys_num_common,$sys_num_iis,$sys_num_powerpath); my ($app_string_common,$app_string_iis,$app_string_powerpath); my ($app_num_common,$app_num_iis,$app_num_powerpath); my @app_config = read_file($config_file1) or die "Config file for $con +fig_file1 is missing or could not read - $!"; my @sys_config = read_file($config_file2) or die "Config file for $con +fig_file2 is missing or could not read - $!"; #------Get App regex-------# foreach my $app_regex(@app_config) { my @fields = split "," , $app_regex; push(@app_common_regex,$fields[0]) if ($app_regex =~ m/Common/ +i); push(@app_iis_regex,$fields[0]) if ($app_regex =~ m/IIS/i); push(@app_powerpath_regex,$fields[0]) if ($app_regex =~ m/Powe +rpath/i); } #-----Split Array based on Application for Application event Log-----# my @app_string_common_regex = (); my @app_num_common_regex = (); my @app_string_iis_regex = (); my @app_num_iis_regex = (); my @app_string_powerpath_regex = (); my @app_num_powerpath_regex = (); foreach(@app_common_regex) { push(@app_string_common_regex,$_) if ($_ !~ /^\d+/); push(@app_num_common_regex,$_) if ($_ =~ /^\d+/); } foreach(@app_iis_regex) { push(@app_string_iis_regex,$_) if ($_ !~ /^\d+/); push(@app_num_iis_regex,$_) if ($_ =~ /^\d+/); } foreach(@app_powerpath_regex) { push(@app_string_powerpath_regex,$_) if ($_ !~ /^\d+/); push(@app_num_powerpath_regex,$_) if ($_ =~ /^\d+/); } my $app_string_common = join("|", @app_string_common_regex); my $app_num_common = join("|", @app_num_common_regex); my $app_string_iis = join("|", @app_string_iis_regex); my $app_num_iis = join("|", @app_num_iis_regex); my $app_string_powerpath = join("|", @app_string_powerpath_regex); my $app_num_powerpath = join("|", @app_num_powerpath_regex); #------Get Sys regex-------# foreach my $sys_regex(@sys_config) { my @fields = split "," , $sys_regex; push(@sys_common_regex,$fields[0]) if ($sys_regex =~ m/Common/ +i); push(@sys_iss_regex,$fields[0]) if ($sys_regex =~ m/IIS/i); push(@sys_powerpath_regex,$fields[0]) if ($sys_regex =~ m/Powe +rpath/i); } #-----Split Array based on Application for System event Log-----# my @sys_string_common_regex = (); my @sys_num_common_regex = (); my @sys_string_iis_regex = (); my @sys_num_iis_regex = (); my @sys_string_powerpath_regex = (); my @sys_num_powerpath_regex = (); foreach(@sys_common_regex) { push(@sys_string_common_regex,$_) if ($_ !~ /^\d+/); push(@sys_num_common_regex,$_) if ($_ =~ /^\d+/); } foreach(@sys_iss_regex) { push(@sys_string_iis_regex,$_) if ($_ !~ /^\d+/); push(@sys_num_iis_regex,$_) if ($_ =~ /^\d+/); } foreach(@sys_powerpath_regex) { push(@sys_string_powerpath_regex,$_) if ($_ !~ /^\d+/); push(@sys_num_powerpath_regex,$_) if ($_ =~ /^\d+/); } my $sys_string_common = join("|", @sys_string_common_regex); my $sys_num_common = join("|", @sys_num_common_regex); my $sys_string_iis = join("|", @sys_string_iis_regex); my $sys_num_iis = join("|", @sys_num_iis_regex); my $sys_string_powerpath = join("|", @sys_string_powerpath_regex); my $sys_num_powerpath = join("|", @sys_num_powerpath_regex); print "System_Common = $sys_string_common\n"; print "System Common num = $sys_num_common\n"; print "Application Common = $app_string_common\n"; print "Application Common = $app_num_common\n"; #Create threads to run the Subroutines parallely my $thr1 = threads->create(\&parseAppEventLog, "Application"); my $thr2 = threads->create(\&parseSysEventLog, "System"); #Join the threads to execute the Subroutines $thr1->join(); $thr2->join(); #<-------Funtion to getLineno AppEvent -----># sub getLinenoApp() { my $start_line_num; my $handle; my $end_line_num; my $recs; my $lines; my $filename = $path."\\app_pos.txt"; unless (-e $filename) #Check for the existence of last Line num fi +le { open (LAST_LINE_NUM,">$filename") or die ("Cannot create file"); $handle=Win32::EventLog->new($eventSource1, '') or die "Can't open Application EventLog on myhost\n"; $handle->GetNumber($recs); $lines += $recs; $handle->Close; print LAST_LINE_NUM $lines; return $lines; close (LAST_LINE_NUM); } else { open (LAST_LINE_NUM,"$filename") or die "Cannot open file"; foreach my $start_num (<LAST_LINE_NUM>) { $start_line_num=$start_num; } open (WRITE_END_LINE_NUM,">$filename") or die ("Cannot write end +line no"); $handle=Win32::EventLog->new($eventSource1, '') or die "Can't open Application EventLog on myhost\n"; $handle->GetNumber($recs); #$handle->GetOldest($base); $end_line_num += $recs; $handle->Close; print WRITE_END_LINE_NUM $end_line_num; if ($start_line_num == $end_line_num) { return 0; } elsif ($end_line_num < $start_line_num) { print "Windows Application Event Log - Event Log has been Tru +ncated or Cleared"; return $end_line_num; } else { my $lineno = $end_line_num - $start_line_num; return $lineno; } close (LAST_LINE_NUM); close (WRITE_END_LINE_NUM); } } #<-------Funtion to getLineno SysEvent -----># sub getLinenoSys() { my $start_line_num; my $handle; my $end_line_num; my $recs; my $lines; my $filename = $path."\\sys_pos.txt";; unless (-e $filename) #Check for the existence of last Line num fi +le { open (LAST_LINE_NUM,">$filename") or die ("Cannot create file"); $handle=Win32::EventLog->new($eventSource2, '') or die "Can't open System EventLog on myhost\n"; $handle->GetNumber($recs); $lines += $recs; $handle->Close; print LAST_LINE_NUM $lines; return $lines; close (LAST_LINE_NUM); } else { open (LAST_LINE_NUM,"$filename") or die "Cannot open file"; foreach my $start_num (<LAST_LINE_NUM>) { $start_line_num=$start_num; } open (WRITE_END_LINE_NUM,">$filename") or die ("Cannot write end +line no"); $handle=Win32::EventLog->new($eventSource2, '') or die "Can't open System EventLog on myhost\n"; $handle->GetNumber($recs); #$handle->GetOldest($base); $end_line_num += $recs; $handle->Close; print WRITE_END_LINE_NUM $end_line_num; if ($start_line_num == $end_line_num) { return 0; } elsif ($end_line_num < $start_line_num) { print "Windows System Event Log - Event Log has been Truncate +d or Cleared"; return $end_line_num; } else { my $lineno = $end_line_num - $start_line_num; return $lineno; } close (LAST_LINE_NUM); close (WRITE_END_LINE_NUM); } } #<-------Funtion to Parse AppEvent -----># sub parseAppEventLog { my ($first,$count); $first = $count = 0; my $msg; my $event; my $found = 0; my $EventLog; my $eventSource = $_[0]; my $limit = getLinenoApp(); #my @app_config_field2 = read_file($config_file1) or die "Config fi +le for $config_file1 is missing or could not read - $!"; $EventLog = new Win32::EventLog( $eventSource, '' ) || die $!; $EventLog->GetOldest($first) || die $!; $EventLog->GetNumber($count) || die $!; $Win32::EventLog::GetMessageText = 1; $EventLog->Read((EVENTLOG_SEEK_READ | EVENTLOG_BACKWARDS_READ),$firs +t+$count,$event); open (my $out_app,">$outputfile1") or die "cannot create App event o +utput file - $!"; if ($limit == 0) { $msg = "Common-App##Windows ".$eventSource." Event Log - Event log + has not increased in size since last run"; print $out_app $msg; #exit (0); } for my $i ($first+$count-$limit+1..$first+$count) { $EventLog->Read((EVENTLOG_SEQUENTIAL_READ|EVENTLOG_BACKWARDS_READ) +,0,$event); my $source = $event->{'Source'}; my $timewritten = $event->{'Timewritten'}; my $id = $event->{'EventID'} & 0xffff; #to get a readab +le EventId my $type = $event->{'EventType'}; my $category = $event->{'Category'}; my $strings = $event->{'Strings'}; my $computer = $event->{'Computer'}; my $eventmsg = $event->{'Message'}; #---Application Common ------# if (length($app_num_common) !=0 ) { my $fr_app_num_common = qr/$app_num_common/; if($id =~ /\b$fr_app_num_common\b/) { #$msg = "1.3.6.1.4.1.326##Windows ".$eventSource." Event Log erro +r-EventID:".$id."-".$eventmsg; $msg = "Common-App##Windows error-EventID:".$id."-".$eventmsg; print $out_app $msg ; } } if (length($app_string_common) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$app_string_common/i) { #$msg = "1.3.6.1.4.1.326##Windows ".$eventSource." Event Log e +rror-EventID:".$id."-".$eventmsg; $msg = "Common-App##Windows error-EventID:".$id."-".$eventmsg; print $out_app $msg; } } #----IIS System ----# if (length($app_num_iis) !=0 ) { my $fr_app_num_iis = qr/$sys_num_iis/; if($id =~ /\b$fr_app_num_iis\b/) { $msg = "IIS##Windows IIS Event Log error-EventID:".$id."-".$event +msg; print $out_app $msg ; } } if (length($app_string_iis) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$app_string_iis/i) { $msg = "IIS##Windows IIS Event Log error-EventID:".$id."-".$ev +entmsg; print $out_app $msg; } } #-------Powerpath System ------# if (length($app_num_powerpath) !=0 ) { my $fr_app_num_powerpath = qr/$sys_num_powerpath/; if($id =~ /\b$fr_app_num_powerpath\b/) { $msg = "Powerpath##Windows Powerpath Event Log error-EventID:".$i +d."-".$eventmsg; print $out_app $msg ; } } if (length($app_string_powerpath) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$app_string_powerpath/i) { $msg = "Powerpath##Windows Powerpath Event Log error-EventID:" +.$id."-".$eventmsg; print $out_app $msg; } } } close($out_app); } #<-------Funtion to Parse SysEvent -----># sub parseSysEventLog { my ($first,$count); $first = $count = 0; my $msg; my $event; my $found = 0; my $EventLog; my $limit_sys = getLinenoSys(); my $eventSource = $_[0]; # print $limit_sys; #my @app_config_field2 = read_file($config_file1) or die "Config fi +le for $config_file1 is missing or could not read - $!"; $EventLog = new Win32::EventLog( $eventSource, '' ) || die $!; $EventLog->GetOldest($first) || die $!; $EventLog->GetNumber($count) || die $!; $Win32::EventLog::GetMessageText = 1; $EventLog->Read((EVENTLOG_SEEK_READ | EVENTLOG_BACKWARDS_READ),$firs +t+$count,$event); open (my $out_sys,">$outputfile2") or die "cannot create sys event o +utput file - $!"; if ($limit_sys == 0) { $msg = "Common-Sys##Windows ".$eventSource." Event Log - Event log + has not increased in size since last run"; print $out_sys $msg; #exit (0); } for my $i ($first+$count-$limit_sys+1..$first+$count) { $EventLog->Read((EVENTLOG_SEQUENTIAL_READ|EVENTLOG_BACKWARDS_READ) +,0,$event); my $source = $event->{'Source'}; my $timewritten = $event->{'Timewritten'}; my $id = $event->{'EventID'} & 0xffff; #to get a readab +le EventId my $type = $event->{'EventType'}; my $category = $event->{'Category'}; my $strings = $event->{'Strings'}; my $computer = $event->{'Computer'}; my $eventmsg = $event->{'Message'}; #---System Common ------# if (length($sys_num_common) !=0 ) { my $fr_sys_num_common = qr/$sys_num_common/; if($id =~ /\b$fr_sys_num_common\b/) { $msg = "Common-Sys##Windows error-EventID:".$id."-".$eventmsg; print $out_sys $msg ; } } if (length($sys_string_common) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$sys_string_common/i) { $msg = "Common-Sys##Windows error-EventID:".$id."-".$eventmsg; print $out_sys $msg; } } #----IIS System ----# if (length($sys_num_iis) !=0 ) { my $fr_sys_num_iis = qr/$sys_num_iis/; if($id =~ /\b$fr_sys_num_iis\b/) { $msg = "IIS##Windows IIS Event Log error-EventID:".$id."-".$event +msg; print $out_sys $msg ; } } if (length($sys_string_iis) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$sys_string_iis/i) { $msg = "IIS##Windows IIS Event Log error-EventID:".$id."-".$ev +entmsg; print $out_sys $msg; } } #-------Powerpath System ------# if (length($sys_num_powerpath) !=0 ) { my $fr_sys_num_powerpath = qr/$sys_num_powerpath/; if($id =~ /\b$fr_sys_num_powerpath\b/) { $msg = "Powerpath##Windows Powerpath Event Log error-EventID:".$i +d."-".$eventmsg; print $out_sys $msg ; } } if (length($sys_string_powerpath) !=0 ) { #print $eventmsg; if($eventmsg =~ m/$sys_string_powerpath/i) { $msg = "Powerpath##Windows Powerpath Event Log error-EventID:" +.$id."-".$eventmsg; print $out_sys $msg; } } } close($out_sys); } #Combine the output of System and Application event logs open (my $combined_file,">$outputfile3") or die "cannot create combine +d output file3 - $!"; if (( -s $outputfile1 > 0) || (-s $outputfile2 > 0)) { my @a = read_file($outputfile1) or die "couldn't read $outputfile1 - $!"; my @b = read_file($outputfile2) or die "couldn't read $outputfile2 - $!"; my $combined = {}; # hashref my $i=0; foreach (@a) { chomp; $combined->{$i}{b} = '' unless defined $combined->{$i}{b}; $combined->{$i++}{a} = $_; } $i=0; foreach (@b) { chomp; $combined->{$i}{a} = '' unless defined $combined->{$i}{a}; $combined->{$i++}{b} = $_; } foreach my $i (sort {$a<=>$b} keys %$combined) { print $combined_file $combined->{$i}{a}, ("\n"),$combined->{$i}{b} +; } } close($combined_file); #Send Trap from Event_Output file my ($session, $error) = Net::SNMP->session( -hostname => '10.233.2.3 +5', -community => 'public', -version => 'snmpv2c', -port => 162 ); my @oids1 = (); my @oids2 = (); my @oids3 = (); my @oids4 = (); my @oids5 = (); my @oids6 = (); my @fields = (); my @message; my @oid_val; $session->max_msg_size(4500); print $session->debug(255); if (!defined($session)) { printf("ERROR: %s.\n", $error); exit 1; } my @Send_Trap = read_file($outputfile3) or die "couldn't read $outputf +ile1 - $!"; my $result; foreach my $evt_output(@Send_Trap) { #chomp $evt_output; @fields = split "##", $evt_output; if ($fields[0] =~ m/Common-App/i) { push @oids1, ('1.3.6.1.2.1.1.3.0', OCTET_STRING, $fields[1]); push @oids1, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '1.3. +6.1.4.1.326'); $result = $session->snmpv2_trap( -varbindlist => \@oids1 ); } elsif ($fields[0] =~ m/Common-Sys/i) { push @oids2, ('1.3.6.1.2.1.1.3.0', OCTET_STRING, $fields[1]); push @oids2, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '1.3. +6.1.4.1.333'); $result = $session->snmpv2_trap( -varbindlist => \@oids2 ); } elsif ($fields[0] =~ m/IIS/i) { push @oids3, ('1.3.6.1.2.1.1.3.0', OCTET_STRING, $fields[1]); push @oids3, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '1.3. +6.1.4.1.328'); $result = $session->snmpv2_trap( -varbindlist => \@oids3 ); } elsif ($fields[0] =~ m/Powerpath/i) { push @oids4, ('1.3.6.1.2.1.1.3.0', OCTET_STRING, $fields[1]); push @oids4, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '1.3. +6.1.4.1.327'); $result = $session->snmpv2_trap( -varbindlist => \@oids4 ); } #print $fields[1]; # push @oids, ('1.3.6.1.2.1.1.3.0', OCTET_STRING, $fields[1]); # push @oids, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '1.3.6. +1.4.1.326'); } # my $outputmsg = join("\n", @message); # print $outputmsg; #push @oids, ('1.3.6.1.6.3.1.1.4.1.0', OBJECT_IDENTIFIER, '$fields'); #print @oids1; #print $result; if (!defined($result)) { printf("ERROR: %s.\n", $session->error); $session->close; exit 1; } $session->close;


Comment on Re^2: Creating thread to call Subroutines parallely
Download Code
Re^3: Creating thread to call Subroutines parallely
by BrowserUk (Pope) on Feb 21, 2013 at 15:32 UTC

    I cannot see anything obvious that might cause your output file to be empty -- other than perhaps there were no events that matched your criteria. Is that possible?

    This is the error i am getting since the file app_output.txt is empty couldn't read C:\log_parsing\app_output.txt - at log_windows_new.pl line 444.

    That still doesn't isn't the full error message.

    Line 444 is:

    my @a = read_file($outputfile1) or die "couldn't read $outputfile1 - $!"; # ........................................^^

    The $! should mean that the full error message would be something like:

    couldn't read C:\log_parsing\app_output.txt - No such file or director +y at log_windows_new.pl line 444. #.............................................^^^^^^^^^^^^^^^^^^^^^^^^ +^

    Or perhaps:code> couldn't read C:\log_parsing\app_output.txt - Permission denied at log_windows_new.pl line 444. #.............................................^^^^^^^^^^^^^^^^^ </code>

    That piece of information would likely point to the source of the problem; but you haven't given us it?

    I'd also suggest that you add $^E to line 44 (and other error messages). Ie:

    my @a = read_file($outputfile1) or die "couldn't read $outputfile1 - $! [$^E]";

    That may provide further clarification of the problem.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Event though if the criteria is not matched i am sending a dummy text to the file" as below atleast this message should write into that file. At the time of error when i open the app_output file it remains blank.
      if ($limit == 0) { $msg = "Common-App##Windows ".$eventSource." Event Log - Event log + has not increased in size since last run"; print $out_app $msg; }
        Event though if the criteria is not matched i am sending a dummy text to the file" as below atleast this message should write into that file.

        Hm. That covers teh situation where there are no new messages. What about where there are new messages, but none of them match your regexes?

        At the time of error when i open the app_output file it remains blank.

        An existing but empty file would not "fail to open" at line 444.

        But then your error message does not reflect the situation you are actually testing.

        If the file is empty, the open (inside File::Slurp::read_file() would succeed, but then the function would return no lines. Your code would detect the absence of any input and report it as an open failure.

        But you are still not supplying the full error message ($!)

        You better next time else I'm giving up on you.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1019953]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (10)
As of 2014-12-21 05:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (103 votes), past polls