in reply to
Re: Multithreaded Script CPU Usage
in thread Multithreaded Script CPU Usage
Illuminatus, yes that is in fact what I am mostly worried about. In fact I think the Mem Usage here is due to one of the threads doing a regex replace, adding backslashes to prepare a file to be imported into mysql.
Edit: Just to clarify, I know that the CPU use should also be high when doing a regex on a large file with many matches, however, the CPU usage of the perl.exe process is always high, even when no regex operations are being performed.
My do_handler function just routes the work to a function in the separate Worker module (I couldn't figure out the syntax to call it directly)
Here's the relevant code from the Worker module:
package File::FileListerWorker;
sub main_run
{ # arguments: 1) job id from Pooler 2) path to scan 3) Path to FileLi
+st.exe 4)table name
# returns: 1) return 1;
#### - START: MAIN EXECUTABLE CODEBLOCK - ####
my $jobid = shift; # read scanpaths from the ARGs
my $scanpaths_line = shift; # get path to scan
my $fl_path = shift; # define path to FileList.exe
my $arg_tbl_name = shift; # table to load data into
my $dbh = &mysql_conn(); # connect to mysql
my $mysql_tablename = &mysql_create_table( $arg_tbl_name, $dbh );
+# create table (IF NOT EXISTS)
my @ta = localtime(time); # get time
my $time_string =
$ta[4] . $ta[3] . ( $ta[5] + 1900 ) . "_" . $ta[2] . $ta[1] . $t
+a[0]; # time string
my $temp_filename = # make unique filename
"\\\\directoryToPlaceTempFileIn\\"
. $jobid . "_"
. $time_string . ".csv";
&do_scan( $fl_path, $scanpaths_line, $temp_filename ); # call to s
+ub that does the scan
my $mysql_temp_filename = &fix_file_4mysql($temp_filename); # add
+backslashes
&mysql_load_data( $mysql_temp_filename, $mysql_tablename, $dbh );
+# mysql import file
&cleanup($temp_filename); #delete temp file to preserve space
&mysql_disc($dbh); # disconnect mysql
#### - END: MAIN EXECUTABLE CODEBLOCK - ####
return 1;
}
sub do_scan
{ #arguments: 1)location of FileList.exe 2)path to scan 3)output fi
+le path
#returns: none
my $fl_path = shift;
my $pth = shift;
chomp($pth);
my $temp_filename = shift;
system(qq{$fl_path $pth >$temp_filename});
}
