Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

I was asked to build this to help make stripping table formatted data out of text files easier for our engineers. I'm really new to Perl and I'd love to hear any suggestions on simplifying things, making it more readable, reliable, ect...

Update- > Thanks for the comments! I fixed some bugs and logic errors I had. Also, I just tested this script on several hundred report files successfully. Still... I know I've got a lot of work to do. I feel like I did everything the hard way .

Update (05/07/2012)-> Again, thanks for the suggestions. So far I implemented strict and warnings, then corrected a handful of logical errors I found. Sounds like threading, large file handling, and a lot more simplifying would be useful so I'll work on that next. Just a side note, I'm doing all this on my work computer, without admin rights so installing new modules is a pain in the ass.

Update (05/08/2012) -> Added some more suggestions from the group, and I switched to the TK debugger so I removed all those prints

#!C:\Program Files\Perl\bin\perl.exe #Created By Bert.Mcguirk@gmail.com die "This script is intended to run on a windows machine" unless ( $^O + eq "MSWin32" ); use File::Path; use File::Find; use Tk; use Tk::LabFrame; use POSIX qw/strftime/; use strict; use warnings; no warnings 'uninitialized'; use Cwd qw(abs_path); require Tk::ROText; our @output; our @errors; #directory the script was launched from our $cwd = abs_path(); $cwd =~ tr/\//\\/; $cwd = "$cwd\\"; our( $directory, $search_string, $lines_above, $scan_up_string, $lines_below, $scan_down_string, $chk_button_recursive, $chk_button_show_file, $chk_button_show_line_number, $chk_button_save_search, $lines_below_to_skip, $skip_until_string, $lines_below_to_end_text_block, $scan_down_string_to_end_text_block); #variables tied to user inp +ut our $eventpad; #message board our $version = "1.6"; # file name, max size in bytes &large_file_cleanup($cwd."search parameters.txt", 1048576); # 1mb &large_file_cleanup($cwd."error log.txt",1048576);# 1 mb &make_gui; MainLoop; ##########################################START GUI FUNCTION########## +################################# sub make_gui{ my $mw = MainWindow->new; # Mainwindow: size x/y, position x/y $mw->geometry("620x575+100+120"); $mw ->title("svgrep $version"); # Logging window $eventpad = $mw->Scrolled( 'ROText', -scrollbars => 'e', # east -background => 'white', -width => 83, # character count -height => 10, )->place( -x => 8, -y => 415); report("Event Logging will be shown here."); my $label1 = $mw->Label( -text => "Simple Visual grep", -font => "Helvetica -20 ", )->place( -x => 210, -y => 05); my $label2 = $mw->Label( -text => "OR", -font => "Helvetica -20 ", )->place( -x => 500, -y => 300); &required_parameter_frame($mw); &add_lines_frame($mw); &advanced_options_frame($mw); &text_block_frame($mw); ################################### load saved parameters by defau +lt ############################################## if (-e "$cwd"."search parameters.txt") { my $last_line; open (SAVED_SEARCH, "<"."search parameters.txt"); my @data = <SAVED_SEARCH>; if ($data[$#data] =~ /,/){ $last_line = $data[$#data]; }else{ for my $line (reverse @data){ $last_line = $line; last if $line =~ /,/; } } close SAVED_SEARCH; ($directory, $search_string, $lines_above, $scan_up_string, $lines_below, $scan_down_string, $chk_button_recursive, $chk_button_show_file, $chk_button_show_line_number, $chk_button_save_search, $lines_below_to_skip, $skip_until_string, $lines_below_to_end_text_block, $scan_down_string_to_end_text_block) = split(/,/,$last_line); report("Saved inputs have been loaded" ); } ########################## end handling saved search parameters ## +########################## $mw->Button( -text => "run grep", #padx/y is to create space IN the button around the te +xt -padx => 5, -pady => 5, -font => "Helvetica -18 ", -command =>\&execute_button )->place( -x => 500, -y => 50); } sub required_parameter_frame{ my $mother = shift; ##################### # Primary frame # ##################### my $frame = $mother->LabFrame( -label=>"Required", -width => 200, -height => 105, # Pixel -font => "Helvetica -12 ", )->place(-x=>7,-y=>35); $frame->Label( -text => 'Enter a Target Directory', -font => "Helvetica -12 ", )->place( -x => 7, -y => 10); $frame ->Entry( -width =>30, # width is in characters, not pixel -textvariable => \$directory )->place( -x => 7, -y => 30); $frame->Label( -text => 'Search String', -font => "Helvetica -12 ", )->place( -x => 7, -y => 50); $frame ->Entry( -width =>30, # width is in characters, not pixel -textvariable => \$search_string )->place( -x => 7, -y => 70); } sub add_lines_frame{ my $mother = shift; my $frame = $mother->LabFrame( -label=>"Add lines to be returned", -width => 200, -height => 210, -font => "Helvetica -12 ", )->place(-x=>7,-y=>165); $frame->Label( -text => '# of lines above search string', -font => "Helvetica -12 ", )->place( -x => 7, -y => 10); $frame ->Entry( -width =>6, # width is in characters, not pixel -textvariable => \$lines_above )->place( -x => 7, -y => 30); $frame->Label( -text => 'OR Scan up to this string', -font => "Helvetica -12 ", )->place( -x => 7, -y => 50); $frame ->Entry( -width =>30, # width is in characters, not pixel -textvariable => \$scan_up_string )->place( -x => 7, -y => 70); $frame->Label( -text => '# of lines below search string', -font => "Helvetica -12 ", )->place( -x => 7, -y => 110); $frame ->Entry( -width =>6, # width is in characters, not pixel -textvariable => \$lines_below )->place( -x => 7, -y => 130); $frame->Label( -text => 'OR scan down to this string', -font => "Helvetica -12 ", )->place( -x => 7, -y => 150); $frame ->Entry( -width =>30, # width is in characters, not pixel -textvariable => \$scan_down_string )->place( -x => 7, -y => 170); } sub advanced_options_frame{ my $mother = shift; my $frame = $mother->LabFrame( -label=>"Advanced Options", -width => 200, -height => 105, # Pixel -font => "Helvetica -12 ", )->place(-x=>250,-y=>35); # check buttons are set to 0 for deselect and 1 for select my $chk1 = $frame-> Checkbutton(-text=>"Recursive Directory Search +", -variable=>\$chk_button_recursive)->place( -x => 7, -y => 7); $chk1 -> deselect(); my $chk2 = $frame -> Checkbutton(-text=>"Show File Name in Output" +, -variable=>\$chk_button_show_file)->place( -x => 7, -y => 28); $chk2 -> deselect(); my $chk3 = $frame -> Checkbutton(-text=>"Show Line Number in Outpu +t", -variable=>\$chk_button_show_line_number)->place( -x => 7, -y => 4 +9); $chk3 -> deselect(); my $chk4 = $frame -> Checkbutton(-text=>"Save search parameters", -variable=>\$chk_button_save_search)->place( -x => 7, -y => 69); $chk4 -> select(); } sub text_block_frame { my $mother = shift; my $frame = $mother->LabFrame( -label=>"Define a text block to be returned (simulate AWK)", -width => 350, -height => 180, -font => "Helvetica -12 ", )->place(-x=>250,-y=>175); $frame->Label( -text => '# of lines to skip below search string to start outp +ut', -font => "Helvetica -12 ", )->place( -x => 60, -y => 30); $frame ->Entry( -width =>6, # width is in characters, not pixel -textvariable=> \$lines_below_to_skip )->place( -x => 7, -y => 30); $frame->Label( -text => 'OR scan down to this string', -font => "Helvetica -12 ", )->place( -x => 150, -y => 60); $frame ->Entry( -width =>20, # width is in characters, not pixel -textvariable=> \$skip_until_string )->place( -x => 7, -y => 60); $frame->Label( -text => 'total # of lines to output', -font => "Helvetica -12 ", )->place( -x => 60, -y => 100); $frame ->Entry( -width =>6, # width is in characters, not pixel -textvariable => \$lines_below_to_end_text_block )->place( -x => 7, -y => 100); $frame->Label( -text => 'OR end text block at this string', -font => "Helvetica -12 ", )->place( -x => 150, -y => 130); $frame ->Entry( -width =>20, # width is in characters, not pixel -textvariable => \$scan_down_string_to_end_text_block )->place( -x => 7, -y => 130); } ##########################################END GUI FUNCTIONs########### +################################ sub execute_button{ @errors=(); @output=(); report("grep started for search string:[".&trim($search_string)."] +\n". "within directory:[".&trim($directory)."]\n"); $lines_above = &validate_inputs($lines_above,"numeric")if $lines_a +bove; $lines_below = &validate_inputs($lines_below,"numeric")if $lines_b +elow; $lines_below_to_skip = &validate_inputs($lines_below_to_skip,"nume +ric")if $lines_below_to_skip; $lines_below_to_end_text_block = &validate_inputs($lines_below_to_ +end_text_block,"numeric")if $lines_below_to_end_text_block; $directory = &validate_inputs($directory,"directory")if $directory +; # Detect variables that have been set to invalid by validate_input +s then exit # the execute_button function. if ($lines_above eq "invalid"|| $lines_below eq "invalid"|| $lines_below_to_skip eq "invalid"|| $lines_below_to_end_text_block eq "invalid"|| $directory eq "invalid"){ # exit button function without doing the search report("grep action aborted due to invalid input parameters." +); push @errors,current_time()."grep action aborted, bad inputs." +; &make_error_log; return; } # catch bad input combinations. if ($lines_above && $scan_up_string ){ report("\nYou must select either a line amount or scan to stri +ng, not both." ); return; } if($lines_below && $scan_down_string ){ report("\nYou must select either a line amount or scan to stri +ng, not both."); return; } if(($lines_above || $lines_below || $scan_up_string || $scan_down_ +string) && ($lines_below_to_skip || $lines_below_to_end_text_block || $scan_down_string_to_end_text_block || $skip_until_string)){ report("\nYou must select either add lines option or text bloc +k option, not both." ); return; } if($lines_below_to_skip && $skip_until_string){ report("\nYou must select either lines below to skip or scan d +own string". ", not both" ); return; } ####################### END INPUT VALIDATION ##################### +######################### if ($chk_button_save_search == 1) { open (SAVED_SEARCH, '>>'."$cwd".'search parameters.txt'); print SAVED_SEARCH (join ',', &trim($directory), &trim($search_string), &trim($lines_above), &trim($scan_up_string), &trim($lines_below), &trim($scan_down_string), &trim($chk_button_recursive), &trim($chk_button_show_file), &trim($chk_button_show_line_number), &trim($chk_button_save_search), &trim($lines_below_to_skip), &trim($skip_until_string), &trim($lines_below_to_end_text_block), &trim($scan_down_string_to_end_text_block),"\n"); close SAVED_SEARCH; } if ($directory && $search_string){ my $retval = &search_files; if ($retval){ report("Results have been found, please check output file. +"); &make_output_file; }else{ report("No results found :("); push @errors,current_time()."No results found in directory +: [$directory]"; &make_error_log; } }else { report("Please enter a diretory and search string."); } } sub search_files { our $positive_match; find({wanted=> sub{ my ($match,$max_depth, $start_line, $stop_line,$i,$break); &trim($search_string); &trim($lines_above); &trim($scan_up_string); &trim($lines_below); &trim($scan_down_string); &trim($chk_button_recursive); &trim($chk_button_show_file); &trim($chk_button_show_line_number); &trim($chk_button_save_search); &trim($lines_below_to_skip); &trim($skip_until_string); &trim($lines_below_to_end_text_block); &trim($scan_down_string_to_end_text_block); #Directory Depth control, count slashes if ($chk_button_recursive){ $max_depth = '99'; }else{ $max_depth = '0'; } my $depth = $File::Find::dir =~ tr[/|\\][]; return if $depth > $max_depth; if (-f $_){ report("Searching file:[$_]"); unless (open FILE, $_) { push @errors, current_time()."Can't open $File::Find:: +name : $!"; report("Can't open $File::Find::name :[$!]"); return; } my @data = <FILE>; #load whole file into an array for (my $line_count = 0 ; $line_count < $#data; $line_coun +t++) { if ( index($data[$line_count],$search_string) >= 0 ) { + $positive_match = 1; unless( $lines_above || $lines_below || $lines +_below_to_skip || $lines_below_to_end_text_block || $scan_down +_string_to_end_text_block || $scan_up_string || $scan_down_string || $skip_ +until_string){ &send_to_output_array($chk_button_show_line_nu +mber,$chk_button_show_file, $File::Find::name,$line_count,$data[$line_ +count]); } #start add lines option -> if ($lines_above || $lines_below || $scan_up_strin +g || $scan_down_string){ $start_line = $line_count - $lines_above; $stop_line = $lines_below + $line_count; if ($scan_up_string){ $match=''; for ( $i = $line_count; $i >= 0; $i-- ){ if (index($data[$i],$scan_up_string) > += 0 ) { $match=1; $start_line=$i; last; } } unless($match){ report("Could not find scan up string: +". "[$scan_up_string] in file:". "[$File::Find::name]"); push @errors,(current_time(). "Could not find scan up string: [$ +scan_up_string]". "in file: $File::Find::name"); $positive_match=''; return; } } if ($scan_down_string){ $match=''; for ($i=$line_count; $i<$#data ; $i++){ if (index($data[$i],$scan_down_string) +>=0) { $stop_line=$i; $match=1; last; } } unless($match){ report("Could not find scan down strin +g:". "[$scan_down_string] in file:". "[$File::Find::name]"); push @errors,(current_time(). "Could not find scan up string: [$ +scan_down_string]". "in file: $File::Find::name"); $positive_match=''; return; } } #send data from start line to matched line if ($lines_above || $scan_up_string){ for ($i=$start_line; $i<$line_count; $i++) + { &send_to_output_array($chk_button_show +_line_number, $chk_button_show_file,$File::Find: +:name,$i,$data[$i]); } } #send matched line out &send_to_output_array($chk_button_show_line_nu +mber,$chk_button_show_file, $File::Find::name, $line_count, $data[$lin +e_count]); #send lines after matched line down to the new + stopping point. if ($lines_below || $scan_down_string){ #start right after matched line for ($i=($line_count+1); $i<=$stop_line;$i +++) { &send_to_output_array($chk_button_ +show_line_number, $chk_button_show_file, $File::Find::name, $i, $data[$ +i]); } } } #end add lines option <- #start textblock options-> if ($lines_below_to_skip || $lines_below_to_end_te +xt_block || $scan_down_string_to_end_text_block || $skip_u +ntil_string ){ #start at matched line if lines below to skip +wasn't if($skip_until_string){ $match=''; for ( $i = $line_count; $i < $#data ;$i++) +{ if (index("$data[$i]",$skip_until_stri +ng) >= 0) { $start_line = ($i+1); #start outpu +t after string $match = 1; last; } } unless($match){ report( "Could not find scan down string r +equested in text block options:". "[$skip_until_string] in file:". "[$File::Find::name]"); push @errors, ( current_time() . "Could not find scan down string int t +ext block options: [$skip_until_string]". "in file: $File::Find::name"); $positive_match=''; return; } }else{ $start_line = ($line_count+$lines_below_to +_skip); } if ($scan_down_string_to_end_text_block){ $match=''; for ( $i = $start_line; $i < $#data ;$i++) +{ if (index("$data[$i]",$scan_down_strin +g_to_end_text_block) >= 0) { $stop_line = ($i-1); #don't grab t +erminator string. $match = 1; last; } } unless($match){ report( "]Could not find string to end tex +t block:". "[$scan_down_string_to_end_text_bl +ock] in file:". "[$File::Find::name]"); push @errors, current_time(). "Could not find scan down string: [$scan_down_string_to_end_text_blo +ck]". "in file: $File::Find::name"; $positive_match=''; return; } } else { $stop_line = ($lines_below_to_end_text_blo +ck + $start_line); } for ( my $i = $start_line; $i <= $stop_line; $ +i++ ) { &send_to_output_array($chk_button_show_lin +e_number,$chk_button_show_file, $File::Find::name, $i, $data[$i]); } } # end text block options <- close FILE; return; } } }else{ report("In valid file found:[$_]\n"); } close FILE; } },$directory); # end of find function return 1 if $positive_match; } sub make_error_log{ if($#errors > 0){ open(ERROR_LOG,'>>'."$cwd".'error log.txt'); foreach (@errors){print ERROR_LOG ($_."\n");} close(ERROR_LOG); report("$#errors erorr(s) have been sent to the error log file +."); } } sub send_to_output_array{ my ($show_line,$show_file_name, $file_name,$line_count, $data) = @ +_; &trim($data); my $line; $line = "$file_name," if $show_file_name; $line .= "$line_count," if $show_line; $line .= $data; push @output,$line; } sub validate_inputs{ my ($user_response, $type) = @_; #return a positive value if a number is found if ($user_response){ if ($type eq "numeric" && $user_response =~ /^\d+$/ || $type eq "directory" && $user_response =~ /^\w:[\\\/] +/ && -d $user_response ){ return $user_response; }else{ #return wasn't executed, no match, do these: report("Invalid entry:[$user_response] expected type:[$typ +e]"); push @errors, current_time()."Invalid entry:[$user_respons +e] for type:[$type]"; return "invalid"; } } } sub large_file_cleanup{ my ($file_name,$file_size_limit)=@_; if ((-s $file_name) > $file_size_limit){ report("large file:[$file_name] has been reduced in size to sa +ve space"); open (FH, "<".$file_name); my @lines = <FH>; close FH; foreach(@lines){chomp($_)}; open (FH, ">".$file_name); for (my $i = ($#lines - 100); $i <= $#lines; $i++){ print FH ($lines[$i]. "\nif $lines[$i]"); } close FH; } } sub trim{ my $string = shift; $string =~ s/^\s+//; #match starting white lines $string =~ s/\s+$//; #match trailing white space chomp($string); return $string; } sub make_output_file{ my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime +(time); my $output_data_file_name = ($cwd."output data "."$hour$min$sec"." +.txt"); open(OUT, ">$output_data_file_name"); print OUT @output; close(OUT); report("[$#output] line(s) of data have been sent to:[$output_data +_file_name]"); } sub Tk::Error{ my ($widget,$error,@locations) = @_; chomp($error) if $error; print ("\nTK ERROR !!\nwidget: [$widget]\nerror:[$error]\nlocation +s:->\n"); foreach (@locations){ chomp($_); print "$_ \n"; } print "<-\n"; } sub current_time{ return "[".(strftime "%m/%d/%y %H:%M:%S", localtime)."]"; } sub report { $eventpad->insert( 'end', current_time() . shift() . "\n" ); $eventpad->update; $eventpad->see('end'); }

In reply to Noob could use advice on simplification & optimization by bgreg

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2024-03-19 02:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found