Perl script end up on saying "Out of Memory !"

[syedumairali]

Hi Monks,

I am creating a perl script which searches for a specific text in the CSV text file (100000 rows & 30 KB) and there are huge huge numbers of files. I am usign hashkeys to first put one file into the hash. And then search the specific text. After It finishes searches. I uses the same hash function to copy the second csv file of same size and search for a specific function.

The script ran perfectly for the 60 odds files but after that it crashes with "Out of Memory !".

While running script I am also observing from task manager the size of available memory continuously decreasing (2GB ram).

I think I am missing clearing the hash variable (@data1) and the the error message comes when my hach fully utilizes the full memory.

Question : How can I erase or clear the hash before my perl script takes the second file ? here is the sample code (shown only relevant code)

# @ lines contain csv files

my %data1;
shift(@lines1); # remove column headings from file
shift(@lines1); # remove column headings from file


    foreach my $line (@lines1) {
        @words = split (/\,/, $line);            
        if ($words[6] > 90) {
        my $abstime = $words[1];
        my $payload = $words[5];
        $srcIPhex = substr $payload, 24, 8;
        my $dstIPhex = substr $payload, 32, 8;
        my $timestamp = substr $payload, 152, 12;            
        
        my $HashKey; # to get total number
        
        $HashKey = $srcIPhex.$abstime;        
        $data1{$HashKey}{ID} = $words[0];        
        $data1{$HashKey}{SRC_IP} = $srcIPhex;
        $data1{$HashKey}{DST_IP} = $dstIPhex;
        
        MeasureFiles(\%data1);
        
        }
                        
sub MeasureFiles
{
    my ($list_a_ref) = @_;        
  my %data1 = %$list_a_ref; # Dereference lists
  ....
  ....
  foreach (keys %data1) {                    
    $SrcIP_captured = inet_ntoa( pack( "N", hex( $data1{$_}{SRC_IP} ) 
+) );
    $DstIP_captured = inet_ntoa( pack( "N", hex( $data1{$_}{DST_IP} ) 
+) );
    foreach(my $i=0;$i<$ind;$i++){
       if ( $SrcIP_captured eq $SrcIP_ref[$i] &&  $DstIP_captured eq $
+DstIP_ref[$i]) {             
               $pkt_received++;                                       
+       
                   }
              }
          } 
         ....
         ....
    open(R1,">> $mainDirectory\\Results\\$file_result") || die("Cannot
+ Open File $file_result");
    my $results = "$SrcIP_ref[$i],$DstIP_ref[$i],$pkt_received";
    print R1 "$results\n";
    close(R1);
}