Need help with searching flatfile and updating it.

[Anonymous Monk]

I'm trying to match a word in the database and record it's occurence, I'm just using a form to post to the script. My form:

<FORM ACTION="cgi-bin/basic2.pl" METHOD="POST">
<SELECT NAME="browser">
    <OPTION VALUE="microsoft">microsoft</OPTION>
    <OPTION VALUE="netscape">netscape</OPTION>
    <OPTION VALUE="aol">aol</OPTION>
    <OPTION VALUE="mozilla">mozilla</OPTION>
    <OPTION VALUE="opera">opera</OPTION>
    </SELECT>
<INPUT TYPE="SUBMIT" VALUE="Submit">
</FORM>

My code:

#!/perl/bin/perl

print "Content-type: text/html\n\n";

$logpath = "data/browser.dat";  

&parse;  

sub parse{
    if ($ENV{'REQUEST_METHOD'} eq 'GET') {
    @pairs = split(/&/, $ENV{'QUERY_STRING'});
    }
    elsif ($ENV{'REQUEST_METHOD'} eq 'POST') {
    read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});
    @pairs = split(/&/, $buffer);
    }
    else {exit;}
    foreach $pair (@pairs) {
        ($name, $value) = split(/=/, $pair);
        $value =~ tr/+/ /;
        $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;
        $value =~ s/<!--(.|\n)*-->//g;
        $form{$name} = $value;
    }
}

$browser = $form{'browser'};

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

$not_found = 1;

open (LOG, ">$logpath");

foreach $line(@data) {
    chomp($line);
    ($fcount,$fbrowser)= split(/\|/,$line);
    if ($fbrowser eq $browser) {
    $not_found = 0;
    $fcount = $fcount+1;
    print LOG "$fcount|$fbrowser\n";
    }
    else {
    print "$line";
    }    
}
close(LOG);

if($not_found) {
open (LOG,">>$logpath");
print "1|$browser";
close(LOG);
}

# just prints out log to browser

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

foreach $line(@data) {
    chomp($line);
print "$line<br>";
}
close(LOG);

print "<p>$not_found";

It's is always not_found

Janitored by Arunbear - added readmore tags, as per Monastery guidelines

[Fletch]

• use CGI, don't reinvent the wheel.
• ALWAYS CHECK THE RETURN ON SYSTEM CALLS SUCH AS open()

[chas]

It isn't at all clear to me that your splitting on '&' of $buffer or $ENV{'QUERY_STRING'} (and your further decoding) is going to get you the exact data you need. It may...I'm not claiming it definitely won't, but life would be much easier if you use CGI.pm and the "param" method to get the data you need. You won't have to worry about GET or POST methods and the decoding of the encoded data will be done automatically for you. If you are going to use Perl, make use of one of the most useful modules available; you'll be glad you did!
chas

[Anonymous Monk]

Thanks for the critisism about cgi.pm, but I really Need help with searching flatfile and updating it, not cgi.pm. Now I have some new code here that maybe you could help me figure out why the script always clobbers and not ammends the file if the $browser is $not_found.

My form:

<FORM ACTION="cgi-bin/basic3.pl" METHOD="POST">
<SELECT NAME="browser">
    <OPTION VALUE="microsoft">microsoft</OPTION>
    <OPTION VALUE="netscape">netscape</OPTION>
    <OPTION VALUE="aol">aol</OPTION>
    <OPTION VALUE="mozilla">mozilla</OPTION>
    <OPTION VALUE="opera">opera</OPTION>
    </SELECT>
<INPUT TYPE="SUBMIT" VALUE="Submit">
</FORM>

My script:

#!/perl/bin/perl

use CGI;

$cgi = new CGI;

for $key ( $cgi->param() ) {
    $form{$key} = $cgi->param($key);
}

$logpath = "data/browser.dat";  

$browser = $form{'browser'};

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

$not_found = 1;

open (LOG, ">$logpath");

foreach $line(@data) {
    chomp($line);
    ($fcount,$fbrowser)= split(/\|/,$line);
    if ($fbrowser eq $browser) {
    $not_found = 0;
    $fcount = $fcount+1;
    print LOG "$fcount|$fbrowser\n";
    }
    else {
    print "$line";
    }    
}
close(LOG);

if($not_found) {
open (LOG,">>$logpath");
print LOG "1|$browser";
close(LOG);
}

# prints out log to browser

print "Content-type: text/html\n\n";

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

foreach $line(@data) {
    chomp($line);
print "$line\n";
}
close(LOG);

print "\n\n$not_found";

[Cody Pendant]

Can you explain what you're trying to do? It's a bit baffling. Can you also show us an example of your log file?

[chas]

Fletch advised (wisely!) that you always check that the calls to open succeeded. Is it possible that the file "data/browser.dat" is never actually opened and that's why you never see it amended? In fact, usually the path has to be given in some suitable way relative to the top directory that the web server recognizes, and you may not have that right. (It's happened to me!)
chas
(Update: Checking that open succeeded might be done by writing an error msg to the browser if the open fails.)

[Cody Pendant]

Just playing with a minimal case, I think I've grasped what you're trying to do -- are you trying to put the lines back into the LOG file which don't match? Put them back unchanged?

In that case, you probably want to change

 else {
    print "$line";
    }

to

 else {
    print LOG "$line\n";
    }

[jasonk]

The bigger problems you have with re-inventing wheels that should not be reinvented notwithstanding, your main problem is the fundamentally broken loop that is looking at the data that came from the file. Without resorting to rewriting code for you, in english your program does this:

• Open $logpath and read in the contents
• Iterate through the lines found there, and
  • If the browser in this line is the one from the form, increment it's counter and print it to the log
  • If the browser in this line is not the one from the form, print it to STDOUT

In a nutshell, this means that the browser that matches the form entry gets printed to the log, while the others get printed to the browser. I bet if you looked at the logfile, you would find it always contains at most 1 line.

There are however, several other problems with this code:

• Always, Always, Always use '-w' and 'use strict'.
• Doing your own parameter parsing instead of using CGI is just asking for trouble.
• Not checking the return code from your open calls means you may be truncating your logfile to rewrite it, even though the script didn't read any data from it in the first place.
• Not handling locking means that if two people submit the form at the same time, your logfile could very likely be nuked by the second one.

[chas]

"In a nutshell, this means that the browser that matches the form entry gets printed to the log, while the others get printed to the browser."
I forgot to remark on that in my reply, but I didn't undertand the point of that print either. In fact, it isn't even being printed to the browser I think, because the print "Content-type: text/html\n\n"; doesn't occur till later.
chas

[pearlie]

Hello,
You can use Tie::file module to read and update the file.

[Anonymous Monk]

#!/perl/bin/perl -w

use CGI;

$q = new CGI;

for $key ( $q->param() ) { $form{$key} = $q->param($key); }

$logpath = "data/browser.dat";  

$temppath = "C:/Documents and Settings/Owner/My Documents/My Website/c
+gi-bin/data/temp.dat";

$browser = $form{'browser'};

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

$not_found = 1;

if($browser) { 
    open (LOG, ">$logpath");
    foreach $line(@data) {
        chomp($line);
        ($fcount,$fbrowser)= split(/\|/,$line);
        if ($fbrowser eq $browser) {
        $not_found = 0;
        $fcount = $fcount+1;
        print LOG "$fcount|$fbrowser\n";
        }
        else {
        print LOG "$line\n";
        }    
    }
    close(LOG);

    if($not_found) {
    open (LOG,">>$logpath");
    print LOG "1|$browser";
    close(LOG);
    }
}

# prints out log and form to browser

print "Content-type: text/html\n\n";

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

foreach $line(@data) {
chomp($line);
print "$line<br>";
}
close(LOG);

print <<EOT;
<FORM ACTION="basic.pl" METHOD="POST">
<SELECT NAME="browser">
    <OPTION VALUE="microsoft">microsoft</OPTION>
    <OPTION VALUE="netscape">netscape</OPTION>
    <OPTION VALUE="aol">aol</OPTION>
    <OPTION VALUE="mozilla">mozilla</OPTION>
    <OPTION VALUE="opera">opera</OPTION>
    </SELECT>
<INPUT TYPE="SUBMIT" VALUE="Submit">
</FORM>

EOT




# If you would like to suggest a better way to sort here is the rest o
+f the code.

sub getBrowsers { 
%browse=();
open(INF,"$logpath") || print "Cannot open logs.dat file\n";
@hits= <INF>;
close(INF);

open(TEMP,">>$temppath") || print "Cannot open temp.dat file\n";
foreach $i (@hits) {
    chomp($i);
    ($hits,$browser)= split(/\|/,$i);
    for ($t=1;$t<$hits;$t++) {
    print TEMP "$browser\n";
    }
}
close(TEMP);

open(TEMP,"$temppath") || print "Cannot open temp.dat file\n";
@temp= <TEMP>;
close(TEMP);


foreach $i (@temp) {
    $browse{$i}++;
}
print <<EOT;
    <TABLE width="99%" cellpadding="3" cellspacing="1" border="0" bgCo
+lor="#0080ff">
    <TR><TD bgColor="#ccccff" height="25" colspan="2"><SPAN class="tit
+le">Top Browsers</SPAN></TD></TR>
EOT
$max = 1;
foreach $key (sort BrowserSort (keys(%browse))) { 
    ($max > $browse{$key}) || ($max = $browse{$key});
    ($max >= 401) && ($divisor = $max/401);
    ($max < 401) && ($multiplier = 401/$max);
    ($divisor) && ($width = sprintf("%0d",$browse{$key}/$divisor));
    ($multiplier) && ($width = sprintf("%0d",$browse{$key} * $multipli
+er));
    print "<tr><td bgColor=\#FFFFFF align=left><a href=$key>$key</a><b
+r><img src=../images/pix.gif width=1 height=2><br>
    <img src=../images/blue.gif width=$width height=10></td><td     bg
+Color=\#FFFFFF align=right valign=bottom>$browse{$key}</td></tr>\n";
}
print "</table><p>\n";
} 

sub BrowserSort { 
  $browse{$b} <=> $browse{$a}; 
} 
    
&getBrowsers;

unlink($temppath);

[eibwen]

sorting is very greedy

Have you read perlre and the Regex Tutorials? Or perhaps you were referring to something else entirely. If you were to post the specific excerpt or at the least be more specific with the regard to the problem, perhaps we can help.

[Anonymous Monk]

What I did was take the information in the database and split the lines, ran them through a for loop and printed them to a temp file. So if the first line was 12|netscape, then I printed netscape 12 times in the temp file, if the second line was 10|microsoft, I printed it 10 times in the temp file, and recounted them and resorted them. This is never going to work when I get to number above 1000. What I would like to do is just sort the lines in the original database numerically, on the first field, but with out rewritting them, if that's possible.

Take the code and chop off everthing after EOT and see if you can sort the $logpath file numerically.

#!/perl/bin/perl -w

use CGI;

$q = new CGI;

for $key ( $q->param() ) { $form{$key} = $q->param($key); }

$logpath = "data/browser.dat";  

$browser = $form{'browser'};

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

$not_found = 1;

if($browser) { 
    open (LOG, ">$logpath");
    foreach $line(@data) {
        chomp($line);
        ($fcount,$fbrowser)= split(/\|/,$line);
        if ($fbrowser eq $browser) {
        $not_found = 0;
        $fcount = $fcount+1;
        print LOG "$fcount|$fbrowser\n";
        }
        else {
        print LOG "$line\n";
        }    
    }
    close(LOG);

    if($not_found) {
    open (LOG,">>$logpath");
    print LOG "1|$browser";
    close(LOG);
    }
}

# prints out log and form to browser

print "Content-type: text/html\n\n";

open (LOG, "$logpath");
@data = <LOG>;
close(LOG);

foreach $line(@data) {
chomp($line);
print "$line<br>";
}
close(LOG);

print <<EOT;
<FORM ACTION="basic.pl" METHOD="POST">
<SELECT NAME="browser">
    <OPTION VALUE="microsoft">microsoft</OPTION>
    <OPTION VALUE="netscape">netscape</OPTION>
    <OPTION VALUE="aol">aol</OPTION>
    <OPTION VALUE="mozilla">mozilla</OPTION>
    <OPTION VALUE="opera">opera</OPTION>
    </SELECT>
<INPUT TYPE="SUBMIT" VALUE="Submit">
</FORM>

EOT

[Anonymous Monk]

Once we muddled through the cgi.pm trench, the main problem was the broken loop or the script printing to the browser instead of the file.

It didn't cause an error so I didn't notice it.

Thanks again everybody.