Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

The CGI Help Guide

by tachyon (Chancellor)
on Jul 04, 2001 at 18:35 UTC ( #93867=perlmeditation: print w/ replies, xml ) Need Help??

This is another tutorial node for which I would request the wisdom of the Monks to correct typos, inaccuracies, omissions, etc.

This node is being developed to help you get your CGI script working. It is not a Perl tutorial and focuses on CGI issues. It owes much to Tom Christiansen's terse but informative The Idiot's Guide to Solving Perl CGI Problems. I recommend you check this out if you can not find the answers you need here.

What follows is a sort of checklist to help you solve the problem quickly yourself. It is divided into four parts:

A. Before uploading to the Server

B. Uploading to the Server

C. Setting Up on the Server

D. Debugging

 

A: Before Uploading to the Server

1. Does your script compile?

Like any Perl program a CGI script needs to successfully compile before it will run. Until it does this you can forget everything else. To test this you need a command prompt and a working Perl interpreter. If you have *nix you are probably already there. If you have DOS/Windows or a Mac you will probably need to install Perl. You can get a binary copy of Perl, ready to install, for almost any platform here. If you want to compile the source yourself go here.

Once you have a copy of perl you can access from a command prompt ( C:\> in this example under Windows) type:

C:\>perl -c myscript.pl

What this does is compile (but not run) your script. You should see this:

C:\>perl -c myscript.pl
myscript.pl syntax OK

Here are some common errors and solutions:

No Perl!

C:\>perl -c test.pl
Bad command or file name
C:\>

This is telling you that your operating system can not find the Perl executable. Either it is not installed and/or it is not included in your path environment variable. So install perl and either include it in your path or type the full path to the executable. How you find perl depends on your system. On *nix type 'which perl' at the command prompt to get the path (usually /usr/bin/perl). On Windows use Start|Find|perl.exe to get the path (probably C:\PERL\BIN).

A bit of background: the PATH environment variable is a list of directories separated by colons (*nix) or semicolons (Dos/Win). When you type a command name without giving an explicit path your shell searches each directory in the PATH list in order, looking for an executable file by that name, and the shell will run the first matching program it finds.

To check the path on Win32 use:

C:\>PATH
C:\;C:\WINDOWS;C:\WINDOWS\COMMAND;
C:\>

To add to the path use:

C:\>C:\PATH %PATH%C:\PERL\BIN;
C:\>

To check the path on *nix use:

$ echo $PATH
/bin:/usr/bin:

For csh users use something like:
setenv PATH :/bin:/usr/bin
For sh or ksh users:
PATH=:/bin:/usr/bin export PATH

Syntax Errors

C:\>perl -c test.pl
String found where operator expected at test.pl line 1, 
near "printt "Hello World!""
(Do you need to predeclare printt?)
syntax error at test.pl line 1, near "printt "Hello World!""
test.pl had compilation errors.
C:\>

This is a simple syntax error typo problem. Perl can not find the function 'printt'. You need to fix all these.

Missing Modules

Modules are Perl's versions of libraries. There are modules available to do everything from read in CGI data to allowing you to write your script in Latin! To use them they need to be installed.

C:\>perl -c test.pl
Can't locate Some/Module.pm in @INC (@INC contains: 
C:/Perl/lib C:/Perl/site/lib .) at test.pl line 1.
BEGIN failed--compilation aborted at test.pl line 1.
C:\>

This is telling you that the module Some::Module required by a 'use Some::Module;" directive is not installed. You will probably need to install the module. To install a module see a detailed How-To for locally installing modules and installing modules on Win32. You *may* be able to get away with commenting out the 'use Some::Module;' directive to test compile you script but this will fail if you have code like 'print Some::Module->Cool_Function;' later on.

2. Are you using strict and warnings?

#!/usr/bin/perl -w # "-w" turns on all sorts of warnings about probable errors. 

use diagnostics;   # optional; causes warnings to be explained in greater detail.

use strict;        # generates compile and run-time errors for certain unsafe constructs.

While use strict and warnings are not required for any perl script to run (including CGI) they can help you sort out all manner of problems and are highly recommended. For a complete discussion of this see Use strict warnings and diagnostics or die

3. Are you using CGI.pm to parse your CGI input data?

The leading block of reusable code for CGI purposes is CGI.pm. In a nutshell there is a lot more to parsing CGI data than first meets the eye - both from the functionality and security point of view. If you can't give ten good reasons not to use CGI.pm then *use it*. It is part of the standard perl distribution so should be available on any system that has Perl. Using it to get your form parameters is as easy as:
use CGI;            # use CGI.pm
my $q = new CGI;    # create new CGI object

# now get value of a form parameter named 'name'
my $name     = $q->param('name');

# now get an array of values of all form parameters named 'option'
my @options  = $q->param('option');  

As you can see it is very easy to use CGI.pm. Good entry points to all things CGI.pm, and why you should be using it are: use CGI or die and no excuses for not using CGI.pm.

B. Uploading to the Server

1. Did you upload in ASCII mode?

Assuming all has gone to plan so far we are ready to upload the script to the server. The server is the computer that is actually connected to the WWW and whose job it is the serve up HTML documents and process CGI scripts. A Perl script is just a text file. Unfortunately different computer operating systems handle text files differently. On *nix the line ending is a line feed (\n). On Dos/Win it is a carriage return line feed (\r\n). On Macs it is a carriage return (\r). Clearly some conversion will be in order when sending files from one system type to another. Never fear the problem has been solved for you. All you need to do is to make the transfer (usually by FTP) in ASCII mode. In this mode the line endings will be automatically converted for you.

Where you can strike problems is when you transfer a file in *BINARY* mode. In this mode the file is sent exactly as is so line endings are not converted. This can and does lead to problems. Make sure you transfer in ASCII mode.

2. Did you upload to the correct directory?

Most servers have a specific directory used to store Perl CGI scripts. Typically this will be something like:

/www.mydomain.com/cgi-bin/

It makes sense that you will want to get your script into this directory. You may need to confer with your systems administrator about this. BTW I can personally recommend Cute FTP for Win32 as very easy to use for this task.

C. Setting Up on the Server

OK so now we have our script in the cgi-bin. We need to set it up so that it can be executed by the server when it is called. Here is how CGI works in a nutshell:

(i). Browser requests a resource via a URL that is aimed at a CGI script, say via a link like:

<a href="http://www.mydomain.com/cgi-bin/myscript.cgi">Run my script</a>

(ii). The server receives the request and goes "Ah ha this is a cgi script that I need to run!"

(iii). The server executes the script passing it data via the environment variables in the $ENV hash and via STDIN. The script can access this data and process it. The script generates output on STDOUT

(iv). The server takes the output of the CGI script from STDOUT and returns it to the browser pretending it received a request for a static HTML document.

We can have problems in any of these areas. Of course!

1. Is the HTML document that calls the script pointed at the correct place?

In order to execute the script you need to ask for it to be executed. If you link to the URL above and your cgi-bin happens to be called localcgi (like mine) then you will get a 404 Document not found error. Of course the server can not find the document in /www.mydomain.com/cgi-bin/, it is after all somewhere else!

2. Is the server configured to execute the script, and are the permissions set?

You can't execute an HTML document, well at least Perl can't compile it! You also don't want to return the code in your script to the browser when it is requested. So it is obvious that the server needs some way of deciding if it needs to return the requested document of execute it. As you might expect there are many ways to achieve this result. Your systems administrator should know the exact details for your system but here is a general guide.

Under *nix all files have permissions. They have permissions for the owner, the group, and everybody else. The permissions are 4=read, 2=write, 1=execute. Thus a permission of 7 means that the file can be read , written and executed as 4+2+1=7. A permission of 5 means the file can be read and executed as 4+1=5. As a general rule you will want permissions of 755 for your file which grants you read, write and execute permission and everyone else read and execute permission. You set this using the command chmod 755. Directories also have permissions which need to be correct but they should be fine if this is a standard cgi-bin.

Some servers are configured to recognise any file that ends in .pl, .pm, .cgi as a Perl executable (especially WIN32) and act accordingly. You will need to use one of these extensions if that is the case so the server knows to execute the file. Still other servers assume that all files in certain directories (ie cgi-bin) are executable. The bottom line is is all else fails then check with your sysadmin.

3. Is the shebang line pointed at the Perl interpreter?

Under *nix if a request for a  file  marked as executable is made then the first 2 bytes are examined for the #! shebang sequence. If this is found then the shell executes that file using the executable file it finds at the path following the #! Thus what this line:

#!/usr/bin/perl

 actually says is: "Dear shell, if this file is called and it has read and execute permission please execute it using the executable file in the /usr/bin/ dir called perl". If the Perl executable is not in the /usr/bin/ directory then the shell will complain about not being able to find Perl. Unless the script has permissions of at least 5 (read and execute) the shell/Perl will not be able to read it (as it must) in order to execute it (as we want)

Some servers are configured not to allow the execution of CGI scripts for security reasons, once again talk to ye sysadmin.

D. Debugging

1. Before doing anything else add this code to the top of your script. 

If you add this code your script will run for long enough that any errors appear in the browser window making it easy to see the problem, presuming of course that you have followed the advice above! This will avoid you getting the less than useful 500 Internal Server Error  or Premature end of script headers/Malformed script headers messages. You add this code just below the shebang line and before everything else. That's everything else. The reason for this is to minimise the lines which can cause problems before we reliably direct output (including syntax errors) to the browser

#!/usr/bin/perl -wT

# ensure all fatals go to browser during debugging and set-up
# comment this BEGIN block out on production code for security
BEGIN {
    $|=1;
    print "Content-type: text/html\n\n";
    use CGI::Carp('fatalsToBrowser');
}

# all the rest of the code goes here

Because this code is in a BEGIN block it is executed before everything else. Even before most of the script is compiled. In it we do three vital things to ensure that all errors from this point on will appear in the browser window thus making debugging *much* easier. You could look in the server logs to get the same information but as you may or may not have access to them and they can be anywhere on the server it is easier to use this instead.

The $|=1; forces buffer flushing. This ensures that output from the script goes immediately to STDOUT. Let's just say trust me on this or read Suffering from Buffering for all the gory details.;

Next we print a complete valid header. In the HyperText Transport Protocol (the http bit in http://www.mydomain.com) every request and response needs a valid header. It may or may not include a body but it must have a header. The end of the header is recognised by the \n\n sequence. This prints one blank line. If, for any reason, a blank line is printed *before* the "Content-type: text/html\n\n" you will get a premature end of script header error. By printing a valid header we avoid this. Note that I have presumed you want to output HTML but you can also output GIFs, JPEGs etc using a different header that specifies these different data types. The headers our script generates without this block will appear in the top left of the browser window where we can check that they are as expected. If you don't see them and your script now works your problem is that you are not outputting any valid header info.

Finally we add CGI::Carp which essentially does the same as lines 1 and 2 but in my experience is not as reliable as desired. Combined with the two lines above virtually all errors appear in the browser window with an explanation.

While we are in this topic the -w switch on the shebang line provides heaps of free help. If you want more detail add the line use diagnostics; as shown:

#!/usr/bin/perl -wT

# comment this out on production code as it uses a lot
# of memory and you won't have any errors left anyway :-)
use diagnostics;
You may be wondering about the -T switch. Or you may not be. Regardless you want to use it. The -T switch switches on taint checking which is Perl's unique anti hacker mode which warns you if you are doing dangerous stuff that makes it easy to hack your server via your script. See The World Wide Web Security FAQ for more details.

2. Are you checking the return values from the functions built in to perl?

Most of the file and system functions set $! and have return values that you can test thus:
open(FILE, "</file.txt") or die "Error opening /file.txt Perl says: $!\n";
$! will contain an error message that will give you more information on where your program is going wrong. The perlfunc man page will give you more information on the return values from functions. It is very handy to know that the reason your script is not working is because you can not find/open/write to a file for instance.

3. Are the modules required by your script actually installed on the server?

As we saw earlier if you script uses modules they need to be installed. Some modules come with perl (like CGI.pm) but others need to be installed. If you have followed the advice so far you will get the typical:

Can't locate Some/Module.pm in @INC (@INC contains: 
C:/Perl/lib C:/Perl/site/lib .) at test.pl line 12.

error message in your browser window. Get you sysadmin to install them. Or see the installing modules links above.

4. Looking at the program as it runs.

Like any program a CGI may run but not produce the output you expect because of logic errors. Provided you have already printed a valid header you can add temporary debugging print statements within your code to print out variable values and flags to allow you to follow the progress of your code as it runs. My code often looks like this:

...
my $debug = 0;  # set to true for debugging
...
print "Variable \$my_var contains '$my_var'\n" if $debug;
...
sub foo {
    print "Running sub foo!\n" if $debug;
    ...
}

The $debug variable makes switching the debugging prints on and off easy - that way I can leave them in my code. The often come in handy again when you make changes.

Firing up the Debugger (*nix only)

For details on how to get a Perl CGI script to fire up a debugging window under X Windows on your local machine see Debugging a CGI by toma

5. Don't be afraid to Ask 

If you have tried all the stuff above and have an error you can't fix then Ask the Monks. Many Monks do this stuff for a living so can probably help. In fact you will often find that if you show you can help yourself people will fall over themselves trying to help you. Good luck and enjoy the journey.

Credits

The following monks have contributed to this document Albannach, sierrathedog04, converter

tachyon

s&&rsenoyhcatreve&&&s&n\w+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

Comment on The CGI Help Guide
Re: CGI Help Guide
by sierrathedog04 (Hermit) on Jul 04, 2001 at 19:30 UTC
    I do not agree that your CGI scripts should be executable by everybody. As you are well aware, the default configuration is for the Apache webserver to run all scripts as "web" or some other generic webuser. There is no reason for everyone to have execute permission or even read permission on a CGI script.

    Update: Removed comment on CGI.pm.

Re: CGI Help Guide
by converter (Priest) on Jul 04, 2001 at 20:38 UTC

    Oh, goodie, another chance to put in a plug for CGI::Debug! I think CGI::Debug should be a standard module. I won't go into all the details, suffice it to say that while developing CGI code, 'use CGI::Debug' at the top of the code will make it much easier to catch and fix bugs.

    A lot of visitors to DALnet #perl have problems with code that runs fine locally or on their shell on the server, but dies when run by the server uid (permissions usually), but they can't get at the logs to find out why. A neat feature of CGI::Carp is the carpout subroutine. carpout takes an open FILEHANDLE and directs STDERR to it. Give carpout a FILEHANDLE opened on a file in a directory under your home and you have access to perl's errors and warnings output. This won't help you with server errors, but with proper error handling in your code, you'll usually have enough to go on.

Re: CGI Help Guide
by marow (Initiate) on Sep 03, 2001 at 23:55 UTC
    Don't know if this appropriate or not, but I have recently been working a lot with cgi and mysql and after a certain amount of debugging hassle all sql statements were moved to a subroutine enabling us to see 1) what we actually tried to run 2) how long , how many records 3) any error returned
    print "\nadding url $addurl to table sajter"; &runsql("insert into ah reason='updated' add=$serverid") print "->\n"; #end debug info sub runsql { my $myquery=$_[0]; my $exitonerror=$_[1] if($_[1]); my $starttime=time(); unless($dbh){ $sqldebug .= "\nTry SQL: connect to db $hostname, 'uptime_admin'"; $dbh = Mysql->Connect($hostname, 'uptime_admin', 'loguser', 'poopypant +s') or die "Can not connect to db\nINFO: $sqldebug"; $sqldebug .= "\nConnect OK'"; } $sqldebug .= "\nTry $myquery"; $runsql_sth = $dbh->query($myquery); my $endtime=time(); if($runsql_sth){ $numrows=$runsql_sth->affectedrows; $sqldebug .= " - $numrows rows"; }else{ $errmsg = $dbh->errmsg(); $sqldebug .= "\n -- ERROR -- '$errmsg' "; if($exitonerror){ die "severe SQL error occurred INFO: $sqldebug"; } } $sqldebug .= sprintf " -- Done in %d secs ",$endtime-$starttime; return $runsql_sth; }
      You will find your code easier to read if you pick a consistent indentation style and stick to it. What exactly you pick, hanging vs inline braces, etc doesn't so much matter as long as the indent is in the range 2-4 spaces and the style is consistent. If you do that then it is possible to see the logical structure of the code at a glance, which greatly simplifies understanding it.

      Also the better way to do the above is to have a logging function that you pass messages to. By default it would do nothing, but turn on a flag and you get a useful debugging trace out...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://93867]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (9)
As of 2014-09-02 10:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (21 votes), past polls