The actual computation takes about 4-5 minutes and I am using a RedHat Linux OS. I return some 1000 rows of data.
Unless the rows are huge, 1000 rows is not a lot of data, so you should be OK, there. The original issue is almost certainly the computation time. Redhat definitely supports fork, which is good.
So you are saying, in the child I store the result in a global variable($temp) and in the parent,I will have only one line that redirects the url to another cgi script with this $temp as parameter. i.e
No. Once the fork() takes place, the parent and child are separate processes. Updates to $temp (or any other memory, for that matter) in one process will not affect the other process whatsoever. Then, the parent just prints the redirect header with the display_id and exits immediately. In five seconds, when the browser sends a request for the display_id, the server creates another entirely new process to service the request.
That's why I suggested a temporary file (or temporary database storage); you need some way to keep persistent state (and know where to find that state), because the client connection is created and destroyed every few seconds when the refresh hits. It's a simple form of IPC.
But how will I know when $temp is populated completely? :(
That's up to you, but one simple method is to designate an "end of transmission" (EOT) marker that will never appear in the normal output. In the child, print the EOT to the end of the same temp file once the operation finishes. In the other CGI that reads the display_id file, if you see that EOT, you know the job has finished, and can take whatever action is appropriate.
