in reply to
cgi,wait for long running external prog
Two thoughts here ...
First of all, in this case you should think of “a web page” as merely being a form of user-interface. The fundamental nature of this user interface is that it appears, it exists, and then (forever ...) it is gone. The user does not want to wait until his request is actually completed; nor should he have to; nor does the HTTP protocol support it. Therefore, if you have long-running work to do, you need a separate mechanism, entirely apart from any and every web-page, entirely apart from the Apache-or-whatever web-server, to do that.
Second, “batch-job managers” already exist for every computing platform. So do (web, and other) interfaces to them. Therefore, your present requirement does not consist of “rolling your own [anything].” It consists of finding, and employing, prior-art. You are in Rome. Find some Romans, and go do what they are [already ...] doing.