Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain

by IrishSteve (Initiate)
on Aug 31, 2008 at 00:16 UTC ( [id://707970]=perlquestion: print w/replies, xml ) Need Help??

IrishSteve has asked for the wisdom of the Perl Monks concerning the following question:

I've spent way too many years working in the comfortable arena of COBOL and proprietary operating systems and databases on multi user minis, after some interesting experiences with things like BASIC on another multi user mini many years ago.
I then got diverted for a number of years into networked PC based flight simulator systems, which meant exposure to some very specialised interface hardware for quite some time, with a bit of DELPHI work along the way, and more recently, I've been doing PC support of things like end user Microsoft (ARRGGHHH!) and related software/hardware issues, of which there have been plenty.

Now, for all sorts of reasons, I'm working on a web based shopping platform for a close to home client, my daughter, and so far, (3 months) while we've got a live and working custom built web shop, there's a lot of things I want to do with it, because we need to, but an absence of in depth knowledge of PERL and all related software is proving an obstacle to the planning of the next phase.

The recent learning curve of enough HTML, JAVAscript, PERL and UNIX to survive on a shared web host has been close to trying to climb the North Face of the Eiger, but it's been worth the effort, in that while the shop is VERY rough round the edges at the moment, we do have a working on line shop with payment system that is reasonably fast, easy to use, and has been attractive enough to persuade people to use it and place orders.

Now, I want and need to tidy it up, (hide my debug code among other things) and add in the condiderable number of extra functions that the buying customer won't see as such, but that will make our end of it a lot easier, faster, professional and more economic to do the job, but by doing it in a way that retains the integrity of the ease of operation which we have right now.

To complicate things, the original plan was that I wasn't doing any of the server side work, but a fundamental disagreement about the way it was going to work, combined with a total absence of documentation from the server owner/operator meant that the original plan and server had to be abandoned in mid flight, which meant taking space on a shared server, and getting to grips with server side concepts, as well as enough PERL & UNIX to get it live.

The longer term plan is to move to our own dedicated facility in house, based on DELL servers, but the exact decisions about windows/SQL or UNIX/MySQL are not yet firm, having had to go with UNIX/Apache/PERL so far is making me think that remaining on that side of the fence may well be an attractive option, and at present, we're not using any database as such, so that decision can be deferred for a little while.

That's where the lack of in depth knowledge is a problem right now, in that I need to do some serious system planning without being sure that the underlying system software I am going to be using can easily accomodate the direction I am thinking of taking.
What I am thinking is that there will be a considerable number of things that I want to do that could be triggered as a response to POST information from a web screen, and at the same time, providing there is an easy way to load the POST information as part of a PERL thread, that would make a very powerful and easy way to also be able to do the same thing as a spawned thread from on line processing.

An example might be that I (will) have a script that will take just an order number in, and produce an E-mail transmittable copy of the invoice relating to that order, based on database lookup, which would be very helpful in allowing a customer to ask for and receive a duplicate copy of the paperwork.

At the same time, if I could load the information into a POST record for that script, it could also then effectively be used as the original production script, and by virtue of the POST and script being a self contained unit, or possibly module, once it's working, it can be used wherever it is needed to produce that document.
The same is then true for many other aspects of the system, from a design testing and operational view, and code maintenance becomes much easier, in that modules can be tested in isolation, with known data from test databsse sources, which makes many tasks much easier.
Not a new concept, it worked very well on the mini's that I used to work on, and I used the concept all the time for many file maintenance and print jobs.

It's a development of the subroutine concept, in that the methods are the same, create a subroutine, give it defined input, and let it get on with the job, but as I will want to use the same subroutine from an on line screen, the parameter input I want to use is the POST record, if that makes sense.

So, my question, based on very little real exposure as yet to complicated PERL, which at present I'm running on a shared hosting UNIX based APACHE box, is initially relatively straightforward.

Is there an easy (or relatively easy) way to spawn a CGI script from within a CGI script, using the parameter concept I've outlined above, while allowing the spawning script to continue to run, which will probably result in a number of other spawns happening?

It's possible that I am looking at the wrong language for this, and that I should be looking at something like PHP, or .NET, but right now, having spent the length of time I have getting started with all that I have, I'm reluctant to make the mix even more complicated, and so far, while the syntax and implementation of PERL has been hard to work with with compared to the luxury of precompiled code etc on the mini, at things stand now, I can see no valid and pressing reasons to go in other additional directions.

So, Pointers, ideas, alternative suggestions all appreciated at the moment, in order to avoid making any more mistakes :>), there's been enough of them already.
Sorry the message is a bit long, I couldn't see an easy way to explain where I'm coming from without giving a bit of my background first.


  • Comment on Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain

Replies are listed 'Best First'.
Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by wfsp (Abbot) on Aug 31, 2008 at 08:07 UTC
    Hi IrishSteve,

    Welcome to the monastery! You've had your introductory lecture on capital letters now we move on to the next hurdle. :-)

    Hurdle 2: We need more information.

    ...spawn a CGI script from within a CGI script...
    In a typical CGI app with several pages each page is often referred to as a runmode. Each runmode handles different aspects depending on what stage the user has reached.

    It may be useful to have a look at the CGI::Application docs (as an example, there are others) which explains how you might tackle a CGI app.

    Perhaps also CGI::Session on how to handle "state" in a stateless environment.

    As an aside, the "POST record" is usually kept in, and referred to as, a query object. Are you familiar with which handles all that side of things for you?

    Apologies if I've missed the point here but some more information would be helpful.

    And be very, very careful with that caps lock. :-)

    update: attempted straightening out of the grammer

Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by ikegami (Patriarch) on Aug 31, 2008 at 02:33 UTC

    I can't find where you say why you want to spawn a child or why you need to interface with this child using CGI.

    But if there is a need for a long running process, merlyn wrote a useful article on spawning a worker process from a CGI script.

    By the way, you seem to have a problem spelling proper names.

    • Perl is not spelled "PERL".
    • JavaScript is not spelled "JAVAscript" (and isn't related to Java).
    • UNIX is a trademark for a specific system, while unix refers to UNIX-like systems such as Linux, FreeBSD, etc.
    • Dell doesn't seem to spell their company name using only uppercase letters except in its logo.
Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by ikegami (Patriarch) on Aug 31, 2008 at 08:02 UTC

    I decided to give it another go. I still don't understand what you're asking, but the key question seems to be

    but as I will want to use the same subroutine from an on line screen, the parameter input I want to use is the POST record

    Are you asking how to extract data from a POST request? If so, it's as simple as using CGI's param method.

    You might also be interested in sessions, data stores that persist between page requests.

Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by bart (Canon) on Aug 31, 2008 at 13:41 UTC
    I think the magic keyword you're looking for, especially if the server is on a Unix-like system, is fork. That way you get 2 copies of the same process, with the same data, so the child process can just continue with the processing, while the parent handles the interface.

    There are some issues to be careful about, especially in a CGI setup, and for that, I recommend looking into the articles (AKA "columns) that Randal Schwartz, author of the famous O'Reilly book "Learning Perl", and locally known as merlyn, has written for several magazines (including Web Techniques and Linux Magazine), and collected on his website. Try the Google query: cgi fork.

Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by SFLEX (Chaplain) on Aug 31, 2008 at 11:49 UTC
    I have started many customers on a shared server and there business/(web site) runs fine to this day. If the traffic is to much for a long period of time. Then you would want to get a dedicated server. But there is no seance in getting an over priced server before its time.

    Here are 2 very nice shopping carts that handle multiple gateways and are free, I believe you can pay $50.00 for a life time membership that allows you to get all the add-on stuff.

    Personally I would use a SQL or query back-end.
    Commerce-CGI Good but flat file.

Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by Anonymous Monk on Aug 31, 2008 at 08:30 UTC
Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by hangon (Deacon) on Aug 31, 2008 at 18:29 UTC

    Wow. Thats a lot of verbage for some vague questions. I know its difficult to articulate about something you're not familiar with, but it might be easier to just ask direct questions and we'll let you know if we need more information.

    An example might be that I (will) have a script that will take just an order number in, and produce an E-mail transmittable copy ...

    One concept that you don't seem too clear on is modules. In Perl, modules are what you use for reusable code. They are also what you use to write a modular program. If your cgi script spawning cgi script concept is intended to achieve modularity, you would be much better off writing modules instead of spawning more scripts.

    Otherwise, be careful forking or using multiple threads with cgi. Unless you specifically need a long running process and need to return a screen to the user before a process is finished, you probably want to avoid this.

    The best advice I can give you at the moment is get to know cpan. There is a very large collection of modules at your disposal that can save you a lot of work. Cpan is your friend.

      Wow, all sorts of ideas, suggestions, brickbats and the like.

      Makes me feel right at home, I used to moderate a flight simulation forum for a number of years, and thread police were every bit as active there too :<)

      OK, clarify, and yes, the point about not knowing the Perl jargon is well made, that's the major hassle, getting to know the terminology of the language.

      I perhaps understated a few things, in that while Perl, HTML and related language use is new to me, on line real time transaction processing and programming is not, I've been doing that for over 20 years, albeit on different hardware and software.

      On the mini, we completed a screen, sent it to the processor, and it ran a thread, (stateless) which at the end probably sent some form of response back to the user, sometimes with data embedded, sometimes as a data collection screen, sometimes both. So far, identical to what is happening on the server, albeit different languages and communications protocol.

      One of the things we could also do was to start an asynchronous thread/task that did not return to the user but did whatever it was supposed to, and then exited.

      In this case, it was a short task, designed to reduce the response time to the user, by dealing with the less urgent stuff at a lower priority, or in a lower priority thread process routine

      Yes, we could also run large "batch" type jobs using a similar system, but the constraint there was things like record and file locking implications, so the "batch" jobs usually had to all share one thread and were queued, to avoid deadly embrace conflicts, and in some cases, they were set to run on a very delayed basis, like after the on line day was completed.

      That's the other difference, there's no such concept in web work, as it's 24/7 unless the server is taken off line for some reason

      So, I'd like to be able to do something like the first thread concept, so my logic flow is:-

      collect screen input,
      create some DB records or whatever,

      set up the mail "screen" record, and start that process, (it might well queue depending on the host machine)

      complete the main line task,
      refresh the user screen with the result of the transaction and finish this thread.

      If it's uniprocessing threads, as soon as that finished, the mail process loads, executes and exits, happy unless something nasty has happened.
      The beauty of that is that the identical process/program/screen can be used stand alone to generate a copy of the original, without any extra code, and testing it a lot simpler, as it's stand alone, with no other integration to worry about, and implementing it in the larger thread is simply a case of define the buffer, load the variables in to it, associate it with the thread and then submit it by whatever means the system supports.

      The other advantage is that I've sent the user the response and then I deal with the internal housekeeping stuff that needs to be done, without making the user wait for the response, which keeps the user happy, as the response time is shorter.

      As far as I can see, that's not Fork, in that I don't want to run another copy of the same thread, and (terminology again) I don't know what else it might be described as in order to search CPAN, which believe me, I have been doing, with considerable success and a few dissapointments along the way. On the mini, it was a background thread, but in every other respect, it's just another script being activated, but instead of coming from a browser, it's coming from the application itself

      Yes, I could run it as a subroutine from a library file, and just include it at runtime, but I still then have to have a front end screen to test it, and do the other things, wheras if I can create it as a complete script, it make for less work, maintenance and documentation, all of which I will appreciate as this thing grows over time, and believe me, it will. I also lose the response time advantage by running it as a sub

      And to the cynic who suggested forget the original and start again, not an option, it's been live for 3 days, and we've taken 15 orders from a cold start, and while I know there are several things that I will be very definitely changing as time goes on, and I get to understand Perl better, the underlying HTML and end user "feel" will not be changing much, as the response we've had so far is very positive.

      Relational database, Global positioning, user history, previous order recall, real time link to delivery drivers, it's all been thought through, and will happen, now that it's earning money.

      Thanks guys and gals, hope this makes what I want to do a bit clearer


        More brickbats I'm afraid. :-)

        I would add to hangon's suggestion, you really want to avoid considering forking and threading until you are really sure you need it. You won't know you need it until you set up your app and it runs to slow. Even then there are other methods that would likely be more effective at speeding things up (e.g. mod_perl, better hoster, hardware). We're keen to see evidence of using a 'library' (module) slowing down a cgi app (give us the numbers :-).

        Using a relational db (e.g. MySQL) will take care of a lot of your worries - that's what it's for.

        Generally, there isn't anything in a typical shopping site that would be considered "a long running process". And, again, you won't know for sure until you've tried it.

        Something like CGI::Application with CGI::Simple, CGI::Session, HTML::Template, DBI could well give you the reponse times you want. You will also have something that you and others will find more straightforward to maintain. There are many successful shopping sites that do just this.

        CGI::App comes with many plugins that will do most of the tasks you need virtually out of the box, it comes with good docs, tutorials, examples (including shopping) and has its own web site (there are others, but I think CGI::App is a good introduction).

        I can't help feeling you're introducing massive complexity when you don't need to. Set the thing up. See how it goes. Identify any bottlenecks and the monks will be pleased to wade in and help. While you're getting 15 orders every 3 days is a good time to try it. :-)

        Good luck!

Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by Anonymous Monk on Aug 31, 2008 at 08:40 UTC
Re: Very new to PERL, want to see if CGI spawn concept is possible and or feasible without major pain
by cosmicperl (Chaplain) on Aug 31, 2008 at 00:38 UTC
    Hi Steve,
      Hope I don't sound rude, but your post is far to long to get through. Although from what I gather you need to look at multi threading with Perl or possibly FastCGI with Perl.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://707970]
Approved by cosmicperl
Front-paged by wfsp
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (3)
As of 2024-07-19 22:41 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.