Using CGI::Application in large projects

[uzzikie]

Fellow Monks,

i need some advice regarding the use of CGI::Application. Its more than a case of Scaling single-script CGI applications....

As stated in the synopsis, it is no doubt useful in creating sophisticated, reusable web-based applications. But what happens if i have hundreds of run-modes? Even though i can organize them neatly into different modules, what worries me is the overhead. Will CGI::Application attempt to load ALL the modules each time the script is accessed, even though they are not used? (now that's scary!)

Any thoughts and recommendations to handling such a situation?

[crazyinsomniac]

How can you possibly have hundreds of run modes (are they really tiny? how are they related? Can you draw me a picture? -- outline the app, it's functionality, the modules it uses, the enviroment it runs in .... a picture)?

Has it dawned on you to split this application up into 3-4-20 different applications?

CGI::Application will LOAD EVERY MODULE YOU USE, as would any other perl program if you use all of them. The alternative to useing a module, is requireing it (perldoc -f require), or simply eval-ing the use statement (perldoc -f use).

Any thoughts and recommendations to handling such a situation?

Yes, use strict, use warnings, use mod_perl, and don't worry about it (or, restructure your app -- can't really suggest how cause I don't have a picture).

 


______crazyinsomniac_____________________________ Of all the things I've lost, I miss my mind the most. perl -e "$q=$_;map({chr unpack qq;H*;,$_}split(q;;,q*H*));print;$q/$q;"

[rob_au]

In such a scenario, I would look at the manner by which the dispatch is ordered, seeking for any hierarchy which may be utilised to further define run-modes - I would then separate those group run-modes of similar functionality into separate dispatches. These separate dispatches could then be moved into separate scripts with a shared server-side backend where required.

If this was not possible, I would investigate the dynamic loading and unloading of module components through the use of the use and no module functions - This should allow memory to be allocated and freed as necessary but the details of whether this memory is returned to the system is dependent upon the runtime C library.

In any case, I would be incorporating the CGI::Application dispatch into the apache binary with mod_perl. Based upon the intended scope of the project described, the performance advantage of employing mod_perl cannot be ignored.

 

perl -le 'print+unpack("N",pack("B32","00000000000000000000001000100100"))'

[IlyaM]

Since both mod_perl and dynamic loading and unloading have been mentioned in same post I'd like to mention that dynamic loading of modules should be avoided in mod_perl scenario as it actually wastes more RAM. You should try to preload all modules you use during server startup. It is covered in mod_perl guide.

Another note is that no Module doesn't unload module. It simply calls module's unimport method. See perlfunc:use.

--
Ilya Martynov, [e-mail]
CTO IPonWEB (UK) Ltd
Quality Perl Programming and Unix Support UK managed @ offshore prices - http://www.iponweb.net
Personal website - http://martynov.org

[Ctrl-z]

as crazyinsomniac pointed out - you may just be implementing it badly. You should check out the CGI::Application mailing list (see POD) for common mistakes.

That said, Ive found there is a point where CGI::Application starts becoming counter intuitive. Theres not much of TMOWTDI and its "methodology" starts creating its own kind of mess, where it cleaned up a previous one.
Take a peek inside - theres no deep magic going on. You could probably hack together a base-class that does exactly what you want and scales how you need. Assuming you can afford the time.

[Ryszard]

I'm using C:A with hundreds of run modes. It works prefectly.

What I have is heaps of seperate and distinct applications, each application is self contained, ie an application for "moving house" (i work for an ISP) and an application for managing IP's. As you can appreciate these are very distinct modules.

What makes them part of a single user interface is session management and access control, which is done centrally. The structure we use is a layer on top of C:A that handles the session management and access control that has use base CGI::Application in it (lets call this module "control")

Whenever a new application is coded we need to add a use base Control; to it, and there you have it, implicit session management and access control.

All the business logic is mapped out at this level. All the external access (to DB's, sockets etc) is abstracted away from this into seperate modules which any application can access.

Experience thus far with this type of design has shown it can scale quite well (we use an RDBMS for access control and session management).