Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Perl tk question

by ghosh123 (Monk)
on Jul 29, 2013 at 16:41 UTC ( #1046889=perlquestion: print w/replies, xml ) Need Help??

ghosh123 has asked for the wisdom of the Perl Monks concerning the following question:

I have a perl tk gui which can run some 100 jobs. Its inter-process communication is by socket communication. The gui interface has no idea about what job the internal engine is running. Through socket they communicate, and the gui display objects updates the gui pass/fail results as per the info through socket. Basically the gui serves as a server.
It stores the results in MySQL database.
My question is what could be the potential problem to make such a gui to be able to run some 20k jobs . What are the factors to be taken care of. The code has as much as 50 perl module files and each file has more or less 1k lines.

Replies are listed 'Best First'.
Re: Perl tk question
by arkturuz (Curate) on Jul 29, 2013 at 18:05 UTC
    the gui serves as a server.

    This is good. There shouldn't be problems if the number of objects increases as long as any one of them doesn't lock the whole application.

    It stores the results in MySQL database.

    The database will be a bottleneck. 100 clients and 20000 is not exactly the same if they want to C, R, U or D every other second or so. Database will have to be optimized for a lot of traffic. You will probably have to pass all queries through small number of server processes dedicated for this. Or you can use some messaging system like RabbitQM or similar although you would probably have to rewrite a lot of code.

    Other than that, it depends on what kind of jobs these programs are doing. Is it one process forked over 100 times, or a bunch of different process forked many times, or really 100 different processes? Do they respawn frequently on errors? (Like die() on every warn() and similar.) Is there a lot of network activity?

    I would concentrate on ensuring that the database access is always working properly when increasing the number of clients, that network issues are handled properly with timeouts and that very frequent respawns in case of errors are detected as soon as possible.

    This is pretty general, but it also depends on what kind of work your software is doing.

Re: Perl tk question
by sundialsvc4 (Abbot) on Jul 30, 2013 at 04:12 UTC

    Probably the most-important concern will be ... no matter if you have “20,000 jobs” to run ... that you are by some means able to effectively control how many simultaneous jobs run at one time.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1046889]
Approved by Old_Gray_Bear
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (2)
As of 2021-08-04 01:15 GMT
Find Nodes?
    Voting Booth?
    My primary motivation for participating at PerlMonks is: (Choices in context)

    Results (41 votes). Check out past polls.