Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re^3: Keeping browser active while computing in Perl/CGI

by sundialsvc4 (Abbot)
on Jul 30, 2014 at 19:02 UTC ( #1095651=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Keeping browser active while computing in Perl/CGI
in thread Keeping browser active while computing in Perl/CGI

I intended that a single computation would be a single job ... not a series of multiple jobs.

What I meant to say was that, if the user had several of those computations to run (using different input data-sets), each one would be its own “job.”   It’s a self-contained description of what program to run and what data to give it.   There would be another process or set of processes, somewhere, on some number of computers (or the same one), which would be dedicated to running them and making the results available by some convenient means.   These are the “batch job runners,” or “initiators,” that actually do the work, perhaps on über-fast machines that do nothing else.   The web-page would thus become a user-interface to that overall process, allowing computations to be requested (with specified inputs supplied using the page), monitored, and results retrieved.   You don’t have to “wait for” anything ... just check-back in a few minutes to see if it’s done yet.

Yes, of course the idea could be extended to support parallelism, clustered computing and so-on, if the nature of the problems being solved were conducive to that sort of thing.   I’ve seen some dedicated workflow-tools, homebrewed by national labs and such, that know a lot about a particular type of work that they spend all day solving, and, having been built with public funds, they are now publicly available.   There’s actually a lot of “prior art” for doing things like this, as well as various commercial products.   It will be a “pick one vs. build” decision that probably will be, “pick one.”

I’ve had great successes with this sort of thing.   For one thing, a good batch-monitor can start another unit of work without incurring any idle-time between jobs, even if the jobs take a little more or a little less time.   Unlike cron, there will never be any collisions, nor any wasted time, nor any under- or over-utilized resources.   And, over the course of a few days or what-not, all that time really adds up.   One client easily pulled 30% more work out of his existing equipment, and was quite surprised at the difference.   It was just wasted time, no longer being wasted!   The power-bills went up somewhat, but they didn’t care.   Work that “might or might not” actually be finished by morning ... always was.   The ability to do automatic checkpoint/restart was just a bonus.


Comment on Re^3: Keeping browser active while computing in Perl/CGI
Re^4: Keeping browser active while computing in Perl/CGI
by Anonymous Monk on Jul 30, 2014 at 19:20 UTC

    Until we hear more from vit about the specs, just say to yourself, "gloves".
    http://thedailywtf.com/Articles/The_Complicator_0x27_s_Gloves.aspx

      Heated handlebars are very much a thing for temperate zone motorcycle couriers. They are always used in addition to gloves. It's a good analogy here though.

      My first thought was to queue the job in the background and tell the user to check back in 20 minutes. Or use a placeholder page that reloads every minute and eventually gets replaced by the results. A batch type scheduling thing might be worth it if there are a number of users.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1095651]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (7)
As of 2014-12-25 07:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (159 votes), past polls