Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

handling tasks and customizing flow of tasks using perl

by asham (Novice)
on Sep 29, 2013 at 12:07 UTC ( #1056210=perlquestion: print w/replies, xml ) Need Help??
asham has asked for the wisdom of the Perl Monks concerning the following question:

Hi Perl Monks, I have some existing code which carries out some special tasks. These tasks have different set of combinations available and on top of that API user (another coder/normal user) may want to customize it let say by doing some actions between consecutive tasks. I would need your suggestions to write good code to do the job. Below is an example of issue above and couple of solutions in my mind.
Issue in detail: Let say we have 4 major tasks: A, B, C, D such that following combinat +ions are considered valid: 1) A->B->C->D 2) A->C->D 3) A->B->C 4) A->D and so on. On top of that, we would like to allow users to customize the flow of +tasks as following: 1) A->myOperations->D 2) A->preTaskDoperations->D->postTaskDoperations. Currently, I have support for user flow without this customization lik +e 1) A->B->C->D 2) A->C->D 3) A->B->C 4) A->D and so on. Note: same was mentioned above earlier.
To support user customization for flows of tasks, I am wondering following: Approach one:
<!--Make an xml file something like below--> <xmlFile> <action name="A" execute="true"> </action> <action name="B" execute="true"> <arguments> <arg name="endToEnd" value="true"/> </arguments> <preTasks> <!--thinking of making this as text--> $ENV{skipActions}='true'; makeAServerdown(); <!--this can be later executed with exec function--> </preTasks> <postTasks> checkBServer(); </postTasks> </action> </xmlFile>
Approach two:
Object Oriented approach. 1) make code file which has such customized details. (derived class) 2) provide override mechanism to end user. 3) support flow of tasks like mentioned in xml but remove perl code fr +om there. There are several alternatives to do this which maybe as si +mple as reading properties file.
So after having read above, could you please suggest any improvements/alternate mechanism to handle the use case? Also, are there any existing apis which support mix of xml/perl? Thanks in advance.

Ps: Based on queries from perl monks, sharing real life example below:

let say you are in automobile factory: buildFrame->addSeats->addMechanicalParts->addDoors->paintCar->touchUp

Today let say we have following flow: 1) buildFrame->addSeats->addMechanicalParts->addDoors->paintCar->touchUp 2) buildFrame->addSeats->addMechanicalParts->paintCar->touchUp

Tomorrow, I want to expand my factory to handle following: (can be on request basis also):

new cases: 1) buildFrame->addSeats->postAddSeatsTask->addMechanicalParts->paintCar->touchUp this postAddSeatsTask maybe like replacing existing seats with leather seats. Note: only few people want it and changing whole assembly line for few people is expensive. few customers may want this and some may want something different like more leg space shorter seats. So customization is different for both and can not be predicted in advance that we provide them an API. some may want to fit couch from some company. so we may provide plug and play feature too.

I hope this gives a sense that customizations are many and with plug and play more or less things should be fine.

Above was for customization but yes tomorrow flow can change, company decides we wont paint car during production phase. but only before delivering to customer. So they remove that step and add it later changing flow of steps as below: buildFrame->addSeats->postAddSeatsTask->addMechanicalParts->touchUp->paintCar.

I hope this covers that flow needs to be flexible and steps may be added or removed as product changes.

Replies are listed 'Best First'.
Re: handling tasks and customizing flow of tasks using perl
by vsespb (Chaplain) on Sep 29, 2013 at 13:08 UTC
    Instead of XML, you can introduce simple text format:
    task=B pretask=makeAServerdown posttask=checkBServer
    Then you can parse it and call pretask/posttask as perl method:
    use strict; use warnings; use Data::Dumper; sub makeAServerdown { print "AHA:", @_, "\n"; } my $line = 'task=B pretask=makeAServerdown posttask=checkBServer'; my $h = { map { my ($k,$v) = split /=/; $k=>$v } split (' ', $line) }; print Dumper $h; { no strict 'refs'; $h->{pretask}->($h->{task}); } __END__ $VAR1 = { 'pretask' => 'makeAServerdown', 'task' => 'B', 'posttask' => 'checkBServer' }; AHA:B
      Hi vsespb, Thanks a lot for sharing your suggestions. I am not sure, if I understood the way you are integrating tasks and customizations but it seems following:
      1) $h (hash) so used in your file(should be standard file?). 2) the module should be then included in task side code where the $h o +f this module will be read.
      Advantage seems to be that xml parser overhead is not required. but then again xml/text have their own pros and cons.
      Some concerns: 1) are you suggesting reading this : "task=B pretask=makeAServerdown p +osttask=checkBServer" from some text file. or hardcoding in code? I a +m assuming text file for now. 2) The customizations suggested by you will work if only known action +is being called. But my use case is that user can customize with anyt +hing they like i.e. they can maybe add any code block they want etc. example of code block: (simple example here) for (int i=0;i<10;i++) { print('just a block for fun: ' . $i . "\n"); }
      Kindly look into this and please correct me as required.
        1) are you suggesting reading this : "task=B pretask=makeAServerdown posttask=checkBServer" from some text file.
        2) The customizations suggested by you will work if only known action is being called.

        If you need arbitrary code execution, write an API for programmers.

        I don't see any reason mixing API for programmers and for normal users, nor I don't see use of XML/other formats in API for programmers.

        Or, parhaps, I misunderstand the OP.
Re: handling tasks and customizing flow of tasks using perl
by Laurent_R (Canon) on Sep 29, 2013 at 17:24 UTC

    I certainly would not use XML for that. XML is useful for things like communications between different systems or storing some configuration into a file on disk, it is really unpractical as an internal data structure.

    You probably want to build a graph, which could be possibly a hash of arrays, in which the array contains all possible successor nodes to the hash key node. Or you could build a list of possible complete paths from a given node.

    Your description:

    1) A->B->C->D 2) A->C->D 3) A->B->C 4) A->D

    is not completely sufficient to figure out the paths that are allowed. Suppose that I am on node B: does that mean that we can go to node C irrespective of what happened before (in which case paths 1 and 3 seem to be redundant), or that I am allowed to go to C only if I was on A before getting to B (so that I can't go anywhere if I am on node B but not coming from A)? Determining the best modeling for your application behavior depends on an answer to such questions.

      Hi Laurent, Thanks a lot for looking into this.

      Yes true xml are not used for such code mix but it would be kind of configuration file only right? as it holds details related to tasks. exampple: should a task be executed or not and if yes than with what configuration parameters. The only extra thing I was thinking of adding was the pre/post tasks which may vary for some users. (also idea of this post is to come up with some good design to handle the use case.)

      Regarding the building graph/hash suggested by you, I had considered that but I was not convinced with the idea because doing that means I am kind of fixing the flow of tasks. Any changes in future require update of code rather than just a tiny update in xml/properties file which can be even exposed to user. (i generally prefer xml/properties file compared to such code updates because of more flexibility and ease of use. As end user may not know the code and can just update the xml file to change/control the flow.)

      Just to let you know that mixing xml/perl is not the best design and that's why I am looking for inputs here. I wanted to check if people were using it or apis are for this. (note: jsp/jsf mix html and java. so thought something on those lines.)

      Regarding paths: 1) A->B->C->D 2) A->C->D 3) A->B->C 4) A->D

      These are just for example, you can check the xml file shared in my first post. note its rough design and any such redundancies will be handled when we actually finalize an approach.

      regarding the details requested by you. I am updating original question with a real life example. Please check, hope it helps in understanding the case more.


        there are many aspects to be taken into account.

        I was writing last week a program template aimed at some other IT persons. The idea was to give them the (quite complicated and thoroughly tested) algorithm, and let them construct the input data, which could be summarized as follows:

        ('table1' => { 'index_fields' => [0, 1, 2], 'ignored_fields' => [6]}, 'table2' => { 'index_fields' => [2], 'ignored_fields' => [4, 7]}, )

        It is not too complicated to give IT persons instructions on how to fill the next records in this structure to achieve the desired result.

        But if your user has no IT background, this might not be workable.

        I still don't think that asking them to fill an XML structure is the right idea. Asking your users to write a config file such as:

        1. A -> B -> D

        is clearer for your user (so long as you explained the details) and actually easier for you to parse.

Re: handling tasks and customizing flow of tasks using perl
by wjw (Priest) on Oct 04, 2013 at 01:53 UTC
    Coming from a manufacturing background, I find your problem interesting. This seems to be much like an MES or ERP module for factory routing. There are two basic lists I would provide to achieve the functionality I understand you describing:

    List of rout elements:
    This would look like:
    1(unique int)buildFrame
    2(unique int)addSeats
    3.. ...

    List of existing routs:
    would look like:

    What your storage solution is does not really matter. Could be text files, XML, RDBMS, JSON, .ini....whatever...

    The key is to be able to easily add "workstations", and easily add routs(ordered collections of workstations)

    The workstation list is just a lookup to fill in the rout. Both the existing routs and workstation list would only be added to. (thus you never loose the history of a rout).

    Really all you need is an array to hole the workstations and a hash of arrays to hold the routs. With those two structures, you can do pretty much anything you need to do for a simplistic system. Both lists could be stored in a single file if that makes it easier, or in a spreadsheet assuming you use something like Spreadsheet::WriteExcel.

    I would use an RDBMS for my storage personally, but again, it hardly matters. In my experience, the priorities are in order:

    1. flexibility(mfg always changes)
    2. Maintainability (KISS)
    3. Resources(skillsets)
    4. historization(traceability)

    Hope there is something useful to you in there...

    • ...the majority is always wrong, and always the last to know about it...
    • my will, and by will alone.. I set my mind in motion
      Hi wjw,

      Thanks a lot for your suggestions. They are very helpful. I would try to pick some inputs from your reply and incorporate in our use case.

        Glad to know you found something useful there asham.

        I enjoy thinking about and working with process flow and find questions like yours inspiring all sorts of other questions, which is what makes them challenging and interesting, particularly when Perl is involved.

        Do feel free to use what ever was useful to you in your project and best of luck with it!

        ...the majority is always wrong, and always the last to know about it...
        Insanity: Doing the same thing over and over again and expecting different results.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1056210]
Approved by vsespb
NodeReaper cracks his knuckles - loudly

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2018-06-22 00:28 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (120 votes). Check out past polls.