Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

make-like dependency tools in perl

by bsb (Priest)
on Jun 24, 2005 at 07:24 UTC ( #469625=perlquestion: print w/replies, xml ) Need Help??
bsb has asked for the wisdom of the Perl Monks concerning the following question:

Dependency driven execution is a problem that keeps on cropping up. My lastest encounter is in progating changes through a data warehouse. Before that is was running one section of a test script but ensuring the dependencies are fulfilled (eg. login first). Build systems and component publishing systems deal with dependencies too.

What I need is something like "make" which has a graph of dependencies, rules for updating them and a means to check the age/change state of items. You can then "make endproduct" and it will update the parts that need to be updated in the right order. The problem with make is that it's tied to files (is there another way to use it?)

I've looked around for either a module or software with the right hooks, but can't find anything. Cons and Makepp are written in Perl but, as with make, the seem too file-system tied.

AAP by Bram "vim" Moolenaar looks more flexible but uses Python as it's scripting language. (I recommend the linked "design decisions" article as an information source for the problem domain).

On CPAN there's Algorithm::Dependency, but it's doesn't address this problem, leaving Graph as the best module to try an adapt.

Is there anything better out there that I've missed?
Or another way of thinking about the problem?


Replies are listed 'Best First'.
Re: make-like dependency tools in perl
by cbrandtbuffalo (Deacon) on Jun 24, 2005 at 11:53 UTC
    You may be able to get some ideas from Module::Dependency. There are three parts; I think the important bits for you would be in the Indexer. I haven't delved into the code, but I've used it and it does a good job.
Re: make-like dependency tools in perl
by adrianh (Chancellor) on Jun 24, 2005 at 13:26 UTC
    Is there anything better out there that I've missed?

    Express your dependency graph as a state machine and use something like FSA::Rules ?

Re: make-like dependency tools in perl
by toma (Vicar) on Jun 25, 2005 at 04:59 UTC
    I've used Algorithm::Dependency::Ordered for this sort of thing. If you can show a specific example of what it doesn't do, perhaps someone will suggest a solution.

    Rebuild the dependency graph when something changes instead of using rules for updating the graph. Incremental graph-update functions are certainly possible to write, but many graph modules don't support it.

    "Age/change state" can be checked with ordinary date modules or functions. Do you need something else?

    How many dependencies do you have? If you have more than about 50,000 you may have a difficult job. If you have more than about 3,000,000 it can be very tough.

    It should work perfectly the first time! - toma
      From my cursory inspection, Algorithm::Dependency didn't seem to buy me much over Graph. I'd still be handling the execution and the graph building (or using the format of A::D::Source::File). Maybe I should have a second look, since you think it may be useful.

      "Age/change state" can be checked with ordinary date modules or functions. Do you need something else?

      The only unusual requirement here is that I should be able to source the date from a database instead of simply from a file's age. Another option in build systems is comparing md5 hashes for changes, but I haven't needed that myself.

      My dependency graphs have all been quite small an simple. Fewer than 20 nodes, just managing task order and execution.

Re: make-like dependency tools in perl
by schwern (Scribe) on Jun 25, 2005 at 01:16 UTC
      Maybe if sufficiently encapsulated.
Re: make-like dependency tools in perl
by nothingmuch (Priest) on Jun 25, 2005 at 21:43 UTC
    For general dependency solving, stvn and I wrote Algorithm::Dependency::Objects because it's more convenient, at least in our opinion, than Algorithm::Dependency.

    For dependency driven action composition, somewhat like Java's and in a perlish set of metaphors, we also wrote Verby.

    Have fun!

    zz zZ Z Z #!perl

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://469625]
Approved by greenFox
Front-paged by tlm
[erix]: comic relief: how to start a riot and a fortune at the same time
[beech]: :) thats too generous to jdporter, don't start none won't be none -- there is about two decades (rounding up) worth of "ineffective" node titles -- leave the pope alone :p
[erix]: infallibility breeds inquisition

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2017-01-17 23:52 GMT
Find Nodes?
    Voting Booth?
    Do you watch meteor showers?

    Results (159 votes). Check out past polls.