Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask

organizing/ running non-module related tests

by geektron (Curate)
on Sep 20, 2005 at 16:19 UTC ( [id://493500] : perlquestion . print w/replies, xml ) Need Help??

geektron has asked for the wisdom of the Perl Monks concerning the following question:

so i have been doing some serious work on the testing aspect of my coding lately, and one of the tasks that's bitten me recently is an irritating database normalization/ porting project. the "legacy" database is rather flat in structure, has lots of incomplete/ missing information, etc.

my first migration of it created duplicated rows and other forms of fubar data because I didn't analyze it enough up front *and* because a mistake at, say, level 3 of the import trickles all the way down to level 10 of the import. (I'm trying to normalize/ port a few tables at a time rather than try to slam the whole thing over in one big chunk.)

to avoid the duplication in the next round, and to try and have more information on later problems, i'm creating a test suite (mostly some SQL that checks for duplicated rows, but there will surely be other tests later).

question is ... would it make more sense for me to learn ExtUtils::MakeMaker (or just write a custom Makefile) to add a level of automation to this process (so once it's done, and i have to migrate the code to a production database) i don't have to sit here and watch it go ... or does a Makefile.PL with make targets even make sense in this kind of scenario. I'm sure that i could just write a wrapper full of  system calls, but that doesn't seem like the Right Thing to Do ™ ....

Replies are listed 'Best First'.
Re: organizing/ running non-module related tests
by diotalevi (Canon) on Sep 20, 2005 at 19:14 UTC
    I believe you're looking for is Test::Harness which is what `make test' uses internally. I hear there are newer, nicer test harness modules out there which produce pretty HTML results. This one is just the old standard.
      that does look like what i had in mind ... but since i'm not a master of the testing game, i wasn't sure if a makefile or something else was the "right" direction. tnx.
        Nope. There's no reason to involve a makefile for the problem you just set out. ExtUtils::MakeMaker is irrelevant to you.
Re: organizing/ running non-module related tests
by graff (Chancellor) on Sep 20, 2005 at 18:34 UTC
    I think no. :) If (like me) you haven't used MM yet, I think it would make more sense to learn it in a context of actually putting together a perl module for distribution, since that is what MM was created to do, primarily, and its man page includes lots of details and external references about things that apply only to porting, building and installing perl modules.

    But you're talking about a database normalize/port task, which is rather different, and depends primarily on the database engine(s) at each end of the porting process, and on the (theoretical and actual) specs for the database schema you're dealing with.

    You could probably find useful things in the "Test::*" namespace on CPAN, but it's not clear to me that ExtUtils::MakeMaker itself, as a whole, gives you much of a direct and relevant head-start.

    update (forgot to mention): I don't understand why you would expect "a wrapper full of system calls" to be a sensible alternative. Wouldn't you be using perl/DBI to pull data out of the legacy db, maybe store it to appropriate (tab- or comma-delimited value) files, analyze the content of each table, and build a modified set of files for loading into the new db? (Of course, upon creating the properly normalized/ported data for each target table, you'd use your db's native import tool to load them.) Maybe some unix tools (cut/sort/uniq) could speed things up a bit for large quantities of data -- that's a judgment call that can't be made based on what you've told us.

      hrm ... maybe more detail would have helped.

      the tests are similar to what goes on in a module distro. i have a t/ directory that's filling up with DBI-based scripts.

      what i'm looking for is an alternative to running each test (or each step of the migration) by hand. the "wrapper" would basically be something like:

      system( 'my/dir/' ); system( 'my/dir/' ); system( 'my/dir/' );
      which looks an *aweful* lot like a  make target to me.

      the alternative is:  [ mybox ] --> ./ ad nauseam.

      and it's one DB to another .. so there's no real need for files ....

        If that's all you're doing, it's not such a big thing to roll on your own:
        my @tests = sort <my/dir/*.pl>; #update: need to sort the glob list my $return = my $testid = 0; while ( $return == 0 ) { warn "Running $tests[$testid]...\n"; $return = system( $tests[$testid++] ); }
        But learning to use MM is bound to be a good thing anyway, so why ask us whether you should do it?

        update: if your scripts are actually moving data, then of course you'll need to prefix that loop with whatever steps are needed to clear out the destination db before the loop starts -- unless the first script in your list takes care of that.

Re: organizing/ running non-module related tests
by ambrus (Abbot) on Sep 20, 2005 at 18:10 UTC

    I think yes. MM has support for scripts as well as modules. You can write a custom Makefile too if you don't want to find out how MM works.

      well, i'm no  make expert ... and since there are multiple ways of handling  make targets, EU::MakeMaker seems like a sensible solution ...