Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW

proving something other than perl

by Tanktalus (Canon)
on Jan 24, 2007 at 18:31 UTC ( #596305=perlquestion: print w/replies, xml ) Need Help??

Tanktalus has asked for the wisdom of the Perl Monks concerning the following question:

I've been looking at some changes to our shell scripts lately, changes that require a fair bit of work to set up the test environment. So I wrote a quick shell script that would blow away the old stuff, re-extract the original files, symlink into my build environment appropriately (the files I'm working on), just to get to a pristine state easily.

And then I have a bunch of tests to run, and check. So there was some command line trickery to get stuff repeating the way I wanted. And now I have to move to a new platform to test it all again. The setup script should be close to what I need, but, of course the stuff I typed at the command line needs to be rewritten on the new machine since I can't just go back into the history there. Maybe some cut&paste, but as the commands are 200+ characters long, it's a pain.

One of the tests was running the same command on each file, looking for stuff. It started out as:

ls I/{some,dirs}/ | xargs -n 1 -t I/foo/bar -q 2>&1 | sed -e 's/stuffT +oRemove//' -e '/linesToIgnore/d' | less

This wasn't enough - too much to wade through. So it eventually became:

find I -type f | perl -nle 'open my $fh, "I/foo/bar -q $_ |" or next; +chomp(my $l = <$fh>); $l =~ s/stuffToRemove//; my $count =()= $l =~ / +stuffToCount/g; print "$_:$count[$l]" if $count > 2'

Eventually. Figuring there had to be an easier way to get this done, I started lamenting to myself about being able to do a "make test" like I do with my CPAN'd modules. A bit of digging showed that this just runs "prove" - which looks like I could just write a few .t scripts and test stuff that doesn't even have anything to do with perl. Some File::Find for the above line, maybe a subroutine wrapper around qx so as to clean up the output for proper "is" testing...

So, my question becomes not whether anyone has gone down this road before, as I'm sure some have (I might be smart (or at least, I think so ;-}), but not that smart), but what gotchas, if any, did that experience teach you? Things to keep in mind as I go down this road. I much prefer learning from other people's mistakes before being forced to learn from my own :-)

Right now, I'm doing this on Linux, AIX, probably Sun and HP. I may want to do this on Windows, eventually, too, since shell scripting is so much harder there ;-)

Update: Zaxo kinda hits on the reason behind the problem: I would like to not only prove that this works (actually, just some sort of unit test environment) so that I can try to push our team towards some sort of standardisation on unit testing. And maybe porting TAP clients to C to test our libraries, etc., would be a good idea ... but this is just kind of like the start of a push to show its usefulness in at least some aspects of what we do. Preferably without reinventing the wheel. Perl strikes me as the way to move things, but I've gotta prove prove to my team, and it's faster to do that based on others' experience than try to discover the gotchas (and cool, neat features) myself ;-)

Replies are listed 'Best First'.
Re: proving something other than perl
by Zaxo (Archbishop) on Jan 24, 2007 at 18:42 UTC

    This doesn't answer your question, but why not use perl for whatever the shell scripts do? Portability seems to be your main problem, and perl's made for that.

    Your shell test line would be much cleaner and easier in perl. As you suggest, File::Find would be a big help, or glob might suffice.

    After Compline,

      I'm a perl addict and I try to do everything I can (within reason) to use my one hammer (perl) on every nail. That said, I think that a nice /bin/sh script is probably more portable than perl — at least in unix-like environments anyway.


        Some things are just easier to do with a bourne shell. This appears to be an either/or situation where it doesn't matter if you use perl or bourne. ;-)

        Regarding the test environment... If your test boxes don't change very much, you could just source an environment file with the appropriate #ifdefs to point to the correct binaries, directories, libs, etc. Once the environment variables are set up, you should be able to create any symbolic links you need.

        Unless you're testing the extraction/build part of your application(s), I would recommend to extract a development or testing release from a version control system (e.g. subversion, clearcase) and perform the tests. That way, you can either correct any issues you find on the test box and check it back into the vcs or you can fix it on your development box, submit the fix to your vcs and extract the new code to your test box. Rinse, lather, repeat ;-)

        You could automate the vcs extraction and testing rather easily (GUI testing is difficult to automate) by putting the process in cron or equivalent scheduler. I know of several big software firms that perform a variation of this.

        Jason L. Froebe

        Help find a cure for breast cancer! Net proceeds benefit the Susan G. Komen Breast Cancer Foundation and the National Philanthropic Trust. Help by donating - I'm walking 60 miles in 3 days in August 2007. (The day I return from TechWave is the first day of the Walk).

        Blog, Tech Blog

Re: proving something other than perl
by clscott (Friar) on Jan 24, 2007 at 20:51 UTC
Re: proving something other than perl
by Solo (Deacon) on Jan 24, 2007 at 19:05 UTC
    And now I have to move to a new platform to test it all again. The setup script should be close to what I need, but, of course the stuff I typed at the command line needs to be rewritten on the new machine...

    Are you asking about building your build scripts? I don't see why build scripts can't be treated as another package. The Unix way to handle cross-platform builds is configure (the docs even say there is Win32 support). The Perlish way is Module::Build or MakeMaker.

    If you don't want to carry Perl around, it sounds like you may just want a variant of make clean that only impacts the 'build scripts', whatever they are. Makefiles are not Perl specific, all the platforms you mention should have a version of make (nmake for windows).

    Or, maybe I've missed your point entirely.


    You said you wanted to be around when I made a mistake; well, this could be it, sweetheart.

      The top part is the part where I explain how I got here. Unfortunately, that may be overemphasising that part, as my real question is: anyone who has started to use the "prove" tool (or, really, any of the Test::* namespace) to test stuff that isn't perl, do you have insight on how to approach this? Is there a particular Test::* module that is well-suited? Are there gotchas to using the Test::* modules in such a way? Obviously, things like "require_ok" or "isa_ok" or "can_ok" are useless. I'm more looking for "don't rely on XYZ to work against qx//" or "Check out Foo::Bar - its Quux function can really help with ..." or things like that.

        Strictly speaking you can "prove" using some wrapped up TAP output! for example a file can be sent (system perl, curl, sftp etc..) to any prove-abled server (if you don't want to speak apache, Net::Server or HTTP::Server::Lite with a trivial CGI comes to mind).

        Now about the OP question: build systematically wrappers for processes-to-be (created from C, C++, java -- for java you often need a wrapper anyway for classpath and libs) and you use the OS (i.e the shell exit status). It is not really tough to generate TAP output, is it? (tee is useful). For a given process, I keep the output of one "correct" run in one file, and compare subsequent ones with that reference file. Some of the basic infrastructure can be automated and some templates are useful (output files can be generated by perl in another machine if you like, but with a decent shell, shell eval is your friend!)

        Welcome to the world of "external testing" it is much harder than the other one so that's why you don't see much of it ;)
        a good shell is ksh that you can get from, grab ksh93s for all the platforms that lack a decent shell; for cygwin (windows!) I recommend using still ksh93r (as there are problems that I will soon report to the ast-users list) and by the way, to use ksh on cygwin you don't need the whole cygwin stuff just the cygwin1.dll and maybe a few commands like curl and tar). So you make your little shell tarball where you don't have a decent shell. Also the perl that comes with the system can obviously be useful and can COOPERATE HEALTHILY with the shell and other commands, no? ;)

        % stephan@armen (/home/stephan) % % cygcheck /usr/ast-ksh93r/arch/cygwin.i386/bin/ksh93r.exe C:/cygwin/usr/ast-ksh93r/arch/cygwin.i386/bin/ksh93r.exe C:\cygwin\bin\cygwin1.dll C:\WINDOWS\system32\ADVAPI32.DLL C:\WINDOWS\system32\ntdll.dll C:\WINDOWS\system32\KERNEL32.dll C:\WINDOWS\system32\RPCRT4.dll C:/cygwin/usr/ast-ksh93r/arch/cygwin.i386/bin\cygshell11.dll C:/cygwin/usr/ast-ksh93r/arch/cygwin.i386/bin\cygast54.dll C:/cygwin/usr/ast-ksh93r/arch/cygwin.i386/bin\cygcmd12.dll C:/cygwin/usr/ast-ksh93r/arch/cygwin.i386/bin\cygdll10.dll
        hth --stephan
        the un*x way? cooperation!
        test stuff that isn't perl

        Apache::Test is the biggest example that comes to mind. I've also used Test::Output, but not in a while.


        You said you wanted to be around when I made a mistake; well, this could be it, sweetheart.
        The reason I wrote prove was so that I could test PHP scripts along with the rest of my Perl programs. The key is to create a Test::Harness::Strap module that does the right thing, and to load that strap with the --strap argument on the command-line.

        I know it's hardly a complete answer, but I hope it gets you pointed in the right direction.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://596305]
Approved by Old_Gray_Bear
Front-paged by Old_Gray_Bear
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (3)
As of 2021-09-21 21:19 GMT
Find Nodes?
    Voting Booth?

    No recent polls found