System testing a WWW transaction application

[dragonchild]

At my new position, we power these WWW treansaction systems. It's a huge state machine, essentially. It's all written in Perl/Apache. (Don't say anything about CGI::Application or nothing. It's a 4yr old application.)

Now, we need to be able to do systematic system testing, preferably black-box testing. Are there any ways people have tested full WWW applications without having to build hooks into the code or manually following every path?

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

[dws]

It's a 4yr old application ... Now, we need to be able to do systematic system testing ... without having to build hooks into the code or manually following every path

Uh oh. Four years into an applications lifecycle is a bit late to discover system testing. Are there unit tests for subsystems, or are you trying to avoid having to restructure the code to make unit tests work?

My preference is to start with unit testing each subsystem, using mock objects as necessary to isolate subsystems from each other. If you wait until everything is assembled until trying to test, the tests are harder to write, since they have to set up and tear down a known environment (e.g., resetting the database configuration to a fixed starting point).

chromatic has a good article on using mock objects to test. See A Test::MockObject Illustrated Example.

Can you say more about the reluctance to build "hooks" into the existing code?

[dragonchild]

I've been here for less than 3 months and have discovered that I have absolutely no idea as to how to tell the QA people how to test my stuff. There are no standardized tests at all.

The system is poorly designed and cannot be changed quickly. Hence, we can only work with external tests. Plus, I'd prefer external testing, anyways.

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

[IlyaM]

There are some difference between FunctionalTests (aka AcceptanceTests) and UnitTests. Idea of unit tests is that they are written by programmer and they test isolated pieces of program while functional tests are implemented by other party (for example QA stuff) and they test whole application. It is important difference!

The purpose of functional and unit tests is different and they are implemented by different people. Obviously they need different tools. Test::MockObject is useful in unit testing but it is wrong tool for functional testing (and this is what about dragonchild is asking).

--
Ilya Martynov (http://martynov.org/)

[dws]

Functional Tests are a different beast than Unit Tests, though if you don't have effective Unit Testing, jumping in to Functional (or Acceptance) Testing is like catching rain in buckets when you should have patched the holes in the roof first.

[FoxtrotUniform]

I'd expect you to find cURL useful, at least for the "poking at the interface" bit.

Why are you so interested in black-box testing? In my experience, the programmers have a fair idea of which bits of the system are robust and which could use some work, so building tests to fit the code is quite profitable.

--
The hell with paco, vote for Erudil!
:wq

[dragonchild]

Black-box testing is a necessity for verifying that applications, as a whole, work. The client uses your application as a black-box. Hence, you should, at least, once, try your application as a black-box prior to letting your client have it.

cURL looks interesting. Do you have anywhere that shows how it's used? The site didn't seem to have stuff about black-box testing WWW interfaces from the way it's displayed on the screen.

------
We are the carpenters and bricklayers of the Information Age.

Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

[ignatz]

The most kick ass testing of a web application that I have ever seen was with a commercial tool from seque solutions called SilkTest. In seconds that sucka would dig up little bugs out of the most hidden of nooks and crannies that there was no way in hell that I would have found on my own. Worth every penny in my book.

()-()
 \"/
  `                                                     

[IlyaM]

You can use Apache::Recorder with HTTP::RecordedSession to record and replay later tests. It doesn't require too much programming.

You can also try to teach QA stuff to use HTTP::WebTest. It provides its own quite simple mini-language for test specification understandable by non-programmers.

--
Ilya Martynov (http://martynov.org/)