I just finished my first version of a semi-automatic test generator for web applications. I've called it "RoboWeb" and it's now available for download from sourceforge, in hopes of it being useful to others. Reviews are gladly accepted.

Here's an excerpt from the README:

RoboWeb is a suite of Perl scripts that allow for recording live user browsing sequences and insert assertions during them to later reproduce and automatically verify correctness of output of a web application. Recording test sequences is easy, just fire the roboweb_proxy script and make your browser point to it. The proxy will generate a test sequence script as you browse and you can insert "must_match" commands in the recorded sequence by entering regular expressions directly in your browser's URL box. These commands get translated into assertions that will be run against the server response text when the sequences are reproduced, to verify that server responses are correct. RoboWeb will also automatically put your web application backend database in a constant 'test-scenario' state before recording test sequences and before executing test scripts. This allows for reliable testing of complex web applications. After you collect a number of test sequences you can execute them via the TEST script, which will produce standard perl Test::Harness output.
If you find this interesting you can download it from its page at Sourceforge.



Replies are listed 'Best First'.
Re: Web Application Test Suite Generator
by princepawn (Parson) on Jul 15, 2001 at 19:44 UTC
    I would be interested in a feature-by-feature comparison of your framework with CGI::Test.
      The intent of Test::CGI is almost the same. It's about crafting test sequences that will be run against a web application in an automated fashion. That said, I think RoboWeb wins hands down, author modesty not withstanding:

      1. RoboWeb makes it much easier to generate a test suite, because test sequence scripts under it are almost completely machine generated, except assertions, which are easily inserted by entering regexps in the URL box as you browse through your app. Test::CGI instead requires that all HTTP requests be hand coded by the test writer.

      2. RoboWeb runs against a live web server, versus Test::CGI, which runs web apps as standalone CGI processes. Test::CGI is less complete in this regard, because it doesn't allow to test apps in the environment they'll be running on (e.g. mod_perl apps that rely on persitent variables are not testable with Test::CGI).

      3. The RoboWeb client (which uses LWP::UserAgent and an HTTP::Cookie jar) supports cookies, which allow it to test most web applications. Test::CGI does not support cookies.

      4. Test::CGI cannot request plain documents, whereas Roboweb can.

      5. Both RoboWeb and Test::CGI produce Test::Harness output.



Re: Web Application Test Suite Generator
by TStanley (Canon) on Jul 15, 2001 at 16:53 UTC
    As a general rule of thumb, the source code in question is generally listed here. Just FYI for next time.

    There's an infinite number of monkeys outside who want to talk to us
    about this script for Hamlet they've worked out
    -- Douglas Adams/Hitchhiker's Guide to the Galaxy
      I thought about doing it but it's just too large to be practical. Maybe a bit of the more interesting code can do instead. Here's the code for the web proxy that generates the perl scripts that reproduce user behavior:

      (Note that the generated script uses a module called NetscapeLikeUA; this is a subclass of LWP::UserAgent that will follow redirects after POST requests, unlike the standard dictates but as Netscape does --see the sourcforge tarball for it)

      #!/usr/bin/perl -w use strict; use HTTP::Daemon; use LWP::UserAgent; use Data::Dumper; use URI::Escape qw(uri_unescape uri_escape); $Data::Dumper::Varname = 'req'; $Data::Dumper::Purity = 1; use RoboWeb::Util qw(pristine_please); my $listen_port = 8080; my $logfile = '/tmp/proxy_log'; sub _gen_page_fetch { my $request = shift; my $path = $request->uri->path(); my $url = $request->uri->as_string(); # we don't test for images being there. return unless ($path !~ /(gif|jpg|jpeg)$/); # don't save the cookies $request->header('Cookie' => ''); my $req_code = Dumper $request; $req_code = uri_escape($req_code); open LOG, ">>$logfile" or die "could not open $logfile"; print LOG "# print STDERR \"fetching $url\\n\\n\";\n\n"; print LOG "\$req_code = uri_unescape(q{$req_code\});\n"; print LOG "eval(\$req_code);\n"; print LOG "die \"badly frozen request \$!\" if \$\@;\n"; print LOG "\$cookie_jar->add_cookie_header(\$req1);\n"; print LOG "\$resp = \$ua->request(\$req1);\n"; print LOG "die \"could not fetch URL $url\" unless(\$resp->is_succ +ess());\n"; print LOG "\$cookie_jar->extract_cookies(\$resp);\n"; print LOG "\$html = \$resp->content();\n"; print LOG "#print STDERR \$html;\n"; print LOG "#print STDERR \"\\n\\n\", '*' x 70, \"\\n\\n\";"; print LOG "\n\n\n\n"; close LOG; } sub _start_rec { &pristine_please; open LOG, ">$logfile" or die "could not open $logfile"; print LOG "#!/usr/bin/perl -w\n\n"; print LOG "use RoboWeb::NetscapeLikeUA;\n"; print LOG "use HTTP::Cookies;\n"; print LOG "use URI::http;\n"; print LOG "use URI::Escape qw(uri_unescape);\n\n"; print LOG "print \"1..__REPLACE_NUM_TESTS__\\n\";\n\n"; print LOG "my (\$req1, \$resp, \$html);\n"; print LOG "my \$cookie_jar = HTTP::Cookies->new;\n"; print LOG "my \$ua = RoboWeb::NetscapeLikeUA->new;\n\n\n"; close LOG; } sub _end_rec { my $filename = shift; $filename =~ s/\W//g; $filename = $filename || 'HERE_I_AM'; $filename .= '.t'; open LOG, "$logfile" or die "could not open $logfile"; open OUTF, ">$filename" or die "could not open /tmp/$filename"; my $text; { local $/ = undef; # slurp mode $text = <LOG>; close LOG; } my $i = 1; $i++ while $text =~ s/__REPLACE_ME__/$i/; $i--; $text =~ s/__REPLACE_NUM_TESTS__/$i/; print OUTF $text; close OUTF; print "Created file: $filename\n"; } sub _must_match { my $path = shift; open LOG, ">>$logfile" or die "could not open $logfile"; print LOG "print 'not ' unless \$html =~ $path;\n"; print LOG "print \"ok __REPLACE_ME__\\n\";\n\n\n"; close LOG; } sub _must_not_match { my $path = shift; open LOG, ">>$logfile" or die "could not open $logfile"; print LOG "print 'not ' unless \$html !~ $path;\n"; print LOG "print \"ok __REPLACE_ME__\\n\";\n\n\n"; close LOG; } my ($req, $resp, $html); my $ua = LWP::UserAgent->new; my $d = new HTTP::Daemon LocalPort => $listen_port; print "I'll be your proxy today. Please contact me at: <URL:", $d->url +, ">\n"; while (my $c = $d->accept) { while (my $r = $c->get_request) { my $serv = $r->uri->host; my $path = $r->uri->path; my $port = $r->uri->port; print STDERR "Getting $serv - $path - $port\n"; if($serv =~ /start_rec$/i) { _start_rec(); } elsif($serv =~ /end_rec$/i) { _end_rec($path); } elsif($serv =~ /must_match$/i) { _must_match($path); } elsif($serv =~ /must_not_match$/i) { _must_not_match($path); } elsif($serv =~ /pristine$/i) { &pristine_please; } else { # proxy code # XXX change port of the request $resp = $ua->request($r); _gen_page_fetch($r); $c->send_response($resp); } } $c->close; undef($c); }


Re: Web Application Test Suite Generator
by princepawn (Parson) on Jul 16, 2001 at 08:20 UTC
    And the other Perl-based test suite, which is only on Sourceforge is: WebMonkey

    I am listing this for completeness, so we know about all viable options.

      Any idea what happened to Webmonkey? The sourceforge project page doesn't have any info that I can find.

      "A Jedi uses the Force for knowledge and defense, never for attack."