bojinlund has asked for the wisdom of the Perl Monks concerning the following question:
The reason for this post is the message
in https://rt.cpan.org/Public/Bug/Display.html?id=67292#txn-1187167."This patch was applied at 1.13. (Note, but it's still failing.)"
There is a summary of the failing tests on Windows below. (There are very few fails on other operating systems.) Hopefully we can help the maintainer of the module with the remaining problems in Windows!
Background
"Bug #66016 for Test-TCP: Tests fail on Windows (even when they pass?)" https://rt.cpan.org/Public/Bug/Display.html?id=66016.
"Bug #66437 for Test-TCP: Tests are blocking in Windows 7" https://rt.cpan.org/Public/Bug/Display.html?id=66437.
"Bug #67292 for Test-TCP: Tests are blocking in Windows 7. With a prposed patch."
https://rt.cpan.org/Public/Bug/Display.html?id=67292.
Problem: The tests in Test::TCP are blocking. Sometimes they get the system in state, so it must be restarted.
The purpose of the (proposed and used) patch is to:
- Reduce the frequency of problems when using kill on a pseudo-proccess in Windows.
- To avoid to use kill on a pseudo-process in the test of Test-TCP.
Summary of the failing test of Test::TCP
The pattern used is:
Test files: List of failing SUBtests. The digit in parentheses indicates the number of times the subtest has failed for version 1.13 - 1.18 of Test::TCP on mswin32 (data from http://matrix.cpantesters.org/?dist=Test%3A%3ATCP).
In "mswin32" and Test::TCP version 1.13 there are 0 fail/29 pass tests. Version 1.14 => 1/8, 1.15 => 3/13, 1.16 => 6/10, 1.17 => 0/19 and 1.18 => 5/53. For version 1.13 - 1.18 there are 15 failing tests and 124 passing.
Test report pattern: Pattern in the test report.
1) Test returned 9 even if all subtests passed
Test files: t/03_return_when_sigterm.t, t/04_die.t, t/06_nest.t(2), t/09_fork.t
Test report pattern:
Dubious, test returned 9 (wstat 2304, 0x900) All x subtests passed
This problem is treated in https://rt.cpan.org/Public/Bug/Display.html?id=66016. The conclusion is that there is a probability that Windows returns 9 instead of the correct zero. One failing subtest in about 140 test runs is a little bit to high!?
2) Child process does not block
Test files: t/04_die.t(4), t/06_nest.t(4)
Test report pattern:
[Test::TCP] Child process does not block(PID: -xxxx, PPID: xxxx) at .. +. \lib/Test/TCP.pm line 121. t/xxxx.t ................. ok
TCP.pm (line 121 marked with comment):
sub start { my $self = shift; if ( my $pid = fork() ) { # parent. $self->{pid} = $pid; Test::TCP::wait_port($self->port); return; } elsif ($pid == 0) { # child process $self->{code}->($self->port); # should not reach here if (kill 0, $self->{_my_pid}) { # warn only parent process sti +ll exists warn("[Test::TCP] Child process does not block(PID: $$, PP +ID: $self->{_my_pid})"); # line 121 } exit 0; } else { die "fork failed: $!"; } }
3) Target machine actively refused
Test files: t/01_simple.t(2), t/10_oo.t(8)
Test report pattern:
[Test::TCP] Child process does not block(PID: -xxxx, PPID: xxxx) at .. +. /Test/TCP.pm line 121. Cannot open client socket: No connection could be made because the tar +get machine actively refused it. at t/10_oo.t line 21. # Looks like you planned 22 tests but ran 20. # Looks like your test exited with 10061 just after 20. t/xxxx.t ................... Dubious, test returned 77 (wstat 19712, 0x4d00) Failed 2/22 subtests
Test-TCP-1.16/t/10_oo.t line 21:
my $sock = IO::Socket::INET->new( PeerPort => $server->port, PeerAddr => '127.0.0.1', Proto => 'tcp' ) or die "Cannot open client socket: $!";
4) Cannot open client socket
Test files: t/09_fork.t(4)
Test report pattern:
[Test::TCP] Child process does not block(PID: -xxxx, PPID: xxxx) at .. +. lib/Test/TCP.pm line 121. # Failed test 'socket is connected' # at t/09_fork.t line 35. # Cannot open client socket: # Looks like you planned 6 tests but ran 5. # Looks like you failed 1 test of 5 run. t/09_fork.t ................. Dubious, test returned 1 (wstat 256, 0x100) Failed 2/6 subtests
Test-TCP-1.16/t/09_fork.t line 28:
# after the child has exited, we need to make sure that # the server hasn't gone away. my $sock = IO::Socket::INET->new( PeerPort => $port, PeerAddr => '127.0.0.1', Proto => 'tcp' ); if (! ok $sock, "socket is connected") { #line 35 return diag("Cannot open client socket: $!"); }
Questions
- Any ideas of the reasons for the failing tests?
- Is there a more portable way to implement the test of Test::TCP?
- How should the failing tests be changed?
- Need Test::TCP to be changed, to be more portable?
- What are the consequences of the "none-deterministic" behaviour of Windows?
- Is it enough to run the test of Perl modules just once or how many times are they needed to be run?
- Is this only a problem with the implementation of Perl 5?
- Or are the similar problems with Perl 6 and Python on Windows?
Regards
Bo Johansson