Re^3: Socket hang. (Windows or Perl? Solutions?) (Updated)

[Illuminatus]

I hesitate to provide a windows reference, given my lack of experience in TCP programming in that environment. That said, however, this page may have something worth trying. According to this, you actually don't want to set SO_DONTLINGER, but rather SO_LINGER, and set the l_linger field to 0. This will force a close via an RST rather than wait for full FIN processing. Maybe you have already tried this. This *could* also result in data not being received, based on timing, but I would not expect that to be the case on a local system.

fnord

[BrowserUk]

I had seen SO_LINGER, but chose to try SO_DONTLINGER as the arguments are simpler. I know from previous dealings with the setting no-blocking mode on windows sockets, that there is no documentation on exactly how to supply the final argument to setsockopt, and that it is tricky. As that parameter for SO_LINGER was true, I reasoned that would be simpler that working out how to pass a struct containing two packed shorts in the right order. But I hadn't seen that documentation for SO_LINGER that suggests that setting of 1,0 might be the fastest option.

Anyway, I worked out how to do it:

my $DONT = pack 'ss', 1, 0;
setsockopt( $svr, SOL_SOCKET, SO_LINGER, $DONT ) or warn $^E;

## verify setting
my $QUERY = getsockopt( $svr, SOL_SOCKET, SO_LINGER ) or warn $^E;
printf "\n\n%u %u\n\n", unpack 's2', $QUERY;

I'm not sure which way around the 1 & 0 should be, so I tried it both ways, and the upshot is, it made no perceivable difference :(.

Then much reading led me to revisiting Corions earlier post, I look again at MaxUserPort and realised I'd typed MaxUserPorts. Correcting that, and setting a value of 65535, I have now made over a million connections without stalling:

C:\test>junk79 12345
server:154323 client:154324 cycles: 385.846/secUse of uninitialized va
+lue $in in concatenation (.) or s
server:682262 client:682261 cycles: 358.073/secUse of uninitialized va
+lue $in in concatenation (.) or s
server:720436 client:720434 cycles: 340.687/secUse of uninitialized va
+lue $in in concatenation (.) or s
server:1121215 client:1121216 cycles: 350.004/secTerminating on signal
+ SIGINT(2)

I then found this: Normally, TCP does not release closed connections until the value of this entry expires. However, TCP can release connections before this value expires if it is running out of TCP control blocks (TCBs). The number of TCBs the system creates is specified by the value of the MaxFreeTcbs entry.

Which basically suggests that the only thing that was causing the stall was the artificial limitation on concurrent user ports, as without it, they get reused if needed regardless of the linger time. The generous view of that is that MS are conserving memory used by systems data structures for the benefit of workstation users who will rarely use programs that will need 65k concurrent connections. The less generous view is that they deliberately ham-string tcp/ip on workstations to persuade people to by the more expensive server variants of Windows. And the truth is probably a little of both.

The most striking thing to come out of this for me, is how the settings and defaults for TCP/IP seem to be throwbacks to a bygone era. The default linger time is defined as: at least equal to twice the maximum segment lifetime (2MSL) of the network. By default, the MSL is defined to be 120 seconds, and the value of this entry is equal to two MSLs, or 4 minutes.

4 minute maybe's in a world that has 100Gb/s transport fabrics is just ridiculous. Indeed, time-outs measured in whole seconds is pretty damn stupid. With all the different delays, time-outs, back-off algorithms and shit involved in every individual action in tcpip, it's no wonder that the HPC guys are going directly to the Ethernet layer to get throughput.

[Anonymous Monk]

Oh wow, that reminds me of some NT networking course circa 2000 , where a popular question was something how long after a machine disconnects from(as?) a domain controller does its netbiosname (lmhosts entry?) live in the cache before it finally expires, with the answer being 45 minutes, because some clever calculation using multiples of 8 ... completely stupid

[BrowserUk]

Indeed. But it is easy to forget that both these protocols were devised at a time when 4Mb/s was fast and 300 baud quite normal. In those circumstances, last time outs measured in minutes probably made sense.

The problem isn't that there was anything specifically wrong with the original formulations of those defaults. More that they have never been brought into line with modern hardware as time has gone on. They are only defaults, but that does means that they tend to get used by default.

Maybe IPv6 has corrected some of these archaic leftovers?