Delaying NFS writes

[ZlR]

Hello Monks,

I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it. I tried different things and ended with the following code :

$| = 1 ; 
my $fh  ;
    for my $master (1..100) {
        for my $fast (1..50000) {
                unless ( print $fh "PRINT OK $master $fast\n" ) {
                        my $now = localtime ;
                        print "Failed $it $master $fast at ", $now,  "
+\n" ;
                        close $fh ;
                        sleep 1 ;
                        my $ok = 0 ;
                        while ($ok == 0)  {
                                if (open $fh, ">>", "/mnt/fs_1/ttt.$it
+.txt") {
                                        print $fh "PRINT OK $master $f
+ast\n" ;
                                        $ok = 1 ;
                                } else {
                                        my $now = localtime ;
                                        print "Failed delayed write fo
+r $it $master $fast ", $now,  "\n" ;
                                        sleep 1 ;
                                }
                        }
                }
   }}

I also tried without closing the filehandle, just repeating the if (print "...) until it's ok, but in both versions i still end up losing some data ! That is, it seems there's a delay between the moment the nfs folder becomes unavailable and the moment the OS (and my script) become aware of it.

Is there a specific way to handle writes to nfs folders ?

Thanks for your advice !

[Generoso]

Just an idea, why don’t you write to a local file in temp and at the end copy it over?

[ZlR]

That would report the problem on the way "copy" deals with stale nfs ! I'm actually trying to understand how i can cope with this myself ;)

[JavaFan]

Have you tried flushing the filehandle after each write?

[ZlR]

I don't know how to do that. I thought $| = 1 would force flushing.

[afoken]

I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it.

Why does the NFS server go down? How can you avoid that? Is it a planned downtime? If so, can you arrange to be notified automatically before and after the downtime?

I think you should not try to work around such problems in your application. Think about different solutions.

rsync could be useful: You write to a local file, and tell rsync to sync it to the storage server. If the connection breaks, you simply restart rsync until the entire file is transfered.

Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm.

Consider using a simple ssh connection (hoping that that won't go down as often as NFS):

#!/usr/bin/perl
use strict;
use warnings;

open my $out,'| ssh server cat ">>" /tmp/log.txt' or die "open failed:
+ $!";
print $out "hello world\n" for 1..10;
close $out;

Alexander

[ZlR]

Well, thanks for these answsers, but the real setup is one of an NFS service failing over to another let's say cluster node, where it will be started over the same IP... It's expected behaviour that it happens at the worst possilbe time ;)

Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm.

I'll have to check what options I have, but i'd expected default mount options to behave like that, ie stall but not lose data :/