Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Delaying NFS writes

by ZlR (Chaplain)
on Dec 10, 2010 at 13:52 UTC ( #876460=perlquestion: print w/ replies, xml ) Need Help??
ZlR has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it. I tried different things and ended with the following code :

$| = 1 ; my $fh ; for my $master (1..100) { for my $fast (1..50000) { unless ( print $fh "PRINT OK $master $fast\n" ) { my $now = localtime ; print "Failed $it $master $fast at ", $now, " +\n" ; close $fh ; sleep 1 ; my $ok = 0 ; while ($ok == 0) { if (open $fh, ">>", "/mnt/fs_1/ttt.$it +.txt") { print $fh "PRINT OK $master $f +ast\n" ; $ok = 1 ; } else { my $now = localtime ; print "Failed delayed write fo +r $it $master $fast ", $now, "\n" ; sleep 1 ; } } } }}

I also tried without closing the filehandle, just repeating the if (print "...) until it's ok, but in both versions i still end up losing some data ! That is, it seems there's a delay between the moment the nfs folder becomes unavailable and the moment the OS (and my script) become aware of it.

Is there a specific way to handle writes to nfs folders ?

Thanks for your advice !

Comment on Delaying NFS writes
Select or Download Code
Re: Delaying NFS writes
by Generoso (Vicar) on Dec 10, 2010 at 15:02 UTC

    Just an idea, why donít you write to a local file in temp and at the end copy it over?

      That would report the problem on the way "copy" deals with stale nfs ! I'm actually trying to understand how i can cope with this myself ;)
Re: Delaying NFS writes
by JavaFan (Canon) on Dec 10, 2010 at 16:03 UTC
    Have you tried flushing the filehandle after each write?
      I don't know how to do that. I thought $| = 1 would force flushing.
Re: Delaying NFS writes
by afoken (Parson) on Dec 11, 2010 at 12:22 UTC
    I'm trying to do consistent writes to a NFS folder that can go offline for maybe 30 seconds while i'm writing in it.

    Why does the NFS server go down? How can you avoid that? Is it a planned downtime? If so, can you arrange to be notified automatically before and after the downtime?

    I think you should not try to work around such problems in your application. Think about different solutions.

    rsync could be useful: You write to a local file, and tell rsync to sync it to the storage server. If the connection breaks, you simply restart rsync until the entire file is transfered.

    Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm.

    Consider using a simple ssh connection (hoping that that won't go down as often as NFS):

    #!/usr/bin/perl use strict; use warnings; open my $out,'| ssh server cat ">>" /tmp/log.txt' or die "open failed: + $!"; print $out "hello world\n" for 1..10; close $out;

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      Well, thanks for these answsers, but the real setup is one of an NFS service failing over to another let's say cluster node, where it will be started over the same IP... It's expected behaviour that it happens at the worst possilbe time ;)

      Also check the NFS mount options: The NFS client code should be able to block your write attempt until the NFS server goes online again. This will make your programm stall for that time, but should do no other harm.

      I'll have to check what options I have, but i'd expected default mount options to behave like that, ie stall but not lose data :/

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://876460]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (8)
As of 2014-11-27 16:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My preferred Perl binaries come from:














    Results (186 votes), past polls