Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: Testing for readdir failure

by Bloodnok (Vicar)
on Apr 22, 2013 at 12:56 UTC ( #1029861=note: print w/ replies, xml ) Need Help??


in reply to Testing for readdir failure

Have you tried use autodie; ?

Just a thought ...

A user level that continues to overstate my experience :-))


Comment on Re: Testing for readdir failure
Download Code
Re^2: Testing for readdir failure
by Bob Cook (Acolyte) on Apr 22, 2013 at 17:03 UTC
    Many thanks to all of you for your thoughtful replies.

    First of all, after reading some of the replies, it became clear that I missed making one point: When the readdir fails to deliver the entire contents of the directory, there is NO warning, nor any other message, issued by Perl. Thus, I don't believe that trapping warnings, nor the warnings pragma, will help.

    I will try $!, and testing for the value of closedir, as well as testing for a "false" value of readdir (though that wouldn't work for readdir in a scalar context) and report back what I find.

    I downloaded IO::Dirent and looked at its C code. I am a complete novice at C, but it looks to me like it uses C's readdir, whose documentation seems to say that it also returns the same thing for both end-of-directory and an error. (There is a different function named readdir_r which can indicate an error, but that's not what IO::Dirent uses.) It occurs to me that the question I'm really trying to answer is: Exactly what does readdir do when it encounters a problem while reading the directory, and does it do anything that the program can recognize?

    The suggestion to stat the directory and verify that the link count matches the number of file names returned is a good one, but I can't use it because those two don't always match in the file system I'm concerned with (called AFS).

    Finally, as many have pointed out, testing any of this is clearly a problem.

      When the readdir fails to deliver the entire contents of the directory, there is NO warning, nor any other message, issued by Perl. Thus, I don't believe that trapping warnings, nor the warnings pragma, will help.

      It may be right, but if you don't try, you'll never know. Especially, if you don't turn on warnings with the warnings pragma, you're not gonna get those warnings. In my tests, when I tried to readdir on a close dir, I did not get any message unless I explicitly added the 'or die "$!"' clause to my readdir instruction.

      Consider this:

      $ perl -e  '@c=readdir DIR ; print "@c\n";'

      No warning, nothing. Compare to this;

      $ perl -w -e '@c=readdir DIR ; print "@c\n";' Name "main::DIR" used only once: possible typo at -e line 1. readdir() attempted on invalid dirhandle DIR at -e line 1.

      Turning on the warnings with the -w command line option gave me two warnings that might explain where my problem is. Or consider this:

      $ perl -e '@c=readdir DIR or print "$!"; print "@c\n";' Bad file descriptor

        Wow, I had no idea that warnings had different "levels" and that by default, some weren't issued. I don't think I want to enable ALL warnings, but the io bunch looks appropriate.

        Thanks for the suggestion.

      > it became clear that I missed making one point: When the readdir fails to deliver the entire contents of the directory, there is NO warning, nor any other message, issued by Perl.

      It also became clear that you never clarified what exactly fails.

      How do you expect us to reproduce and catch unidentified errors?

      Are you even sure these are errors and not just race conditions because the content of a directory changed in the meantime?

      Cheers Rolf

      ( addicted to the Perl Programming Language)

        > It also became clear that you never clarified what exactly fails.

        The program runs weekly. Many weeks, it "fails" on a small percentage of directories. Those directories were last changed years ago. And yet, on this run, according to the results of readdir, there were some things missing that were present on previous runs, AND are present when I go to look at them after the failure. (And by the way, each week it fails on a different small subset of directories.)

        > How do you expect us to reproduce and catch unidentified errors?

        I don't know how to reproduce it. That's a real problem. I was looking for possible side effects from readdir that the program could check for. And some of the suggestions have been quite helpful, including yours.

        > Are you even sure these are errors and not just race conditions because the content of a directory changed in the meantime?

        I hope the first part of this text above answers this. Most of the directories haven't changed.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1029861]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (10)
As of 2014-12-22 02:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (110 votes), past polls