nested <FILE> read returns undefined?

[argv]

It's hard to search for possible answers to this, though I've ried exhaustively. It's particularly curious that I've never run into it before, as it seems like it'd be pretty common. In essence, the second read from the FILE handle doesn't work. Why?

open FILE, "foo" || die "can't open file"; # assume -T foo

for my $line (<FILE>) {
    my $x = <FILE>;
    if (!defined($x)) {
        print "couldn't read from filehandle.\n";
    }
}

Assume that foo is an arbitrarily large text file.

[tirwhan]

I'm not sure what you're trying to accomplish here, nor why you think it would be a common thing to do. Anyway, the reason it doesn't work is the reason why you shouldn't normally use for for reading from files (especially large ones), it slurps the whole file into a list in memory first before processing it. So by the time you try to assign to $x you're already at the end of the file and naturally can't read any further. Try using while instead of for.

[argv]

FWIW, I suspected the control loop operator was probably the culprit, but I only changed the SECOND for loop to use while. (In my code, there's another loop. The example I gave was seriously trimmed back to illustrate the point.) It was a fluke that I happened not to bother changing the first control loop, and only the second.

Why I did it is simple: I'm reading a file that contains a formatted data set in ascii. If one of the records I'm reading doesn't start with the correct pattern, I skip ahead (in the second loop), looking for the next record.

I feel I'm pretty familiar with the ORA perl book, and nowhere did I ever see anything on for() reading the entire file into memory. Granted, I'm not saying it's not there. I just haven't seen it, and I checked the index everywhere for references to file handles, reading, and so on. It wouldn't occur to me to check for for.

Scaling up my soap box...
IMHO, whether for reads a whole file or not seems to me to be an artifact of implementation that should not affect the logic of the program. If perl feels it can be more efficient this way, then fine--it can do so--but it's a bug to have the kind of effect it has on programs such as the one I illustrated. Indeed, Duff has even pointed out that Perl 6 will "do the right thing", so I probably needn't even point out that the current behavior should be considered a bug. But my minor flame on the subject is what keeps me from getting into worse trouble elsewhere.

[rjray]

It's not a matter of for reading the whole file or not. The for expression has a list-context within the parens, and in a list context <> reads the entire file.

[tirwhan]

Why I did it...I skip ahead (in the second loop), looking for the next record.

I may be missing something (you haven't said what the second loop actually does, so I can't be sure), but couldn't this be better handled with next ?

Regarding the soap box, as rjray pointed out, the important part here is the list context. I don't have the camel on me at the moment, but the behaviour of filehandles in list context is documented in perldoc perlintro as well as perldoc perltrap.

[duff]

Because you've used a for loop, the first instance of <FILE> reads the entire file into memory (the place where you've put the read is in list context and perl slurps in list context). The second read occurs at the end of the file, so you get undef. If you really want to do this, use a while loop instead:

while (my $line = <FILE>) {
   my $x = <FILE>;
   # ...
}

The conditional of a while is in scalar context, so only one line at a time will be read.

It's interesting to note that perl6 will change this common idiom such that a for loop does the Right Thing. (for those interested, it would look something like this: for =$filehandle -> $line { my $x = =$file_handle; ... } or when you're going to do the same as while (<>) { ... } in perl5, it's for =<> { ... } in perl6.)

[NetWallah]

Your code :

  open FILE, "foo" || die "can't open file";

will not die when you need it to. "||" has a high precedence, so the "die" will never happen as long as the file name evaluates to TRUE (which will ALWAYS be true for a non-null file name).

It can be fixed either by using parens, or by using the or operator instead of "||", since "or" has a very low precendence.

See perlop, or a multitude of posts such as this one on this subject.

[strat]

1. your error handling doesn't work:

open FILE, "foo" || die "can't open file"; # assume -T foo

groups like

open FILE, ("foo" || die ...)

either use parantheses, or just or (or both):

open( FILE, $file) or die "Error: can't open file '$file': $!\n";

2. better use while( my $line = <FILE> ) instead of the for my $line (<FILE>) (already mentioned) because while works in scalar context (i.e. it reads one line and then goes through the loop body, and then the next), whereas for uses listcontext (i.e. it reads the whole file into memory and then loops over the content which uses by far more RAM)

3. since FILE-handles are global, there may be the same problem like with each global variable:

open( FH, $file1 ) or die $!;
while( my $line1 = <FH> ) {
  print "$file1: $line";
  open( FH, $file2 ) or die ...;
  while( my $line2 = <FH> ) {
    print "$file2: $line2";
  }
  close( FH );
} # while
close( FH );
<code>
<p>gives you an error because the first <code>close( FH );

closes the filehandle so you can't continue reading in the outer loop. So better use lexical filehandles, because with them you don't need to check the whole script if this filehandle name is already used and active:

open( my $FILE, $file )
  or die "Error in reading file '$file': $!\n";
while( my $line = <$FILE> ) {
  # $line must be defined now, e.g.
  chomp( $line );
  print "$.: $line\n";
} # while
close( $FILE );