Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

perl -i -pe ... with eof() testing

by Anonyrnous Monk (Hermit)
on Feb 08, 2011 at 18:38 UTC ( #887026=perlquestion: print w/replies, xml ) Need Help??
Anonyrnous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Occasionally, I get the opportunity to impress collegues with an elegant one-liner in our beloved language. This time, though, the advocacy didn't work out too well so far... ;)

The task is to go through a number of config files and fix up a certain entry if it exists, or else (if the entry doesn't exist) append it to one of the files (last file processed is fine, but so is any other).

It's important the entry does not exist in multiple files as a result of the fixing, and it's preferable to leave it where it was, in case it was found. It can be assumed the entry exists exactly only one or zero times in the original input files. Also, files should be edited in place.

Somewhat simplified:

Case 1 — entry "opt=foo" already exists, fix it with s/opt=.*/opt=bar/:

input: desired output: file1: file1: line1 line1 opt=foo opt=bar line3 line3 file2: file2: line1 line1 line2 line2 line3 line3

Case 2 — entry doesn't exist, append "opt=bar" to last file:

input: desired output: file1: file1: line1 line1 line2 line2 line3 line3 file2: file2: line1 line1 line2 line2 line3 line3 opt=bar

The first idea was to use an END block to conditionally output the last line

$ perl -i -pe '$f||=s/opt=.*/opt=bar/; END { print "opt=bar\n" if !$f +}' file1 file2

This doesn't work, however, because the print goes to stdout, instead of the file being edited in-place. (print ARGV ... doesn't work either; $! says "Bad file descriptor", presumably because ARGV is already closed at that point.)

So, next idea was to use eof to be able to move the test into the the loop implied by -p.  However, while this in principle works fine with eof (which is true at the end of every file), it doesn't work with eof(), which should be true only once after all files have been processed — which is what would really be needed to avoid multiple insertions of the entry in question.

Test script:

#!/bin/bash #echo -e "line1\nopt=foo\nline3" >file1 # case 1 echo -e "line1\nline2\nline3" >file1 # case 2 echo -e "line1\nline2\nline3" >file2 echo --- cat before --- cat file1 file2 echo --- perl output to stdout --- perl -i -pe '$f||=s/opt=.*/opt=bar/; $_.="opt=bar\n" if eof() && !$f' +file1 file2 echo --- cat after --- cat file1 file2


--- cat before --- line1 line2 line3 line1 line2 line3 --- perl output to stdout --- line3 opt=bar --- cat after --- line1 line2 line3 line1 line2

Desired output:

--- cat before --- line1 line2 line3 line1 line2 line3 --- perl output to stdout --- --- cat after --- line1 line2 line3 line1 line2 line3 opt=bar

Interestingly, not even case 1 (where the eof() wouldn't really be needed) works. Here, the last line ("line3") is also being written to stdout instead of to the file, in case eof() is being tested...

Can anyone explain what's going on here?  Is this behavior documented somewhere?  (Of course, there are many more or less clumsy ways to solve this another way, but I'm primarily interested in why this approach doesn't do what I'd expected.)

Update: Solution: in cases like these, replace eof() with !@ARGV && eof, which doesn't have the undesirable side effect...

Replies are listed 'Best First'.
Re: perl -i -pe ... with eof() testing
by ikegami (Pope) on Feb 08, 2011 at 19:43 UTC

    eof (any form) performs a read if there's nothing in the handle's buffer. If you're at the end of the last file in @ARGV during a read of ARGV, ARGV moves on to the next file. The file to which you want to add is now closed.

    Based on black box testing, reading from ARGV (include when done via eof()) under -i is pretty close to

    #XXX Doesn't handle empty @ARGV sub fill_argv { if (!eof(ARGV)) { return 1; } while (@ARGV) { $ARGV = shift(@ARGV); if (!open(ARGV, '<', $ARGV)) { warn("Can't open $ARGV: $!"); next; } if (!unlink($ARGV)) { warn("Can't remove $ARGV: $!, skipping file.\n"); next; } if (!open(ARGVOUT, '>', $ARGV)) { warn(???); next; } select(ARGVOUT); return 1; } $ARGV = undef; close(ARGVOUT); select(STDOUT); return 1; }

    OT trivia: Note how it always returns success since it always handles errors itself.

    Update: Added expansion of reading from ARGV under -i. And then did some tweaks.

      This is what I figured, too (kind of). But why does it then "work" with the eof form?  This produces

      --- cat before --- line1 line2 line3 line1 line2 line3 --- script output to stdout --- --- cat after --- line1 line2 line3 opt=bar line1 line2 line3 opt=bar

      (I say "work" in quotes because it's not what is needed here, but what I would've expected.)

      Also, perl -MO=Deparse ... doesn't highlight any difference, except the eof vs. eof() itself.

        The only way (eof) can check for per-file eof as documented would be if it read from the real file handle instead of the magical handle. The magical handle doesn't move to the next file because it's not used.

Re: perl -i -pe ... with eof() testing
by hsinclai (Deacon) on Feb 08, 2011 at 20:01 UTC
    As an alternate approach you could craft a one-liner using Config::Simple , that module has most of what you want I think.

      Config::Simple is certainly a useful module in other contexts, and I'm generally all for using modules, if appropriate.  For the task at hand, however, it wouldn't really provide any benefits.

      For one, it doesn't properly parse the config format in question (no .ini or similar, but some custom format for an ancient Fortran program), so using a regex substitution is much easier here.

      Also, it wouldn't free me from checking for the existence of the respective option in all of the configs, and using a corresponding flag variable, etc.

      Finally, it's not a core module, and installing it on every host the fixup script is supposed to run on would only create unnecessary administrative overhead...  But thanks for the response anyway.

        >> it doesn't properly parse the config format in question
        You are right, it doesn't handle your particular format .. suggestion retracted!

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://887026]
Approved by FalseVinylShrub
Front-paged by ww
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (8)
As of 2017-09-25 11:41 GMT
Find Nodes?
    Voting Booth?
    During the recent solar eclipse, I:

    Results (279 votes). Check out past polls.