Replacing a string in a file

[Avi]

Hey Monks,
I wanted to see what you guys can come up with for a way to simple change a string in a file.

I know, a very basic question, but I'm look for a better and more secure (in terms of writing -- would definitely not want this file to become corrupt or overwritten by simultaneous uses) way than what I have now.

Right now, I open up, read it's contents, close it, open up a temp file, write the new contents with the string replaced, close it, delete the original file and move the temp file to the original. Horrible, huh? There's got to be a better way than this. I just can't think of it! :( If there's a module out there to do this as well, let me know.

Thanks in advance.

Avi

[graff]

The suggestion about file locking above is pertinent, but perhaps not adequate, because there may be some "loopholes" when trying to lock the file that you're actually editing (see the article cited in the code commentary below).

Better to use a "semaphore" file -- one that has no content itself, but serves only to control access to the file that you intend to edit. A semaphore module that I've used to good effect is included below, together with a snippet to show its usage.

Apart from preventing simultaneous edits on a single file, there are other common sources of "corruption" that you need to watch out for:

• the disk you're writing on runs out of space before the output file is complete
• the edit attempt turns out to have no effect -- the input and output are identical, even though they shouldn't be
• the edit applied by the script turns out to have "unintended" side effects

The first point is just a matter of error checking when writing and closing the file, and/or comparing the input and output file sizes (or the tails of the two files). The second can be solved by running "diff old.file new.file" in a back-tick (qx) operator. The third is rather tricky and application-dependent (good luck with that, if it happens to be an issue for you).

All in all, to be careful, you want to stick with your current basic approach (with file locking and validation added): (a) get a lock on the file, (b) read it and create an edited version of it, (c) validate the edited version, (d) rename the edited version to replace the original, (e) release the lock. As a rule, it's good to keep the duration of the locking to a minimum, but if I understand your task, there's no reducing the five steps mentioned (a-e).

Here's the semaphore-file module that I use in a unix/solaris environment -- haven't tried it on ms-windows yet (update:... now that I think of it, I sort of recall testing this on a couple ms-windows versions (98, 2000)):

package Semfile;

use strict;
use FileHandle;
use Carp;
 
# copied (and documented) by Dave Graff from 
# The Perl Journal, issue #23 (Vol.6 No.1):
# "Resource Locking with Semaphore Files"
# by Sean M. Burke.
# (removed obsolete url for that article -- see update below)

# Create and manage semaphore files, which will guarantee that
# simultaneous processes competing for a single resource do not
# collide when using that resource.

# The semaphore (locked) file has no content -- it is simply the
# thing to check, and lock if it's available, before using/altering
# the actual shared resource.

sub new {
    my $class = shift(@_);
    my $filespec = shift(@_) or Carp::croak("What filespec?");

    my $fh = new FileHandle;
    $fh->open( ">$filespec" )
        or Carp::croak("Can't open semaphore file $filespec: $!");
    chmod 0664, $filespec;      # make it ug+rw
    use Fcntl 'LOCK_EX';
    flock $fh, LOCK_EX;
    return bless {'fh' => $fh}, ref($class) || $class;
}

sub release {
    undef $_[0]{'fh'};
}
1; # End of module

And here is how I would normally use it:

use Semfile;
...
# All instances of this process (and any other perl process that 
# uses the same shared resource) will use the same semaphore file:

my $lock = Semfile->new( "my_semafore.filename" );

# When that call returns, I have the lock; I'll hold it
# till I finish doing sensitive tasks, like:
#  - read the shared resource
#  - alter and save the value of that resource
#  - confirm that I haven't botched it
# then:

$lock->release;

(Update -- Aug. 8 2005: had to remove the obsolete url, because the TPJ article is not reachable that way anymore. You can still go to www.tpj.com/search/ and look for "semaphore file Sean Burke" to find the reference. If you subscribe to TPJ (highly recommended), you can read the article.)

(Another update -- Nov. 7, 2006: www.tpj.com is gone. Sincere thanks to davebaker for finding that article again: http://interglacial.com/~sburke/tpj/as_html/tpj23.html)

[davebaker]

Nov. 7, 2006

I found Sean M. Burke's article about semaphore locking files at this URL: http://interglacial.com/~sburke/tpj/as_html/tpj23.html .

(This avoids having to do a ddj.com search; what a mess... I couldn't find Sean's article that way when I searched today. There's no more www.tpj.com/search/ ... what were the Dr. Dobb's webmasters thinking when they yanked tpj.com?)

[vnpandey]

Dear Monk, Yes..you are right.. This is a more comprehensive way..

Thanks!
pandey

[Aristotle]

Hmm.. the simplest form is something like

$ perl -pi.bak -e's/foo/bar/' file1 file2 ..

Look for the documentation of the -p and -i switches in perldoc perlrun. In what context are you trying to do this?

Makeshifts last the longest.

[artist]

#!perl -pi.bak
#replace.pl
s/currentString/replacementString/sg;

usage: replace.pl myfile
It should make a backup file myfile.bak which would be your original.

artist

[bm]

Why bother with the temp file? You can open a file for reading and writing at the same time. This will be quicker and require much less code.

[perlplexer]

The amount of code required to accomplish this task depends greately on what kind of file he's dealing with. Using one file instead of two is not always simpler/easier to code.


If files that you're dealing with are small; i.e., config files, etc., it's probably easier to slurp them into memory, perform modifications, truncate the file, and then write data back -- no temp files needed.

With larger files and variable-length records, it's easier to work with temp files. If you attempt to update a variable-length record file without having a temp file, the code will get very ugly in no time. Sometimes, however, you just have to bite the bullet and do it; i.e., when you deal with huge files and you can't spare space for a temp file of the same size.

If you're working with large fixed-length record files, it is almost always better and more elegant, in my opinion, to work with the original file without creating temp files.


--perlplexer

[Avi]

For opening the file for reading writing, how do you go about doing this? Do you have sample code? It is for a config file which is very small, so this solution would be great.

Thanks,
Avi

[perlplexer]

Here's how I would approach this (not tested)

use strict;
use Fcntl ':flock';

my $iniFile = 'config.ini';
if (open my $ini, "+<$iniFile"){
  if (flock $ini, LOCK_EX){
    my @data = <$ini>;
    # Work with @data -- delete lines, update lines, etc.
    if (truncate($ini, 0) and seek($ini, 0, 0)){
      print "Can't write to $iniFile : $!\n"
                        unless print $ini @data;
    }else{
      print "Can't truncate/seek : $!\n";
    }
  }else{
    print "Can't lock : $!\n";
  }
  close $ini;
}else{
  print "Can't open $iniFile : $!\n";
}

--perlplexer

[vnpandey]

I also presume that to give your program exclusive access to the file you have also locked it using the flock command.
flock( NAMEGET, 2 );
You can use the value 2 to lock and change to 8 after you have done the modification to remove the access.

[simonm]

Unsurprisingly, there's already a module on CPAN to handle the temp-file maneuvers:

use IO::AtomicFile;

# Write a temp file, and have it install itself when closed:
my $FH = IO::AtomicFile->open("bar.dat", "w");
print $FH "Hello!\n";
$FH->close || die "couldn't install atomic file: $!";

[Skeeve]

If I want to be sure that my files don't get corrupted by a second process writing to it, I usually do it the way I described in Re: What is the best way to lock a counter file?.

Maybe this will help you.

[Mago]

sub altLine {
    my ($arqName, $param, $value) = @_;
    my $arqTmp = $$ . '.tmp';
    my $arqLine;
    my $arqParam;
    my @aux;
    open(ARQ, "< $arqName");
    open(TEMP, "> $arqTmp");
    while ($arqLine = <ARQ>) {
          chomp $arqLine;
          @aux = split('=', $arqLine);
          $arqParam = $aux[0];
      $arqParam = &Trim($arqParam);
          if ($arqParam eq $param) {
             print TEMP "$param = $value \n";
          } else {
                 print TEMP "$arqLine\n";
          }
    }
    close ARQ;
    close TEMP;
    rename($arqTmp, $arqName);
    return;
}