Prepending to a file

[electronicMacks]

A “simple” question:

Is there any elegant way to preprend data to a file in *nix with Perl? I can easily come up with several hacks (copying the file, slurping the file etc...) but I really hope there’s a better way.

Humbly,
electronicMacks

[chipmunk]

This question is answered in the FAQ: How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file?

I found it by searching the site for 'append beginning'.

[Adam]

No.
Files are just a stream of 1s and 0s recorded to disk. The only way to edit a file is to read the file into memory, edit it, and re-write it... or to remove the EOF marker and append more data to the end.

[electronicMacks]

That's what I suspected, but I was hoping the monks and saints here could find a loophole past that....

[MeowChow]

Sure, if you want to manipulate the FAT32/MFT, or inode tables...

Not very cross-platform though :)

[extremely]

The easiest way to prepend is to move the file to backup, write new file with new data, then stream the backup file onto the end of the new data file.

--
$you = new YOU;
honk() if $you->love(perl)

[dkubb]

One way that I can see how to do this is with the following subroutine called prepend_file():

#!/usr/bin/perl -w

use strict;
use IO::File;

use constant FILE        => 'test.txt';
use constant DATA        => "this should be the first line\n";
use constant BUFFER_SIZE => '8096';

my $fh = prepend_file(FILE, DATA, BUFFER_SIZE);

while(my $line = <$fh>) {
  print $line;
}

sub prepend_file {
  my $file        = shift;
  my $data        = shift;
  my $buffer_size = shift;

  #Open a temporary and source file handle
  my $temp_fh = IO::File->new_tmpfile
    or die "Could not open a temporary file: $!";

  my $fh= IO::File->new($file, O_RDWR)
    or die "Could not open file ", FILE, ": $!";

  #Write the first bit of data
  $temp_fh->syswrite($data);

  #Copy all the $data from the $fh to the temp file handle
  $temp_fh->syswrite($data) while $fh->sysread($data, $buffer_size);

  $temp_fh->sysseek(0, 0);
  $fh->sysseek(0, 0);

  #Write out the new file from the temporary file handle
  $fh->syswrite($data) while $temp_fh->sysread($data, $buffer_size);

  #could return anything here, I just chose the file handle just
  #in case we needed to use it for something.
  return $fh->sysseek(0, 0) && $fh;
}

__END__

It uses IO::File's new_tmpfile() method to create a temporary file. You then only have to deal with the single filehandle, and IO::File takes care of throwing away the temp file when you're done. I wanted to make sure it could handle most sizes of files, even those that exceed available memory, this is why I used a temporary file and not just memory/slurping.

[ibm1620]

Instead of

  or die "Could not open file ", FILE, ": $!";

don't you want

  or die "Could not open file ", $file, ": $!";

[sierrathedog04]

A now-defunct website that I found on Google offers solutions which you can run from within Perl by using the backtick operator. These solutions depend upon the OS within which Perl runs.

Given a textfile, file1, one may wish to prepend or insert an external file, fileT, to the top of it before processing the file. Normally, this should be done from the Unix or DOS shell before passing file1 on to sed (MS-DOS 5.0 or lower needs 3 commands to do this; for DOS 6.0 or higher, the MOVE command is available):

      copy fileT+file1 temp                   # MS-DOS command 1
      echo Y | copy temp file1                # MS-DOS command 2
      del temp                                # MS-DOS command 3
      cat fileT file1 >temp; mv temp file1    # Unix commands

UPON FURTHER CONSIDERATION...I was wrong. The UNIX cat solution above is probably simply reading the file into memory as text and writing it out again.

The DOS copy fileT+file1 temp only seems to work with text files, not with binary ones. Therefore it also is probably simply copying a file into memory as text and writing it out again.

Therefore the correct answer is probably the one given earlier in this thread. You cannot do it without complicated manipulation of file pointers. Maybe you cannot do it at all. And there is no ready-made command in Perl or the DOS or UNIX shells to do it.

[baku]

The DOS 'copy' command has a -b option to act upon files in 'binary mode,' analogous to Perl's binmode; and depending upon your 'cat' implementation, the fileT and file1 are most likely read through memory in chunks (4K? I think? --something about that being the optimal size for disc transactions...), so that should be a valid way to do this without bring the whole file into RAM. e.g.

[dos-prepend.bat]

  copy /b prepend+original newfile
  if errorlevel 1 goto abort
  copy /y newfile original
  if errorlevel 1 goto abort
  del newfile
  del prepend
abort: rem end

# vs.

[unix-prepend]
#!/bin/sh
  cat prepend original > newfile && (
    mv -f newfile original;
    rm -f prepend )

Of course, dkubb's subroutine is a much better way to cope with this situation.