Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re: What is the best way to remove(delete) specific lines from a very large file?

by tye (Sage)
on Dec 03, 2001 at 22:03 UTC ( #129124=note: print w/replies, xml ) Need Help??


in reply to What is the best way to remove(delete) specific lines from a very large file?

If the file is so big that you don't have enough disk space available for both the "before" and "after" versions, then you can go to quite a bit of extra work to save disk space:
my $fileName= "toBeTrimmed.txt"; open( FILE, "+< $fileName" ) or die "Can't update $fileName: $!\n"; my $readPos= tell(FILE); my $writePos; while( <FILE> ) { if( wantLine($_) ) { $readPos= tell(FILE) or die "Can't tell after read: $!\n"; if( defined($writePos) ) { seek( FILE, $writePos, 0 ) or die "Can't seek to write: $!\n"; print FILE $_ or die "Failed writing to file: $!\n"; $writePos= tell(FILE) or die "Can't tell after write: $!\n"; seek( FILE, $readPos, 0 ) or die "Can't seek to read: $!\n"; } } elsif( ! defined($writePos) ) { $writePos= $readPos; } } if( defined($writePos) ) { truncate( FILE, $writePos ) or warn "Can't truncate: $!\n"; }
But be sure to test this code on a subset of your file and make a backup of the file beforehand as a failure of this code will simply leave your file corrupted.

        - tye (but my friends call me "Tye")
  • Comment on Re: What is the best way to remove(delete) specific lines from a very large file?
  • Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://129124]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (8)
As of 2021-05-07 12:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (91 votes). Check out past polls.

    Notices?