Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Large file processing

by arivu198314 (Sexton)
on Oct 01, 2010 at 03:44 UTC ( [id://862898]=perlquestion: print w/replies, xml ) Need Help??

arivu198314 has asked for the wisdom of the Perl Monks concerning the following question:

I have a file with 50,000 lines of find and replace string.

for example
/Test/Sample/
/A/X/
...

Now i want to process the file with input file.

I have tried with usual method, it takes more than 1 hour.

Please advice.

|Arivu|

Replies are listed 'Best First'.
Re: Large file processing
by GrandFather (Saint) on Oct 01, 2010 at 04:06 UTC

    Show us your code. For a standard sort of "read the file line by line, edit each line and write it out" I would expect a modest size file such as you imply to be processed in a matter of seconds.

    True laziness is hard work
      Indeed, and even faster if the entire file is loaded into memory at once, processed, and dumped.
        My code is here.
        #Usage: <Support-File> <Input-File> undef $/; open(TEXT, $ARGV[0]) or die $!; my $text = <TEXT>; close TEXT; open(XML, $ARGV[1]) or die $!; my $xml = <XML>; close XML; while($text =~ m/\/(.+)\/([^\/]+)\/[^\/]+$/mgi) { my $FindWord = $1; my $ReplaceWord = $2; $xml =~ s/(>[^>]*\b)\Q$FindWord\E(\b[^>]*<)/$1$ReplaceWord$2/gi; } open(OUTXML, ">$ARGV[1]") or die $!; print OUTXML $xml; close OUTXML;

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://862898]
Approved by ikegami
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (7)
As of 2024-04-16 17:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found